JP5091781B2 - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and method which is capable of extending a dynamic range through one photographing, without producing a composite image from a plurality of images taken under different exposure settings. <P>SOLUTION: In this imaging apparatus, an extended D-range extrapolation section 50 in a YUV converting section 36, including: a brightness level judging section 60 which detects an output of each pixel with the RGB filter from digital image data input and judges if an output of each pixel reaches a predetermined saturation level or not; and a pixel output compensation processing section 61 in which, if an output of a pixel with a G filter is judged to be more than a predetermined saturation level, the pixel output is compensated based on an output of an adjacent unsaturated pixel having R and B filters. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method such as a digital still camera and a digital video camera, and more particularly to an imaging apparatus and an imaging method that can expand a dynamic range of a captured image.

  Compared to the dynamic range of images taken with a conventional silver salt camera using a silver salt photographic film, the dynamic range of images taken with a digital still camera or a digital video camera having a solid-state image sensor such as a CCD is extremely narrow. When the dynamic range is narrow, a phenomenon called “blackout” occurs in the dark part of the subject, and conversely, a phenomenon called “overexposure” occurs in the bright part of the subject, and the image quality deteriorates.

Therefore, in order to expand the dynamic range of an image captured by a solid-state imaging device such as a CCD, for example, the same subject is shot multiple times with different exposure amounts to obtain a plurality of images with different exposure amounts. A technique for adding these images to generate a composite image with an expanded dynamic range has been known (see, for example, Patent Document 1).
JP 2000-92378 A

  By the way, in order to expand the dynamic range, in the method of performing photographing a plurality of times while changing the exposure amount as in the above-mentioned Patent Document 1, when photographing a moving object, the subject is doubled. Therefore, the image may not be synthesized correctly.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an imaging apparatus and an imaging method capable of expanding the dynamic range by one shooting without changing the exposure amount and performing a plurality of shootings to combine images. .

In order to achieve the above object, an imaging apparatus according to claim 1, a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a front side of each pixel A pixel output detector that determines whether or not the output from any of the pixels has reached a predetermined saturation level or more with respect to the output from each pixel. And the pixel output detection means, when it is determined that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, a filter other than the specific color around the filter is arranged based on the output from the pixels, wherein the predetermined pixel output correcting means for interpolating the determined pixel output the saturation level or higher, the output of the pixel in which the specific color filter is disposed said predetermined The pixel output data, which is output from the pixel output correction processing means when the saturation level is reached or higher and is once extended from the first bit number below the predetermined saturation level to the second bit number, is again obtained. Bit compression conversion means for compressing to the first number of bits, wherein the bit compression conversion means outputs a pixel output below the saturation level rather than a compression rate for data corresponding to the pixel output in the area above the saturation level. It is characterized in that compression is performed with a reduced compression rate for data corresponding to .

The image pickup apparatus according to claim 2, wherein a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are provided on the front side of each pixel. A pixel output detecting means for determining whether or not an output from any of the pixels has reached a predetermined saturation level or more with respect to an output from each of the pixels, and the pixel output detection When it is determined by the means that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, the pixel from the pixel in which the filter of the same color as the specific color in the surrounding area is arranged based on the output, the a predetermined saturation level above the determined pixel output correcting means for interpolating the pixel output was the output of the pixel in which the specific color filter is disposed reaches beyond the predetermined saturation level If the pixel output data output from the pixel output correction processing means is expanded from the first number of bits below the predetermined saturation level to the second number of bits, the first bit again Bit compression conversion means for compressing into numbers, the bit compression conversion means for the data corresponding to the pixel output below the saturation level rather than the compression rate for the data corresponding to the pixel output in the region above the saturation level. It is characterized by compressing with a reduced compression rate .

  The imaging apparatus according to claim 3, wherein a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are provided on the front side of each pixel. A pixel output detecting means for determining whether or not an output from any of the pixels has reached a predetermined saturation level or more with respect to an output from each of the pixels, and the pixel output detection When it is determined by the means that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, the output from the pixel in which the filter other than the specific color in the surrounding area is arranged First pixel output correction processing means for interpolating the pixel output determined to be equal to or higher than the predetermined saturation level, and the pixel output detection means from the pixel in which the filter of the specific color is arranged. When it is determined that the force has reached the predetermined saturation level or higher, it is determined that the force is equal to or higher than the predetermined saturation level based on an output from a pixel in which a filter having the same color as the specific color is arranged. Based on the interpolation information respectively output from the second pixel output correction processing means for interpolating the pixel output, the first pixel output correction processing means and the second pixel output correction processing means, the predetermined saturation level Interpolation processing means for interpolating the pixel output determined as described above is provided.

  The image pickup apparatus according to claim 4 is characterized in that a unit of processing when the output from each pixel is detected by the pixel output detecting means is a size of 2 × 2 pixels in the horizontal and vertical directions.

  The image pickup apparatus according to claim 5, wherein a color filter having the same color as the specific color is provided on the front side or the rear side of the optical system so as to be freely inserted and removed, and the color filter is provided on the front side or the rear side of the optical system. Accordingly, the sensitivity to the brightness of the filter of the other color is relatively lowered with respect to the sensitivity to the brightness of the filter of the specific color.

  The imaging device according to claim 6 is characterized in that a plurality of the color filters are provided, and the plurality of color filters have different light transmittances.

  The image pickup apparatus according to claim 7, in order to select and execute the operation to perform the interpolation when the output of the pixel in which the filter of the specific color is arranged exceeds the predetermined saturation level. It is characterized by comprising the operation selection means.

The imaging apparatus according to claim 8 , wherein the output of the pixel in which the filter of the specific color is arranged is equal to or lower than the predetermined saturation level output from the interpolation processing unit when the output reaches the predetermined saturation level or higher. A bit compression conversion unit that compresses all the data once expanded from the first bit number to the second bit number into the first bit number, and the bit compression conversion unit is a region above the saturation level. The compression rate for the data corresponding to the pixel output at the saturation level or lower is made smaller than the compression rate for the data corresponding to the pixel output in, and compression is performed.

The image pickup apparatus according to claim 9 , wherein the data corresponding to the pixel output at the low luminance level below the saturation level has substantially the same value before the bit compression by the bit compression conversion unit and after the bit compression. It is characterized by using such a compression rate.

The imaging apparatus according to claim 10 , wherein a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are provided on the front side of each pixel. A pixel output detecting means for determining whether or not an output from any of the pixels has reached a predetermined saturation level or more with respect to an output from each of the pixels, and the pixel output detection When it is determined by the means that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, the output from the pixel in which the filter other than the specific color in the surrounding area is arranged based on, and a pixel output correcting means for interpolating the pixel output is determined that the above predetermined saturation level, the pixel output correcting means are output pixel is determined that the above predetermined saturation level When performing the calculation for interpolating, the calculation result is characterized in that only if larger than the original of the pixel output, the interpolation of the pixel output is determined that the above predetermined saturation level.

The imaging apparatus according to claim 11 , wherein a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are provided on the front side of each pixel. A pixel output detecting means for determining whether or not an output from any of the pixels has reached a predetermined saturation level or more with respect to an output from each of the pixels, and the pixel output detection When it is determined by the means that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, the output from the pixel in which the filter other than the specific color in the surrounding area is arranged based on, and a pixel output correcting means for interpolating the pixel output is determined that the above predetermined saturation level, the pixel output correcting means are output pixel is determined that the above predetermined saturation level When performing the calculation for interpolating, if the calculation result is smaller than the original of the pixel output is characterized in that to enlarge the range of pixels to be used for the calculation.

The imaging device according to any one of claims 12 and 15 , when expanding a range of pixels used for the calculation, also uses a pixel having a filter of the same color as the pixel having the filter of the specific color. It is characterized by.

The imaging apparatus according to claim 13 , wherein when the first and second pixel output correction processing means perform an operation for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level, the calculation result Only when the pixel output becomes larger than the original pixel output, the pixel output determined to be equal to or higher than the predetermined saturation level is interpolated.

15. The imaging apparatus according to claim 14 , wherein when the first and second pixel output correction processing units perform a calculation for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level, the calculation result is obtained. Is smaller than the original pixel output, the range of pixels used for the calculation is expanded.

The imaging method according to claim 18 , wherein a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are provided on the front side of each pixel. In the imaging method of an imaging apparatus including an imaging device in which is arranged, a determination processing step for determining whether or not an output from any pixel reaches or exceeds a predetermined saturation level with respect to an output from each pixel When the determination processing step determines that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, a filter other than the specific color around it is arranged. based on the output from the pixel, and interpolation step of interpolating the determined pixel output and the above predetermined saturation level, the output of the pixel in which the specific color filter is arranged the plants The pixel output data, which is output from the interpolation processing step when the saturation level has reached or exceeds the predetermined saturation level and expanded once from the first bit number below the predetermined saturation level to the second bit number, is output again. A bit compression conversion step of compressing to a first number of bits, wherein the bit compression conversion step generates a pixel output below the saturation level rather than a compression ratio for data corresponding to the pixel output in the region above the saturation level. It is characterized by compressing the corresponding data with a reduced compression rate .

The imaging method according to claim 19 , wherein a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are provided on the front side of each pixel. In the imaging method of an imaging apparatus including an imaging device in which is arranged, a determination processing step for determining whether or not an output from any pixel reaches or exceeds a predetermined saturation level with respect to an output from each pixel When the determination processing step determines that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, the filter of the same color as the surrounding specific color is arranged based on the output from the pixel, the predetermined and interpolation processing step of interpolating the determined pixel output saturation level or higher, the output of the pixel in which the specific color filter is arranged before The pixel output data, which is output from the interpolation processing step when reaching a predetermined saturation level or higher and once expanded from the first bit number below the predetermined saturation level to the second bit number, is again A bit compression conversion step of compressing to the first number of bits, wherein the bit compression conversion step is a pixel output below the saturation level rather than a compression rate for data corresponding to pixel output in the region above the saturation level. It is characterized in that compression is performed with a reduced compression rate for data corresponding to .

The imaging method according to claim 20 , wherein a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are provided on the front side of each pixel. In the imaging method of an imaging apparatus including an imaging device in which is arranged, a determination processing step for determining whether or not an output from any pixel reaches or exceeds a predetermined saturation level with respect to an output from each pixel And when the output from the pixel in which the filter of the specific color is disposed has reached the predetermined saturation level or higher in the determination step, the pixel in which a filter other than the specific color is disposed around it A first interpolation processing step for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level based on the output from the output, and a filter for the specific color is arranged by the determination step. When it is determined that the output from the selected pixel has reached or exceeded the predetermined saturation level, the predetermined saturation is determined based on the output from the pixel in which the filter having the same color as the specific color is disposed. A second interpolation processing step for interpolating a pixel output determined to be equal to or higher than the level, and the interpolation information output from the first interpolation processing step and the second interpolation processing step, respectively, to be equal to or higher than the predetermined saturation level And a third interpolation processing step for interpolating the pixel output determined as.

According to the inventions described in claims 1 and 18, when it is determined that the output from the pixel in which the filter of the specific color is arranged has reached a predetermined saturation level or more, the surroundings are not saturated. Based on the output from the pixel where the color filter is placed, the pixel output in the region above the saturation level is interpolated to expand the dynamic range, thereby changing the exposure amount and shooting multiple times to compose the image In addition, the dynamic range can be expanded by one shooting. In addition, by compressing the compression rate for data corresponding to pixel output below the saturation level, compared with the compression rate for data corresponding to the pixel output in the region above the saturation level, the gradation characteristics below the saturation level are improved. Can be held in.

According to the second and 19th aspects of the invention, when it is determined that the output from the pixel in which the filter of the specific color is arranged has reached a predetermined saturation level, a plurality of surrounding non-saturated Based on the output from the pixel where the filter of the same color as the specific color is placed, the pixel output in the region above the saturation level is interpolated to expand the dynamic range, thereby changing the exposure amount and shooting multiple times. Without combining the dynamic range, the dynamic range can be expanded by one shooting. In addition, by compressing the compression rate for data corresponding to pixel output below the saturation level, compared with the compression rate for data corresponding to the pixel output in the region above the saturation level, the gradation characteristics below the saturation level are improved. Can be held in.

According to the inventions of claims 3 and 20, when it is determined that the output from the pixel in which the filter of the specific color is arranged has reached a predetermined saturation level or more, other surroundings that are not saturated Based on the output from the pixel where the color filter is placed, the output of the pixel that has reached the saturation level or higher is interpolated, and a filter of the same color as the surrounding specific colors is placed. By interpolating the output of pixels that have reached the saturation level or higher based on the output from the selected pixels, and performing appropriate interpolation processing based on both of these interpolation information, the exposure range can be increased. Instead, the dynamic range can be expanded by one shooting without performing multiple shootings and synthesizing images.

  According to the fifth aspect of the present invention, by inserting a color filter of the same color as the specific color on the front side or the rear side of the optical system, the filter of another color is sensitive to the sensitivity to the luminance of the filter of the specific color. Therefore, the dynamic range can be easily further expanded.

  According to the sixth aspect of the present invention, since the plurality of color filters have different light transmittances, the magnification of the dynamic range can be easily changed.

  According to the seventh aspect of the present invention, the operation selection unit selects an operation for predictive interpolation when the output of the pixel in which the filter of the specific color is disposed reaches the predetermined saturation level or more. This can be done at the photographer's discretion.

According to the invention described in claim 8, by compressing the data corresponding to the pixel output at the saturation level or lower than the compression rate for the data corresponding to the pixel output in the region above the saturation level, the compression is performed. Good gradation can be maintained at a saturation level or lower.

According to the ninth aspect of the present invention, the data corresponding to the pixel output at the low luminance level below the saturation level has substantially the same value before the bit compression by the bit compression conversion means and after the bit compression. By using such a compression rate, it is possible to maintain good gradation at a low luminance level.

According to the inventions described in claims 10 and 13 , filters of other colors around them are arranged for an output from a pixel in which a filter of a specific color reaching a predetermined saturation level or more is arranged. When the output from the pixel is significantly small, the output from the pixel where the filter of the specific color is placed by the interpolation process can be prevented from becoming smaller than the original, thus preventing the dynamic range from becoming smaller. can do.

According to the inventions described in claims 11 and 14 , filters of other colors around them are arranged for an output from a pixel in which a filter of a specific color reaching a predetermined saturation level or more is arranged. When the output from the pixel is significantly small, it is possible to perform interpolation processing so as to be larger than the original pixel output by using the output information from the outside pixel.

According to the inventions according to claims 12 and 15 , the output information from the pixel in which the filter of the same color as that of the pixel in which the filter of the specific color is arranged is also used, thereby performing better interpolation processing. Can do.

Hereinafter, the present invention will be described based on the illustrated embodiments.
<Embodiment 1>

  FIG. 1A is a front view showing a digital still camera (hereinafter referred to as “digital camera”) as an example of an imaging apparatus according to Embodiment 1 of the present invention, and FIG. 1 (c) is a rear view thereof, and FIG. 2 is a block diagram showing an outline of a system configuration in the digital camera shown in FIGS. 1 (a), (b), and (c).

(Appearance structure of digital camera)
As shown in FIGS. 1A, 1B, and 1C, a release button (shutter button) 2, a power button 3, a shooting / playback switching dial 4 are provided on the upper surface side of the digital camera 1 according to the present embodiment. In the front (front) side of the digital camera 1, a lens barrel unit 6 having a photographing lens system 5, a strobe light emitting unit (flash) 7, and an optical viewfinder 8 are provided.

  On the back side of the digital camera 1, there are a liquid crystal monitor (LCD) 9, an eyepiece 8 a of the optical viewfinder 8, a wide-angle zoom (W) switch 10, a telephoto zoom (T) switch 11, and a menu (MENU) button 12. , A confirmation button (OK button) 13 and the like are provided. Further, a memory card storage unit 15 for storing a memory card 14 (see FIG. 2) for storing captured image data is provided inside the side surface of the digital camera 1.

(Digital camera system configuration)
As shown in FIG. 2, the digital camera 1 includes a CCD 20 serving as a solid-state imaging device on which a subject image incident through a photographing lens system 5 of a lens barrel unit 6 forms an image on a light receiving surface, and electrical signals output from the CCD 20. An analog front-end unit (hereinafter referred to as “AFE unit”) 21 that processes (analog RGB image signal) into a digital signal, a signal processing unit 22 that processes a digital signal output from the AFE unit 21, and temporarily stores data SDRAM 23 for controlling, ROM 24 for storing a control program, a motor driver 25 for driving the lens barrel unit 6 and the like.

  The lens barrel unit 6 includes a photographic lens system 5 having a zoom lens, a focus lens, and the like, an aperture unit 26, and a mechanical shutter unit 27. The drive units of the photographic lens system 5, the aperture unit 26, and the mechanical shutter unit 27 are as follows. It is driven by the motor driver 25. The motor driver 25 is driven and controlled by a drive signal from a control unit (CPU) 28 of the signal processing unit 22.

  In the CCD 20, RGB primary color filters (see FIG. 7: hereinafter referred to as “RGB filters”) are arranged on a plurality of pixels constituting the CCD 20, and electrical signals (analog RGB image signals) corresponding to the RGB three primary colors are output. The

  The AFE unit 21 is sampled by a TG (timing signal generating unit) 30 that drives the CCD 20, a CDS (correlated double sampling unit) 31 that samples an electrical signal (analog RGB image signal) output from the CCD 20, and the CDS 31. An AGC (analog gain controller) 32 that adjusts the gain of the image signal, and an A / D converter 33 that converts the image signal gain-adjusted by the AGC 32 into a digital signal (hereinafter referred to as “RAW-RGB data”) are provided. Yes.

  The signal processing unit 22 outputs a screen horizontal synchronization signal (HD) and a screen vertical synchronization signal (VD) to the TG 30 of the AFE unit 21, and an A / D conversion unit 33 of the AFE unit 21 in accordance with these synchronization signals. CCD interface (hereinafter referred to as “CCD I / F”) 34 that captures RAW-RGB data output from the CPU, a memory controller 35 that controls the SDRAM 23, and a YUV format that can display and record the captured RAW-RGB data. A YUV conversion unit 36 for converting to image data, a resizing processing unit 37 for changing the image size in accordance with the size of image data to be displayed or recorded, a display output control unit 38 for controlling display output of the image data, and an image A data compression unit 39 for recording data by JPEG formation and the like, and writing image data to the memory card 14; A digital camera based on a control program stored in the ROM 24 based on a media interface (hereinafter referred to as “media I / F”) 40 for reading image data written in the memory card 14 and operation input information from the operation unit 41. 1 is provided with a control unit (CPU) 28 for performing overall system control and the like.

  The operation unit 41 includes a release button 2, a power button 3, a shooting / playback switching dial 4, and a wide-angle zoom provided on the external surface of the digital camera 1 (see FIGS. 1A, 1B, and 1C). The switch 10, the telephoto zoom switch 11, the menu button 12, the confirm button 13, and the like, and a predetermined operation instruction signal is input to the control unit 28 by the photographer's operation.

  The SDRAM 23 stores the RAW-RGB data captured by the CCD I / F 34, and stores the YUV data (YUV format image data) converted by the YUV conversion unit 36. Further, the SDRAM 23 stores the RAW-RGB data. The compressed image data such as JPEG formation is stored.

  Note that YUV of the YUV data is luminance data (Y), color difference (difference (U) between luminance data and blue (B) component data, and difference (V) between luminance data and red (R) component data). It is a format that expresses color with information.

(Digital camera monitoring and still image shooting)
Next, the monitoring operation and still image shooting operation of the digital camera 1 will be described. In the still image shooting mode, the digital camera 1 performs a still image shooting operation while executing a monitoring operation as described below.

  First, when the photographer turns on the power button 3 and sets the photographing / playback switching dial 4 to the photographing mode, the digital camera 1 is activated in the recording mode. When the control unit 28 detects that the power button 3 is turned on and the shooting / playback switching dial 4 is set to the shooting mode, the control unit 28 outputs a control signal to the motor driver 25 to switch the lens barrel unit 6. The camera 20 is moved to a photographing position and the CCD 20, the AFE unit 21, the signal processing unit 22, the SDRAM 23, the ROM 24, the liquid crystal monitor 9 and the like are activated.

  Then, by directing the photographic lens system 5 of the lens barrel unit 6 toward the subject, a subject image incident through the photographic lens system 5 is formed on the light receiving surface of each pixel of the CCD 20. Then, an electrical signal (analog RGB image signal) corresponding to the subject image output from the CCD 20 is input to the A / D conversion unit 33 via the CDS 31 and the AGC 32, and 12 bits (bit) by the A / D conversion unit 33. To RAW-RGB data.

  This RAW-RGB data is taken into the CCD I / F 34 of the signal processing unit 22 and stored in the SDRAM 23 via the memory controller 35. The RAW-RGB data read from the SDRAM 23 is converted into YUV data (YUV signal) in a format that can be displayed by the YUV converter 36, and then the YUV data is stored in the SDRAM 23 via the memory controller 35. The

  Then, the YUV data read from the SDRAM 23 via the memory controller 35 is sent to the liquid crystal monitor (LCD) 9 via the display output control unit 38, and a photographed image (moving image) is displayed. At the time of monitoring in which a photographed image is displayed on the liquid crystal monitor (LCD) 9 described above, one frame is read out in a time of 1/30 second by thinning out the number of pixels by the CCD I / F 34.

  In this monitoring operation, the photographed image is only displayed on the liquid crystal monitor (LCD) 9 functioning as an electronic viewfinder, and the release button 2 is not yet pressed (including half-pressed).

  The photographer can confirm the photographed image by displaying the photographed image on the liquid crystal monitor (LCD) 9. In addition, it can output as a TV video signal from the display output control part 38, and can display a picked-up image (moving image) on external TV (television) via a video cable.

  The CCD I / F 34 of the signal processing unit 22 calculates an AF (automatic focus) evaluation value, an AE (automatic exposure) evaluation value, and an AWB (auto white balance) evaluation value from the captured RAW-RGB data.

  The AF evaluation value is calculated by, for example, the output integrated value of the high frequency component extraction filter or the integrated value of the luminance difference between adjacent pixels. When in the in-focus state, the edge portion of the subject is clear, so the high frequency component is the highest. By utilizing this, at the time of AF operation (at the time of focus detection operation), an AF evaluation value at each focus lens position in the photographing lens system 5 is acquired, and the point where the maximum is obtained is used as the focus detection position for AF. The action is executed.

  The AE evaluation value and the AWB evaluation value are calculated from the integrated values of the RGB values in the RAW-RGB data. For example, the screen corresponding to the light receiving surfaces of all the pixels of the CCD 20 is equally divided into 256 areas (16 horizontal divisions and 16 vertical divisions), and the RGB integration of each area is calculated.

  Then, the control unit 28 reads the calculated RGB integrated value, and in the AE process, calculates the luminance of each area of the screen and determines an appropriate exposure amount from the luminance distribution. Based on the determined exposure amount, exposure conditions (the number of electronic shutters of the CCD 20, the aperture value of the aperture unit 26, etc.) are set. In the AWB process, an AWB control value that matches the color of the light source of the subject is determined from the RGB distribution. By this AWB process, white balance is adjusted when the YUV conversion unit 36 performs conversion to YUV data. The AE process and AWB process described above are continuously performed during the monitoring.

  When the still image shooting operation in which the release button 2 is pressed (half-pressed to full-press) is started during the monitoring operation described above, the AF operation and the still image recording process that are the focus position detection operations are performed. .

  That is, when the release button 2 is pressed (half-pressed to full-pressed), the focus lens of the photographing lens system 5 is moved by a drive command from the control unit 28 to the motor driver 25, and is referred to as so-called hill-climbing AF, for example. The contrast evaluation AF operation is executed.

  When the AF (focusing) target range is the entire region from infinity to close, the focus lens of the photographing lens system 5 moves to each focus position from close to infinity, or from infinity to close, and CCDI / The control unit 28 reads out the AF evaluation value at each focus position calculated in F34. Then, the focus lens is moved to the in-focus position with the point where the AF evaluation value at each focus position is maximized as the in-focus position, and in-focus.

  Then, the AE process described above is performed, and when the exposure is completed, the mechanical shutter unit 27 is closed by a drive command from the control unit 28 to the motor driver 25, and an analog RGB image signal for a still image is output from the CCD 20. As in the monitoring, the A / D conversion unit 33 of the AFE unit 21 converts the data into RAW-RGB data.

  The RAW-RGB data is taken into the CCD I / F 34 of the signal processing unit 22, converted into YUV data by a YUV conversion unit 36 described later, and stored in the SDRAM 23 via the memory controller 35. The YUV data is read from the SDRAM 23, converted into a size corresponding to the number of recorded pixels by the resizing processing unit 37, and compressed into image data such as JPEG format by the data compression unit 39. The compressed image data such as JPEG format is written back to the SDRAM 23, read out from the SDRAM 23 through the memory controller 35, and stored in the memory card 14 through the media I / F 40.

(Dynamic range expansion processing by the YUV converter 36)
The YUV conversion unit 36 of the digital camera 1 according to the present embodiment has a dynamic range expansion processing function for expanding the dynamic range.

  A Bayer array RGB filter (see FIG. 7) is arranged on each pixel constituting the CCD 20 of the digital camera 1, but a normal RGB filter for light having a wide wavelength band such as sunlight. Have different luminance sensitivities for each color.

  For example, as shown in FIG. 3, an RGB filter (a, b, c in FIG. 3) has a sensitivity of a G (green) filter that is about twice that of an R (red) filter and a B (blue) filter. In the case of the CCD 20 having the same, when the same amount of light having a wide wavelength band, such as sunlight, is incident on the RGB filter, the G filter (shaded portion in FIG. 3c) is applied to each pixel output of the R and B filters. The pixel output first reaches the saturation level A. In FIG. 3, f is the pixel sensitivity characteristic of the G filter, g is the pixel sensitivity characteristic of the R and B filters, and the pixel sensitivity characteristic of the G filter is about twice the pixel sensitivity characteristics of the R and B filters. It has the sensitivity of.

  By the way, in a digital camera having a solid-state image sensor such as a CCD having a conventional RGB filter, a saturation level A corresponding to the pixel output of a highly sensitive G filter, such as the RGB filters of a, b, and c in FIG. In addition, the dynamic range is set. That is, even when the pixel output of the G filter reaches the saturation level A, the pixel output of the R and B filters is about ½ of the saturation level A.

  On the other hand, in the present invention, even if the pixel output of the G filter exceeds the saturation level A, as in the RGB filters of d and e in FIG. From within the pixel output level of the R and B filters, the pixel sensitivity characteristics of the R and B filters (g in FIG. 3) and the pixel sensitivity characteristics of the G filter (f in FIG. 3) Based on the above, the pixel output level of the G filter is subjected to predictive interpolation (dotted line portion), and the dynamic range is expanded by this predicted interpolation.

  The dynamic range expansion processing operation in this embodiment will be described below.

  In the present embodiment, when the photographer presses the menu (MENU) button 12 (see FIG. 1C), for example, a photographing setting screen as shown in FIG. 4 is displayed on the liquid crystal monitor (LCD) 9. By selecting the item “dynamic range doubling” from this display screen, a control signal is output from the control unit 28 to the YUV conversion unit 36, and a processing operation to double the dynamic range is executed.

  For example, when there is an extremely bright part of the background of the subject, the menu (MENU) button 12 is pressed according to the photographer's judgment and the item “dynamic range double” is selected to Range expansion processing is performed.

  In the present embodiment, as described above, it is assumed that the pixel sensitivity characteristic of the G filter has a sensitivity about twice that of each pixel sensitivity characteristic of the R and B filters. Under the blue light source, the pixel sensitivity of the R and B filters may be saturated rather than the pixel sensitivity of the G filter. If the dynamic range expansion process described above is performed under such circumstances, accurate gradation and color reproduction cannot be obtained. In such a case, the item “dynamic range doubling” described above is determined by the photographer. Do not select.

  The dynamic range expansion process is performed by the YUV conversion unit 36. As shown in FIG. 5, the YUV conversion unit 36 includes a dynamic range expansion prediction interpolation unit (hereinafter referred to as “D range expansion prediction interpolation unit”) 50, a bit compression conversion unit 51, a white balance control unit 52, and a synchronization unit 53. A tone curve converter 54, an RGB-YUV converter 55, an image size converter 56, a luminance histogram generator 57, and an edge enhancer 58.

  As illustrated in FIG. 6, the D range expansion prediction interpolation unit 50 detects the pixel output of each pixel provided with the RGB filter from the input RAW-RGB data, and the pixel provided with the G filter having the highest sensitivity. Luminance output determination unit 60 for determining whether or not the pixel output (hereinafter referred to as “G filter pixel output”) has reached a saturation level or higher, and the G level pixel output at the luminance level determination unit 60 is higher than or equal to the saturation level. Pixel output of the G filter that has reached the saturation level or higher due to the pixel output of the pixels provided with the surrounding R and B filters (hereinafter referred to as “R and B filter pixel outputs”). When the pixel output correction processing unit 61 that performs the predictive interpolation processing and the luminance level determination unit 60 determine that the pixel output of the G filter has not reached the saturation level, the G filter image Output, and R, and includes with respect to the pixel output of the B filter, the bit extension processing unit 62 which performs only each bit extension to 14 bits from 12 bits without performing conversion of the output level.

  When the luminance level determination unit 60 determines whether or not the pixel output of the G filter has reached the saturation level or higher, in the present embodiment, for each pixel of the CCD 20 having the RGB filter, as shown in FIG. In addition, 2 × 2 pixels (two G filter pixels, one R filter pixel and one R filter pixel) in the thick frame A are set as processing units (minimum units). When at least one pixel output of two G filter pixels in the processing unit (thick frame A) has reached the saturation level or higher, the sensitivity of the G filter is the R and B filters as described above. Therefore, the pixel output value of the G filter is calculated from the following equation (1).

G = {(R + B) / 2} × 2 Formula (1)
The pixel output correction processing unit 61 calculates the average value of the pixel outputs of the R and B filters and doubles the average value of the pixel outputs of the R and B filters as shown in the equation (1) to calculate the pixel output value of the G filter. The calculated pixel output values of the G filter are replaced with the pixel output values of the two G filters in the processing unit (2 × 2 pixels). Since the pixel output value of the G filter exceeds 12 bits, it is replaced here with 14-bit data. Therefore, since the maximum values of the pixel outputs of the R and B filters are both 4095 (12 bits), the maximum value of the pixel output of the G filter is 8190 and can be handled as 14-bit data.

  By the way, before the luminance level determination unit 60 determines whether or not the pixel output of the G filter has reached the saturation level or higher, the correction of the defective pixel needs to be completed. That is, if there is a defective pixel in the pixel provided with the G filter and there is a pixel that always outputs a saturated value, the pixel provided with the G filter in the same processing unit is replaced with a large value. A defective pixel will be generated. Further, when a pixel provided with the R and B filters has a defective pixel, the conversion of the pixel provided with the G filter according to the equation (1) becomes an incorrect value. Therefore, in the present embodiment, the CCD I / F 34 includes a defective pixel removal processing unit (not shown) that removes defective pixels.

  The D range expansion prediction interpolation unit 50 outputs the R and B filter pixel output data and the G filter pixel output data that has been subjected to the prediction interpolation processing after reaching the saturation level to the bit compression conversion unit 51. For example, the bit compression conversion unit 51 expands to 14 bits by conversion characteristics as shown in FIG. 8A (four-segment broken line approximation characteristics in which three nodes are designated and approximated by a straight line between them). Of the R, G, and B filter pixel outputs, the G filter pixel output is compressed to 12 bits. In FIG. 8A, a is a 12-bit range, and b is a simple linear conversion characteristic (one-dot chain line portion) that halves the data of the maximum value 8190.

  In the conversion characteristics shown in FIG. 8A, since the maximum value of the pixel output of the G filter is 8190, compression is performed so that 8190 becomes 4095. Then, the pixel output values of the R and B filters are also compressed in accordance with the compression magnification of the pixel output of the G filter.

  As described above, in the present embodiment, as an example of the case where the pixel output of the G filter whose maximum value is expanded to 8190 is compressed to 4095, three nodes as shown by the solid line in FIG. A conversion characteristic with In the present embodiment, the following two effects that cannot be obtained with a linear conversion characteristic without a simple node (b in FIG. 8A) are obtained.

  As a first effect, a larger number of bits can be assigned to data with high data reliability. That is, when predictive interpolation processing is performed on the pixel output of the G filter that has reached the saturation level or higher, the prediction interpolation is performed for the range that is equal to or greater than the specified value near the saturation level of the pixel output of the G filter. The prediction interpolation is not performed in the range below the specified value. Therefore, the accuracy of the data is different between the range where the predictive interpolation is performed and the range where the predictive interpolation is not performed.

  That is, for example, when predictive interpolation (correction) is performed on the pixel output value of the G filter that is saturated according to the equation (1), the luminance level of the subject is accurately reproduced in the range where the predictive interpolation is performed depending on the color of the main subject. It may not be possible. On the other hand, the range in which the prediction interpolation is not performed is data obtained by A / D conversion of actual data (analog RGB image signal) output from the CCD 20 having the RGB filter, and thus the reliability of this data is high. It becomes.

  That is, in the conversion characteristic (part indicated by the solid line) in this embodiment shown in FIG. 8A, for example, when the input 14-bit data is 1024, the output 12-bit data is 1024, and the original data is It is used as it is. On the other hand, for example, when the input 14-bit data is 3072, the output 12-bit data is 2560. In this range, the bit allocation is smaller than the bit allocation before the prediction interpolation, so that a bit error occurs. .

  In this way, by adopting a conversion characteristic (part indicated by a solid line) having three nodes as in this embodiment, instead of a characteristic for performing linear conversion without a simple node (part indicated by a one-dot chain line). Since the bit allocation can be gradually reduced, a larger number of bits can be allocated to data with high data reliability.

  As a second effect, gradations at low and medium luminance can be accurately stored. That is, when bit compression is performed with a simple linear conversion characteristic (b in FIG. 8A), the bit allocation becomes ¼ in a range where predictive interpolation on the low luminance side is not performed. For this reason, the image has no tone. On the other hand, when bit compression is performed with the conversion characteristic (solid line portion) in the present embodiment as shown in FIG. 8A, the data corresponding to the pixel output at the low luminance level below the saturation level is applied. Thus, by using a compression rate that gives substantially the same value before bit compression by the bit compression conversion unit 51 and after bit compression, it is possible to maintain good gradation at a low luminance level.

  In this embodiment, when the 14-bit data of the pixel output of the expanded G filter is reduced to 12 bits, three nodes are designated as shown in FIG. 8A and approximated by a straight line between them. Although the bit compression is performed with the broken line approximation characteristic (conversion characteristic) of four sections, the number of sections is not particularly limited. For example, it may be a polygonal line approximate characteristic of two sections designating one node, but the above-mentioned two effects are reduced by changing the bit allocation in the vicinity of the node. Therefore, a broken line approximation characteristic (conversion characteristic) having three or more sections is preferable.

  Further, the conversion characteristic for reducing the 14-bit data of the expanded G filter pixel output to 12 bits may be a conversion characteristic based on a curve having no plurality of nodes as shown in FIG. That is, with respect to the conversion characteristic having four sections in FIG. 8A, the conversion characteristic by this curve is obtained by setting the number of sections to 8192. In FIG. 8B, a is a 12-bit range.

  Further, by providing a lookup table having numeric data after reduction conversion to 12 bits for 0 to 8192 of input 14-bit data, the pixel output of the expanded G filter with the conversion characteristics by this curve is provided. The 14-bit data can be successfully compressed to 12 bits.

  The pixel output data of the R, G, and B filters that have been compression-converted from 14 bits to 12 bits by the bit compression conversion unit 51 are input to the white balance control unit 52. The white balance control unit 52 amplifies input pixel output data of the R, G, and B filters. At this time, the control unit 28 calculates a correction value for adjusting the white balance based on the AWB evaluation value calculated by the CCD I / F 34 and outputs the calculated correction value to the white balance control unit 52. The white balance control unit 52 adjusts the white balance based on the input correction value.

  Then, the pixel output data (12 bits) of the R, G, and B filters with the white balance adjusted from the white balance control unit 52 is input to the synchronization unit 53. The synchronizer 53 performs interpolation calculation processing on RAW data having only one color data per pixel, such as a Bayer array, and generates all RGB data for one pixel.

  All the RGB data (12 bits) generated by the synchronization unit 53 is input to the tone curve conversion unit 54. The tone curve conversion unit 54 performs γ conversion for converting 12-bit RGB data into 8-bit RGB data using a conversion table as shown in FIG. 9 to generate an 8-bit RGB value, and RGB-YUV The data is output to the conversion unit 55.

  The RGB-YUV conversion unit 55 converts input RGB data (8 bits) into YUV data by matrix calculation and outputs the YUV data to the image size converter unit 56. The image size converter unit 56 reduces or enlarges the input YUV data (8 bits) to a desired image size, and outputs it to the luminance histogram generation unit 57 and the edge enhancement unit 58.

  The luminance histogram generation unit 57 generates a luminance histogram based on the input YUV data. The edge enhancement unit 58 performs processing such as edge enhancement according to the image on the input YUV data, and stores the processed data in the SDRAM 23 via the memory controller 35.

  As described above, in this embodiment, even when the pixel output of the G filter with high sensitivity in the processing unit exceeds the saturation level, the image is saturated based on the pixel output of the R and B filters with low sensitivity. By performing predictive interpolation processing on the pixel output of the G filter, the extended region (G of d and e in FIG. 3) obtained by predictive interpolation of the pixel output of the G filter (d and e in FIG. 3) is obtained as shown in FIG. Based on the pixel output of the filter, the dynamic range can be doubled by one shooting.

  Therefore, even when there is a high-luminance portion in the background or the like in the captured image, it is possible to prevent the occurrence of overexposure and obtain good gradation.

  FIG. 10A shows a histogram generated by the luminance histogram generation unit 57 when the dynamic range expansion process in the present embodiment described above is performed when the pixel output of the G filter exceeds the saturation level. It is an example. As is apparent from this histogram, by performing the dynamic range expansion processing, whiteout in the maximum luminance portion (255) hardly occurs, and reproduction is performed with good gradation.

  On the other hand, FIG. 10B is generated by the luminance histogram generation unit 57 when the dynamic range expansion process in the present embodiment is not performed when the pixel output of the G filter exceeds the saturation level. It is an example of the done histogram. As is apparent from this histogram, it can be seen that, by not performing the dynamic range expansion process, the maximum luminance portion (255) has a frequency and whiteout occurs.

  In the description of the first embodiment and FIG. 3, the saturation level A in FIG. 3 that is a predetermined saturation level determination value and 4095 that is the maximum value of 12 bits after correction are matched. However, the present invention is not limited to this. For example, in a CCD having an RGB filter whose output linearity (linearity) is not good in a high-luminance part near where the output is completely saturated, for example, 4032 which is a value smaller than 4095 which is completely saturated is set to a predetermined saturation level. As a value (saturation level A in FIG. 3), a pixel output exceeding the value may be a target of the prediction interpolation process.

  Also, depending on the system configuration of the digital camera, even a high-luminance subject may not reach 4095, which is the maximum value of 12 bits. In that case as well, the predetermined saturation level may be set to a value lower than 4095.

  As described above, even when the predetermined saturation level is less than 4095, the output of the bit compression conversion unit 51 can be set to 4095 by switching the conversion curve of FIG. 9 according to the characteristics, and the subsequent processing can be performed. The dynamic range can be expanded without change.

  In the present embodiment, for example, when the values of the pixel output of the G filter and the pixel output of the R filter reach a saturation level or higher, the value of the pixel output of the G filter subjected to the prediction interpolation calculated from the equation (1) is used. May become inaccurate, and since the bit compression is performed at the compression rate used in the pixel output of the G filter without predictive interpolation of the pixel output value of the R filter, the hue may change.

Therefore, when the values of the pixel output of the G filter and the pixel output of the R filter (or B filter) have reached at least the saturation level, the dynamic range expansion processing by the predictive interpolation is not performed. Is preferred. Alternatively, the fact that the pixel output values of a plurality of pixels (G filter pixel output and R filter (or B filter)) have reached a saturation level or higher assumes that the brightness of the processing unit area is extremely bright, A predetermined value for the pixel output value of the G filter, for example,
G filter pixel output = 4096 x 1.8 = 7372 (14 bits)
You may set it.

  In the present embodiment, as shown in FIG. 5, 14-bit RAW-RGB data (pixel output data of R, G, and B filters) output from the D range expansion prediction interpolation unit 50 is converted into a bit compression conversion unit. 51, the white balance control unit 52 and the synchronization unit 53 perform the 12-bit data processing. However, the bit compression conversion unit 51 is also provided after the synchronization unit 53. The 14-bit data output from the synchronization unit 53 may be reduced to 12-bit data.

<Embodiment 2>
In the first embodiment, as shown in FIG. 7, 2 × 2 pixels are used as the processing unit (minimum unit) for the CCD 20 having the RGB primary color filters. However, in this embodiment, as shown in FIG. 11, 5 pixels in the thick frame A (one G filter pixel, two R (R1, R2) and B (B1, B2) filter pixels) are set as processing units (minimum units), and the processing unit is the above-described implementation. This is an example in which the range is wider than that of the first embodiment. The configuration of the digital camera, the monitoring operation, the still image shooting operation, and the dynamic range expansion processing operation are the same as those in the first embodiment.

  When the pixel output of the G filter in the processing unit of the thick frame A shown in FIG. 11 has reached the saturation level or higher, the sensitivity of the G filter is about twice the sensitivity of the R and B filters as described above. Therefore, the pixel output value of the G filter is calculated from the following equation (2).

G = [{(R1 + R2) / 2 + (B1 + B2) / 2} / 2] × 2 (2)
Then, the pixel output correction processing unit 61 of the D range expansion prediction interpolation unit 50 shown in FIG. 6 sets the pixel output value of the G filter calculated by the equation (2) within the processing unit (see FIG. 11). The pixel output value of a certain G filter is replaced, and the same processing as in the first embodiment is performed.

  In this way, by widening the processing unit, it is possible to reduce the influence due to the sensitivity difference of the other R1, R2 filter pixels and B1, B2 filter pixels in the processing unit. More accurate dynamic range expansion prediction interpolation can be performed on the output.

<Embodiment 3>
In the present embodiment, as shown in FIG. 12, the processing unit (thick frame A) is wider than that in the second embodiment with respect to the CCD 20 having the RGB filter. The configuration of the digital camera, the monitoring operation, the still image shooting operation, and the dynamic range expansion processing operation are the same as those in the first embodiment.

  If the processing unit is widened, processing is performed based on a wide range of luminance information, which is equivalent to applying a low-pass filter. Therefore, the edge portion of the luminance change is lost. Therefore, in the present embodiment, the size of the widened processing unit is partially changed using, for example, the AF evaluation value.

  That is, the CCD I / F 34 of the signal processing unit 22 shown in FIG. 2 calculates an AF evaluation value for performing AF as described above. This is an output of a high-pass filter (HPF), and a large value is output in a portion where there is a change in luminance in the screen of the captured image. Then, the control unit 28 reads out an AF evaluation value at the time of still image shooting, and discriminates between a portion having a luminance change and a portion having no luminance change in the screen. Then, the control unit 28 sets the D range expansion prediction interpolation unit 50 based on this determination data so that the processing unit is narrowed in a portion where there is a luminance change, and in a portion where there is no luminance change, As shown in FIG. 12, setting is performed so that the processing unit is in a wide range.

  Thus, even when the processing unit is further widened, accurate dynamic range expansion prediction interpolation can be performed without reducing the resolution by setting the processing unit to be narrower in a portion where there is a luminance change. .

<Embodiment 4>
In the present embodiment, as shown in FIG. 13, for the CCD 20 having RGB filters, 5 × 5 pixels of 5 pixels in the horizontal direction and 5 pixels in the vertical direction in the thick frame A are used as processing units, and the center of this processing unit. A plurality of G filter pixels are also arranged in the diagonal direction (the a direction and the b direction in FIG. 13) with respect to the G filter of the portion. It is determined whether or not the pixel output of the G filter at the center of the processing unit (inside the thick frame A) has reached the saturation level or higher.

  Further, the D range expansion prediction interpolation unit 50 of the YUV conversion unit 36 in this embodiment includes a luminance level determination unit 60, a pixel output correction processing unit 61a, and a bit expansion processing unit 62, as shown in FIG. Yes. The other configuration of the digital camera, the monitoring operation, and the still image shooting operation are the same as those in the first embodiment.

  The luminance level determination unit 60 determines whether or not the pixel output of the G filter at the center of the input processing unit (see FIG. 13: thick frame A) has reached or exceeded the saturation level. When the luminance level determination unit 60 determines that the pixel output of the central G filter has reached the saturation level or higher, the pixel output correction processing unit 61a outputs the central G filter output from the luminance level determination unit 60. A central portion that reaches a saturation level or higher due to pixel outputs of a plurality of other G filters arranged in a diagonal direction (a direction and b direction in FIG. 13) of the processing unit (thick frame A) with respect to the pixel output. Predictive interpolation processing of the pixel output of the G filter is performed.

  When the luminance level determination unit 60 determines that the pixel output of the G filter has not reached the saturation level, the bit extension processing unit 62 performs the pixel output of the G filter and the pixel output of the R and B filters. Only bit expansion is performed from 12 bits to 14 bits without converting the output level.

  In the present embodiment, the pixel output of the G filter at the center of the processing unit (thick frame A) shown in FIG. When it is determined that the saturation level has been reached, the pixel output correction processing unit 61a performs the pixel output of the central G filter output from the luminance level determination unit 60 with reference to FIGS. As shown, the pixel output of the saturated G filter is predicted from the distribution of the pixel output values of the plurality of G filters arranged in each diagonal direction (a direction and b direction in FIG. 13) of the processing unit. I tried to do it.

  FIG. 15A shows the pixel output value of each G filter on the diagonal line in the a direction, and the predicted value a1 of the pixel output of the central G filter is about 6000 (14 bits) from these output values. Can be predicted. FIG. 15B shows the pixel output value of each G filter on the diagonal line in the b direction. From these output values, the predicted value b1 of the pixel output of the central G filter is about 5000 (14 bits). Can be predicted. In FIGS. 15A and 15B, 4095 (12 bits) is the saturation level of the pixel output of the central G filter.

  From these prediction results, it can be predicted that the pixel output value of the G filter reaching the saturation level or more in the center of the processing unit (see FIG. 13: thick frame A) is about 5000 to 6000. When determining the value of the pixel output of the G filter, for example, the average of the two predicted values is 5500.

  As described above, when the pixel output of the central G filter within the processing unit reaches the saturation level or higher, the pixel output of the G filter saturated from the distribution of the pixel output values of the other G filters in the processing unit. Predicting the value enables accurate dynamic range expansion prediction interpolation.

  In the present embodiment, when predicting the pixel output value of the G filter that has reached the saturation level or higher, prediction is performed from the pixel output values of a plurality of G filters arranged in the diagonal direction within the processing unit. In addition, the pixel output of the G filter that has reached the saturation level or more is considered in consideration of the values of the pixel outputs of the plurality of G filters arranged in the horizontal direction and the vertical direction in the processing unit. May be predicted. This enables more accurate dynamic range expansion prediction interpolation.

  In the present embodiment, only the G filter pixels in the processing unit have been described. However, the R and B filter pixels can be similarly processed.

<Embodiment 5>
As shown in FIG. 16, the D range expansion prediction interpolation unit 50 of the YUV conversion unit 36 in the present embodiment includes a luminance level determination unit 60, a bit expansion processing unit 62, a first pixel output correction processing unit 63, and a second pixel output. A correction processing unit 64 and a correction data combining unit 65 are provided. The luminance level determination unit 60 and the bit extension processing unit 62 are the same as those in the first embodiment shown in FIG. The other configuration of the digital camera, the monitoring operation, and the still image shooting operation are the same as those in the first embodiment.

  The first pixel output correction processing unit 63 (corresponding to the “pixel output correction processing unit 61” in the first embodiment shown in FIG. 6) is the luminance level determination unit 60, and the pixel output of the G filter in the processing unit is equal to or higher than the saturation level. In the same way as in the first embodiment (including the second and third embodiments), the G filter that has reached the saturation level or higher from the pixel output of the R and B filters in the processing unit is determined. Predictive interpolation of pixel output.

  The second pixel output correction processing unit 64 (corresponding to the “pixel output correction processing unit 61a” in the fourth embodiment shown in FIG. 14) is a luminance level determination unit 60 in which the pixel output of the G filter in the processing unit is equal to or higher than the saturation level. In the same way as in the fourth embodiment, the pixel output of the G filter that has reached the saturation level or higher is determined from the distribution of the pixel output values of the other G filters in the processing unit. Predictive interpolation.

  The correction data synthesizing unit 65 performs appropriate predictive interpolation processing based on the following processing conditions (first to fourth processing conditions).

(First processing condition)
Under the first processing condition, the prediction interpolation information output from the first and second pixel output correction processing units 63 and 64 is averaged.

  That is, when each prediction interpolation information for the G filter pixel output is input from the first and second pixel output correction processing units 63 and 64, the correction data synthesis unit 65 averages both prediction interpolation information and calculates the average. Based on the predicted interpolation information, the pixel output of the G filter that has reached the saturation level or higher is predicted interpolated.

(Second processing condition)
In the second processing condition, the predicted interpolation value information on the first pixel output correction processing unit 63 side is given priority over the predicted interpolation value information output from the first and second pixel output correction processing units 63 and 64, respectively. .

  For example, when the main subject is in the green background area, the pixel outputs of the R and B filters are very low values. In the green background region, the pixel output of the G filter has reached the saturation level or higher, and the pixel output of the pixel output when the first pixel output correction processing unit 63 performs the predictive interpolation on the pixel output of the G filter. The predicted value may be lower than the pixel output value of the G filter that has reached the original saturation level or higher.

  Then, the correction data combining unit 65 predicts the interpolation value for the pixel output of the G filter that has reached the saturation level from the first pixel output correction processing unit 63 and the value used for the determination of the saturation level by the luminance level determination unit 60. When the predicted interpolation value from the first pixel output correction processing unit 63 is higher than the saturation level, the predicted interpolation value from the first pixel output correction processing unit 63 is used as the final prediction. When the predicted interpolation value from the first pixel output correction processing unit 63 is lower than the saturation level, the predicted interpolation value from the second pixel output correction processing unit 64 is used as the final prediction value. Adopted as an interpolation value.

(Third processing condition)
In the third processing condition, the prediction interpolation value information on the second pixel output correction processing unit 64 side is given priority over the prediction interpolation value information output from the first and second pixel output correction processing units 63 and 64, respectively. .

  When the pixel output of the G filter in the processing unit has reached the saturation level, the pixel output of the surrounding G filter may have reached the saturation level. Therefore, as described above, when the luminance level determination unit 60 determines that the pixel output of the G filter in the processing unit has reached the saturation level or more, the second pixel output correction processing unit 64 performs other processing in the processing unit. From the distribution of the pixel output values of the plurality of G filters, the pixel output of the G filter that has reached the saturation level or higher is predictively interpolated.

  At this time, the second pixel output correction processing unit 64 can correctly predict and interpolate the pixel output of the G filter that has reached the saturation level or higher from the distribution of the pixel output values of the plurality of other G filters in the processing unit. If not, the output value from the second pixel output correction processing unit 64 is set to “0”.

  Then, the correction data synthesis unit 65 uses the predicted interpolation value for the pixel output of the G filter that has reached the saturation level from the second pixel output correction processing unit 64 and the value used for the determination of the saturation level by the luminance level determination unit 60. When the predicted interpolation value from the second pixel output correction processing unit 64 is smaller than the saturation level, the predicted interpolation value from the first pixel output correction processing unit 63 is used as the final prediction. When the predicted interpolation value from the second pixel output correction processing unit 64 is adopted as the interpolation value and is higher than the saturation level, the predicted interpolation value from the second pixel output correction processing unit 64 is used as the final prediction. Adopted as an interpolation value.

(Fourth processing condition)
In the fourth processing condition, the first to third processing conditions are appropriately selected according to the predicted interpolation value information output from the first and second pixel output correction processing units 63 and 64, respectively.

  That is, the correction data combining unit 65 receives the first interpolation output value information of the pixel output of the G filter that has reached the saturation level from the first and second pixel output correction processing units 63 and 64, and One processing condition is selected, both prediction interpolation values are weighted and averaged, and the averaged prediction interpolation value is adopted as the final prediction interpolation value.

  When the output from the first pixel output correction processing unit 63 is higher than the saturation level, the correction data combining unit 65 selects the second processing condition and selects the first pixel output correction processing unit. The predicted interpolation value from 63 is adopted as the final predicted interpolation value. When the output from the second pixel output correction processing unit 64 is higher than the saturation level, the correction data combining unit 65 selects the third processing condition and selects the second pixel output correction processing unit. The predicted interpolation value from 64 is adopted as the final predicted interpolation value.

  As described above, the correction data combining unit 65 outputs the pixel output of the G filter that has reached the saturation level or higher according to the prediction interpolation value information output from the first and second pixel output correction processing units 63 and 64, respectively. Since the value can be appropriately predicted and corrected, accurate dynamic range expansion prediction interpolation can be performed.

<Embodiment 6>
In this embodiment, as shown in FIG. 17, two first and second green filters 70a and 70b having different light transmittances and openings are provided between the taking lens system 5 of the barrel unit 6 and the aperture unit 26. A rotatable turret plate 72 having a portion 71 is installed. The turret plate 72 is rotated by the motor driver 25 driven by a signal from the control unit 28, and the first and second green filters 70 a and 70 b and the opening 71 are selected and on the optical axis of the photographing lens system 5. Configured to move to. The other configuration of the digital camera, the monitoring operation, and the still image shooting operation are the same as those in the first embodiment.

  In a state where the opening 71 of the turret plate 72 is moved on the optical axis of the photographic lens system 5, the configuration is the same as that of the first embodiment in which there is nothing between the photographic lens system 5 and the aperture unit 26.

  In a state where the first green filter 70a or the second green filter 70b of the turret plate 72 is moved on the optical axis of the photographing lens system 5, the sensitivity of the R and B filters of the CCD 20 having the RGB filter is greatly reduced. . Note that the sensitivity of the G filter does not change. FIG. 18A shows the light transmittance characteristic with respect to the wavelength of light of the first green filter 70a, and FIG. 18B shows the light transmittance characteristic with respect to the wavelength of light of the second green filter 70b. As shown in these light transmittance characteristics, the first green filter 70a has a light transmittance that is about twice as high as that of the second green filter 70b.

  In a state in which the first green filter 70a is moved on the optical axis of the photographing lens system 5, the first green filter 70a transmits light so that the sensitivity of the R and B filters is about ½ of the normal sensitivity. It has rate characteristics. Therefore, in this state, the sensitivities of the R and B filters are about 1/4 of the sensitivity of the G filter (as described above, the sensitivity of the G filter is originally about twice as high as the R and B filters). ).

  When the second green filter 70b is moved on the optical axis of the photographing lens system 5, the second green filter 70b has a sensitivity of the R and B filters that is about 1/4 of the normal sensitivity. It has light transmittance characteristics. Therefore, in this state, the sensitivities of the R and B filters are approximately 1/8 of the sensitivity of the G filter (as described above, the sensitivity of the G filter is originally about twice as high as the R and B filters). ).

  FIG. 19 is a diagram showing each pixel output of the RGB filter when the first green filter 70a is moved on the optical axis of the photographic lens system 5. The a filter of a has a high sensitivity G filter. This pixel output is also in a state of a saturation level A or less.

  Also in this embodiment, the pixel output of the G filter has reached the saturation level A or higher as in the RGB filters of b, c, d, and e in FIG. When within the range of the saturation level A, the pixel output level of the G filter is determined from the pixel output level of the R and B filters, and the pixel sensitivity characteristics of the R and B filters are determined from the pixel output levels of the R and B filters. Based on g) in FIG. 19 and the pixel sensitivity characteristic of the G filter (f in FIG. 19), the pixel output level of the G filter is subjected to predictive interpolation (dotted line portion), and the dynamic range is expanded by this predictive interpolation. .

  Then, for example, when there is an extremely bright part in the background of the main subject, the menu (MENU) button 12 (see FIG. 1C) is pressed by the photographer's judgment, and the “dynamic range is 4 times. The control signal is output from the control unit 28 to the YUV conversion unit 36, and the dynamic range expansion process similar to that in the first embodiment is performed.

  For example, when the pixel output of the G filter in the same processing unit as in the first embodiment (see FIG. 7) has reached the saturation level, the sensitivity of the G filter is R, in the present embodiment, as described above. Since the sensitivity of the B filter is about 4 times, the pixel output value of the G filter is calculated from the following equation (3).

G = {(R + B) / 2} × 4 (3)
Then, the pixel output correction processing unit 61 of the D range expansion prediction interpolation unit 50 shown in FIG. 6 uses the G filter pixel output value calculated from the equation (3) as the G filter pixel in the processing unit. Replace as output value. Since the pixel output value of the G filter exceeds 12 bits, it is replaced here with 14-bit data. Therefore, since the maximum values of the pixel outputs of the R and B filters are both 4095 (12 bits), the maximum value of the pixel output of the G filter is 16383 (14 bits).

  In addition to the processing unit in the first embodiment (see FIG. 7), the processing unit may be each processing unit in the second, third, and fourth embodiments (see FIGS. 11, 12, and 13).

  The bit compression conversion unit 51 reduces the pixel output of the G filter expanded to 14 bits by conversion characteristics as shown in FIG. 20 to 12 bits. In the conversion table shown in FIG. 20, since the maximum value of the pixel output of the G filter is 16383, compression is performed so that 16383 becomes 4095. Then, the pixel output values of the R and B filters are also compressed in accordance with the compression magnification of the pixel output of the G filter. Thereafter, the same processing as in the first embodiment is performed.

  In the conversion characteristics in the present embodiment, the data corresponding to the pixel output at the low luminance level and below the saturation level is also omitted before the bit compression by the bit compression conversion unit 51 and after the bit compression. By using a compression rate that gives the same value, it is possible to maintain good gradation at a low luminance level.

  As described above, in the present embodiment, the first green filter moved on the optical axis of the photographing lens system 5 even at the time of photographing where the pixel output of the G filter in the processing unit has reached the saturation level or more. 70a can relatively reduce the sensitivity of the R and B filters to about 1/4 of the sensitivity of the G filter. As a result, as shown in FIG. 19, once based on the extended region of predictive interpolation of the G filter pixel output (dotted line portion of the G filter pixel output of b, c, d, and e in FIG. 19). With this shooting, the dynamic range can be expanded four times.

  In the sixth and seventh embodiments, a rotatable turret plate 72 having two first and second green filters 70a and 70b having different light transmittances and an opening 71 is photographed by the lens barrel unit 6. The turret plate 72 may be provided on the front side of the photographing lens system 5 (on the side opposite to the aperture unit 26), although it is configured between the lens system 5 and the aperture unit 26.

<Embodiment 7>
In the sixth embodiment, the first green filter 70a of the turret plate 72 is moved on the optical axis of the photographic lens system 5 to expand the dynamic range by a factor of four. In this example, the second green filter 70b of the turret plate 72 shown in FIG. 17 is moved on the optical axis of the photographing lens system 5 to expand the dynamic range by eight times. The other configuration of the digital camera, the monitoring operation, and the still image shooting operation are the same as those in the first embodiment.

  As described above, when the second green filter 70b is moved on the optical axis of the photographic lens system 5, the sensitivity of the R and B filters is relatively about 1 of the sensitivity of the G filter as described above. / 8.

  Then, for example, when there is an extremely bright part of the background of the main subject, the menu (MENU) button 12 (see FIG. 1C) is pressed by the photographer's judgment and the “dynamic range is 8 times” The control signal is output from the control unit 28 to the YUV conversion unit 36, and the dynamic range expansion process similar to that in the first embodiment is performed.

  For example, when the pixel output of the G filter in the same processing unit (see FIG. 7) as in the first embodiment has reached the saturation level or higher, the sensitivity of the G filter is R in this embodiment as described above. Therefore, the pixel output value of the G filter is calculated from the following equation (4). In addition to the processing unit in the first embodiment (see FIG. 7), the processing unit may be each processing unit in the second, third, and fourth embodiments (see FIGS. 11, 12, and 13).

G = {(R + B) / 2} × 8 Formula (4)
Then, the pixel output correction processing unit 61 of the D range expansion prediction interpolation unit 50 shown in FIG. 6 uses the G filter pixel output value calculated from the equation (4) as the G filter pixel in the processing unit. Replace as output value. Since the pixel output value of the G filter exceeds 12 bits, it is once replaced with 14-bit data.

  Then, the pixel output of the G filter expanded to 14 bits by the bit compression conversion unit 51 is reduced to 12 bits. Then, the pixel output values of the R and B filters are also reduced in accordance with the reduction ratio of the pixel output of the G filter. Thereafter, the same processing as in the first embodiment is performed.

  As described above, in the present embodiment, the second green filter moved on the optical axis of the photographing lens system 5 even at the time of photographing where the pixel output of the G filter within the processing unit has reached the saturation level or more. 70b can relatively reduce the sensitivity of the R and B filters to about 1/8 of the sensitivity of the G filter. As a result, it is possible to expand the dynamic range by 8 times in one shooting.

<Embodiment 8>
As shown in FIG. 21A, when the pixel output of the G filter in the processing unit (see FIG. 7) has reached the saturation level A or higher, the pixel outputs of the surrounding R and B filters are also obtained. If it is high to some extent (saturation level A or less), the dynamic range is expanded by interpolating the pixel output of the G filter to be larger than the saturation level based on equation (1) as described in the first embodiment. can do.

  However, for example, as shown in FIG. 21B, when the pixel output of the G filter in the processing unit (see FIG. 7) has reached the saturation level A or higher, the surrounding R and B filters May be extremely small compared to the pixel output of the G filter. In this case, when the pixel output of the G filter is interpolated by the expression (1) of the first embodiment, the pixel output of the G filter is reversely below the saturation level A due to the influence of the R and B filters having extremely small pixel outputs in the surrounding area. This causes a problem of being reduced.

  Therefore, in the present embodiment, the pixel output of the G filter that has reached the saturation level or higher by the pixel output correction processing unit 61 (see FIG. 6) of the D range expansion prediction interpolation unit 50 described in the first embodiment is expressed by Expression (1). Only when the calculation result is larger than the pixel output of the original G filter as shown in FIG. 21A, interpolation is performed so that the pixel output of the G filter is expanded to the saturation level or more. .

  Therefore, in the present embodiment, when the pixel output of the G filter that has reached the saturation level or more is calculated by the equation (1), the calculation result is the pixel output of the original G filter as shown in FIG. If it is smaller than the value, the interpolation processing for the pixel output of the G filter is stopped.

  As described above, according to the present embodiment, even when the pixel output of the G filter reaches the saturation level or more, the pixel outputs of the surrounding R and B filters are extremely smaller than the pixel output of the G filter. Since the interpolation processing for the pixel output of the G filter is stopped, the pixel output of the G filter is lowered to a saturation level or less, thereby preventing color misregistration.

<Embodiment 9>
As described in the eighth embodiment, for example, as shown in FIG. 21B, when the pixel output of the G filter in the processing unit (see FIG. 7) has reached the saturation level A or higher, The pixel output of the surrounding R and B filters may be extremely small relative to the pixel output of the G filter. In this case, when the pixel output of the G filter is interpolated by the expression (1) of the first embodiment, the pixel output of the G filter is reversely below the saturation level A due to the influence of the R and B filters having extremely small pixel outputs in the surrounding area. This causes a problem of being reduced.

  Therefore, in the present embodiment, when the pixel output of the G filter that has reached the saturation level or higher is interpolated, the pixel outputs of the surrounding R and B filters are extremely small relative to the pixel output of the G filter. For example, as shown in FIG. 11 of the second embodiment, the processing unit is expanded as compared with the case of FIG. 7 of the first embodiment. In this case, the pixel output of the G filter is interpolated by the expression (2) of the second embodiment.

  As described above, according to the present embodiment, when the pixel output of the G filter reaches the saturation level or higher, the pixel outputs of the surrounding R and B filters are extremely small with respect to the pixel output of the G filter. In this case, by expanding the processing unit at the time of interpolation processing, it becomes possible to perform interpolation so that the pixel output of the G filter is expanded to the saturation level or more by the pixel output of the other expanded R and B filters. .

  Even when the processing unit is expanded as shown in FIG. 11 described above, when the pixel output of the G filter is still reduced below the saturation level, for example, as shown in FIG. May be further enlarged. When the processing unit is further expanded as shown in FIG. 12, the pixel output of another G filter having the same color as that of the G filter whose pixel output has reached the saturation level or more is used for calculation when performing interpolation processing. This makes it possible to perform better interpolation processing.

  In each of the embodiments described above, the RGB three primary color filters are arranged as the color separation filters. However, the present invention is similarly applied to a configuration in which the complementary color filters are arranged as the color separation filters. Can do.

(A) is the front view which shows the digital camera as an example of the imaging device which concerns on Embodiment 1-8 of this invention, (b) is the top view, (c) is the back view. The block diagram which shows the outline | summary of the system configuration | structure in the digital camera as an example of the imaging device which concerns on Embodiments 1-8 of this invention. The figure for demonstrating the principle of the dynamic range expansion in Embodiment 1 of this invention. The figure which shows an example of the imaging | photography setting screen displayed on the liquid crystal monitor. The block diagram which shows the structure of the YUV conversion part in Embodiment 1 of this invention. The block diagram which shows the structure of the D range expansion prediction interpolation part in Embodiment 1 of this invention. The figure which shows the pixel arrangement position and processing unit of CCD which have the RGB filter in Embodiment 1 of this invention. (A) is a figure which shows the conversion characteristic which reduces the 14-bit data of the pixel output of the extended G filter in Embodiment 1 of this invention to 12 bits, (b) is other of Embodiment 1 of this invention The figure which shows the conversion characteristic which reduces 14-bit data of the pixel output of the extended G filter in an example to 12 bits. The figure which shows the conversion table which converts 12-bit RGB data into 8-bit RGB data (gamma conversion). (A) is a figure which shows the histogram at the time of performing the expansion process of the dynamic range in Embodiment 1 of this invention, (b) shows the histogram at the time of not performing the expansion process of the dynamic range in this embodiment. Figure. The figure which shows the pixel arrangement position and processing unit of CCD which have the RGB filter in Embodiment 2 of this invention. The figure which shows the pixel arrangement position and processing unit of CCD which have an RGB filter in Embodiment 3 of this invention. The figure which shows the pixel arrangement position and processing unit of CCD which have an RGB filter in Embodiment 4 of this invention. The block diagram which shows the structure of the D range expansion prediction interpolation part in Embodiment 4 of this invention. (A) is a figure which shows the pixel output value of each G filter in the diagonal line of a direction, and the predicted value of the pixel output of G filter of a center part, (b) is the pixel output of each G filter in the diagonal line of b direction. The figure which shows the predicted value of the pixel output of the value of G, and G filter of the center part. The block diagram which shows the structure of the D range expansion prediction interpolation part in Embodiment 5 of this invention. The figure which shows the structure in the lens barrel unit of the digital camera in Embodiment 6 of this invention. (A) is a figure which shows the light transmittance characteristic with respect to the wavelength of the light of a 1st green filter, (b) is a figure which shows the light transmittance characteristic with respect to the wavelength of the light of a 2nd green filter. The figure for demonstrating the principle of the dynamic range expansion in Embodiment 6 of this invention. The figure which shows the conversion characteristic which reduces the 14-bit data of the pixel output of the extended G filter to 12 bits in Embodiment 6 of this invention. (A) shows a state in which, when the pixel output of the G filter reaches a saturation level or higher, the pixel output of the G filter is expanded and interpolated to the saturation level or higher based on the pixel outputs of the surrounding R and B filters. In the figure, (a) shows that when the pixel output of the G filter reaches or exceeds the saturation level, when the pixel output of the surrounding R and B filters is extremely small, the pixel output of the G filter is less than or equal to the saturation level. The figure which shows the state by which reduction interpolation was carried out.

Explanation of symbols

1 Digital camera (imaging device)
5 Shooting lens system (lens system)
6 Lens unit 9 LCD monitor 12 Menu button (operation selection means)
20 CCD (imaging device)
21 Analog front end (analog / digital conversion means)
22 signal processor 23 SDRAM
28 Control Unit 34 CCD Interface 35 Memory Controller 36 YUV Conversion Unit 50 D Range Expansion Prediction Interpolation Unit 51 Bit Compression Conversion Unit (Bit Compression Conversion Unit)
60 Luminance level determination unit (pixel output detection means)
61, 61a Pixel output correction processing unit (pixel output correction processing means)
62-bit extension processing unit 63 first pixel output correction processing unit (first pixel output correction processing means)
64 Second pixel output correction processing section (second pixel output correction processing means)
65 Correction data synthesis unit (predictive interpolation processing means)
70a First green filter (color filter)
70b Second green filter (color filter)
72 Turret Board

Claims (20)

An object image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and an image pickup device in which a plurality of color separation filters are arranged on the front side of each pixel is provided.
Pixel output detection means for determining whether the output from any pixel reaches or exceeds a predetermined saturation level with respect to the output from each pixel;
When the pixel output detection means determines that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or higher, the pixel in which the filter other than the specific color is arranged around it based on the output from the pixel output correcting means for interpolating the pixel output is determined that the above predetermined saturation level,
When the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined saturation level, the pixel output correction processing means outputs the first bit number below the predetermined saturation level. Bit compression conversion means for compressing the pixel output data once expanded to the number of bits of 2 to the first number of bits again,
The bit compression conversion means compresses the compression corresponding to the pixel output at the saturation level or lower than the compression corresponding to the data corresponding to the pixel output in the region above the saturation level .
An imaging apparatus characterized by that. An object image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and an image pickup device in which a plurality of color separation filters are arranged on the front side of each pixel is provided.
Pixel output detection means for determining whether the output from any pixel reaches or exceeds a predetermined saturation level with respect to the output from each pixel;
When it is determined by the pixel output detection means that the output from the pixel in which the filter of the specific color is disposed has reached the predetermined saturation level or more, a filter of the same color as the surrounding specific color is disposed. Pixel output correction processing means for interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the output from the pixel ;
When the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined saturation level, the pixel output correction processing means outputs the first bit number below the predetermined saturation level. Bit compression conversion means for compressing the pixel output data once expanded to the number of bits of 2 to the first number of bits again,
The bit compression conversion means compresses the compression corresponding to the pixel output at the saturation level or lower than the compression corresponding to the data corresponding to the pixel output in the region above the saturation level .
An imaging apparatus characterized by that. An object image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and an image pickup device in which a plurality of color separation filters are arranged on the front side of each pixel is provided.
Pixel output detection means for determining whether the output from any pixel reaches or exceeds a predetermined saturation level with respect to the output from each pixel;
When the pixel output detection means determines that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or higher, the pixel in which the filter other than the specific color is arranged around it First pixel output correction processing means for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level based on an output from
When it is determined by the pixel output detection means that the output from the pixel in which the filter of the specific color is disposed has reached the predetermined saturation level or more, a filter of the same color as the surrounding specific color is disposed Second pixel output correction processing means for interpolating the pixel output determined to be equal to or higher than the predetermined saturation level, based on the output from the processed pixel,
Interpolation processing means for interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the interpolation information respectively output from the first pixel output correction processing means and the second pixel output correction processing means. ,
An imaging apparatus characterized by that. The processing unit when the output from each pixel is detected by the pixel output detection means is a size of 2 × 2 pixels in the horizontal and vertical directions.
The imaging apparatus according to any one of claims 1 to 3, wherein A color filter of the same color as the specific color is provided on the front side or the rear side of the optical system, and the color filter is inserted on the front side or the rear side of the optical system. Reducing the sensitivity of the other color filters relative to the luminance relative to the luminance;
The imaging apparatus according to any one of claims 1 to 3, wherein A plurality of the color filters are provided, and the plurality of color filters have different light transmittances.
The imaging apparatus according to claim 5. An operation selecting means for selecting and executing the operation to be interpolated when the predetermined saturation level is reached or higher with respect to the output of the pixel in which the filter of the specific color is disposed;
The imaging apparatus according to any one of claims 1 to 3, wherein When the output of the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or higher, the interpolation processing means outputs the second bit from the first bit number below the predetermined saturation level. Bit compression conversion means for compressing all data once expanded to the number of bits into the first number of bits again,
The bit compression conversion means compresses the compression corresponding to the pixel output at the saturation level or lower than the compression corresponding to the data corresponding to the pixel output in the region above the saturation level.
The imaging apparatus according to claim 3. For the data corresponding to the pixel output at the low luminance level below the saturation level, use a compression ratio that is substantially the same value before and after bit compression by the bit compression conversion means,
The imaging apparatus according to claim 8 . An object image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and an image pickup device in which a plurality of color separation filters are arranged on the front side of each pixel is provided.
Pixel output detection means for determining whether the output from any pixel reaches or exceeds a predetermined saturation level with respect to the output from each pixel;
When the pixel output detection means determines that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or higher, the pixel in which the filter other than the specific color is arranged around it Pixel output correction processing means for interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the output from
The pixel output correction processing means, when performing an operation for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level, only when the calculation result becomes larger than the original pixel output. Interpolate pixel output determined to be above saturation level,
It shall be the said imaging device. An object image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and an image pickup device in which a plurality of color separation filters are arranged on the front side of each pixel is provided.
Pixel output detection means for determining whether the output from any pixel reaches or exceeds a predetermined saturation level with respect to the output from each pixel;
When the pixel output detection means determines that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or higher, the pixel in which the filter other than the specific color is arranged around it Pixel output correction processing means for interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the output from
The pixel output correction processing means, when performing an operation for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level, if the operation result is smaller than the original pixel output, Enlarge the range of pixels used for
It shall be the said imaging device. When enlarging the range of pixels used for the calculation, also use a pixel in which the filter of the same color as the pixel in which the filter of the specific color is arranged
The imaging apparatus according to claim 11 . When the first and second pixel output correction processing units perform a calculation for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level, the calculation result is larger than the original pixel output. Only when the pixel output determined to be equal to or higher than the predetermined saturation level is interpolated,
The imaging apparatus according to claim 3. When the first and second pixel output correction processing units perform an operation for interpolating the pixel output determined to be equal to or higher than the predetermined saturation level, the calculation result is smaller than the original pixel output. In this case, the range of pixels used for the calculation is expanded.
The imaging apparatus according to claim 3. When enlarging the range of pixels used for the calculation, also use a pixel in which the filter of the same color as the pixel in which the filter of the specific color is arranged
The imaging apparatus according to claim 14 . The pixel output correction processing means, when performing an operation for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level, only when the calculation result becomes larger than the original pixel output. Interpolate pixel output determined to be above saturation level,
The imaging apparatus according to claim 1 or 2, wherein The pixel output correction processing means, when performing an operation for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level, if the operation result is smaller than the original pixel output, Enlarge the range of pixels used for
The imaging apparatus according to claim 1 or 2, wherein An imaging device including an imaging device in which a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are arranged on the front side of each pixel. In the imaging method of the apparatus,
A determination processing step for determining whether the output from any of the pixels has reached a predetermined saturation level or higher with respect to the output from each of the pixels;
When it is determined in the determination processing step that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, the surrounding pixels from which the filter other than the specific color is arranged An interpolation processing step of interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the output of
When the output of the pixel in which the filter of the specific color is disposed has reached the predetermined saturation level or higher, the second output from the first number of bits below the predetermined saturation level is output from the interpolation processing step. A bit compression conversion step of compressing the pixel output data once expanded to the number of bits into the first number of bits again,
The bit compression conversion step compresses the compression corresponding to the pixel output at the saturation level or lower than the compression corresponding to the data corresponding to the pixel output in the region above the saturation level .
An imaging method characterized by the above. An imaging device including an imaging device in which a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are arranged on the front side of each pixel. In the imaging method of the apparatus,
A determination processing step for determining whether the output from any of the pixels has reached a predetermined saturation level or higher with respect to the output from each of the pixels;
When it is determined in the determination processing step that the output from the pixel where the filter of the specific color is arranged has reached the predetermined saturation level or more, a filter of the same color as the surrounding specific color is arranged based on the output from the pixel, and interpolation processing step of interpolating the pixel output is determined that the above predetermined saturation level,
When the output of the pixel in which the filter of the specific color is disposed has reached the predetermined saturation level or higher, the second output from the first number of bits below the predetermined saturation level is output from the interpolation processing step. A bit compression conversion step of compressing the pixel output data once expanded to the number of bits into the first number of bits again,
The bit compression conversion step compresses the compression corresponding to the pixel output at the saturation level or lower than the compression corresponding to the data corresponding to the pixel output in the region above the saturation level .
An imaging method characterized by the above. An imaging device including an imaging device in which a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are arranged on the front side of each pixel. In the imaging method of the apparatus,
A determination processing step for determining whether the output from any of the pixels has reached a predetermined saturation level or higher with respect to the output from each of the pixels;
When it is determined in the determination step that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, the surrounding pixels from the pixel in which the filter other than the specific color is arranged A first interpolation processing step for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level based on an output;
When it is determined in the determination step that the output from the pixel in which the filter of the specific color is disposed has reached the predetermined saturation level or more, a filter of the same color as the specific color in the surrounding area is disposed A second interpolation processing step of interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the output from the pixel;
And a third interpolation processing step of interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the interpolation information respectively output from the first interpolation processing step and the second interpolation processing step. ,
An imaging method characterized by the above.

Images