JP2009038818A - Image processing method and digital camera - Google Patents

Abstract

A digital camera suitable for obtaining an image having a wide dynamic range is provided.
An analog imaging signal acquired by a CCD is digitally converted by an analog signal processing circuit and output as image data having a gradation value for each pixel. The gradation conversion unit checks the gradation value of the input image data, detects the maximum value of the lightness average value, and determines the gradation conversion table according to the maximum value of the lightness average value. Thereafter, the gradation conversion unit performs gradation conversion processing based on the gradation conversion table. In the gradation conversion process, the input image data is set so that the slope of the input / output characteristics of the gradation value is large in the low brightness area corresponding to the main subject, and the slope of the input / output characteristics is small in the high brightness area. Are converted.
[Selection] Figure 5

Description

  The present invention relates to an image processing method for performing predetermined gradation conversion processing on image data having a signal value for each pixel, and a digital camera.

  As one of imaging devices, digital cameras are widely used. The digital camera is provided with an image sensor such as a CCD or CMOS behind the photographic lens, generates image data based on an image signal output from the image sensor, and the generated image data is stored in a storage medium such as a memory card. To record.

  When a low-brightness subject is imaged using a digital camera, the obtained imaging signal is small as a whole, and the signal value range of the imaging signal reproduced as the gradation width when reproducing the subject is also narrow. As a result, when the image data is reproduced, it is reproduced as a dark image with poor gradation.

  For this reason, when imaging a low-luminance subject, a sensitivity improvement technique called gain-up is used. The gain increase is to amplify the image signal output from the image sensor at a constant magnification. By performing the gain increase, even a low-luminance subject can be reproduced brightly and with rich gradation.

  However, if a high-brightness subject is imaged and the gain is increased, the imaging signal is amplified beyond the reproducible gradation width, and is reproduced as a so-called whiteout. For these reasons, gain-up is not suitable when the subject has a portion with high brightness and a portion with low brightness.

Therefore, in Patent Document 1 and Patent Document 2 below, a first image sensor suitable for performing high-sensitivity imaging and reproducing a low-luminance portion of the subject, and low-sensitivity imaging are performed. An image having a wide dynamic range can be obtained by providing a second image sensor suitable for reproducing a high-luminance portion and synthesizing two types of images obtained by these two types of image sensors. An imaging device is described.
JP 2001-8104 A JP 2002-165226 A

  However, when two types of image pickup elements are provided separately, the apparatus becomes large and the cost increases. In addition, when two types of image pickup devices are provided in a common area by alternately arranging a high-sensitivity image pickup device and a low-sensitivity image pickup device, the resolution of the obtained image is lowered. was there.

  These problems can be solved even with one type of image sensor by using a method of first shooting in high sensitivity mode, then shooting in low sensitivity mode, and combining the two images after shooting. . However, this method not only takes time to shoot, but also makes it impossible to obtain a normal image if the subject moves during two shootings.

  The present invention has been made to solve the above-described problems, and provides an image processing method and a digital camera capable of expanding a dynamic range without causing deterioration in image quality with a simple configuration. For the purpose.

  In order to achieve the above object, the image processing method of the present invention includes a plurality of gradation conversion characteristics in which the input / output characteristics of the gradation values change nonlinearly, and the gradation conversion characteristics selected according to the luminance of the image. On the basis of this, a gradation value conversion process is performed.

  In addition, the digital camera of the present invention has a plurality of gradation conversion characteristics in which the input / output characteristics of the gradation values change nonlinearly, and the gradation value conversion is performed based on the gradation conversion characteristics selected according to the luminance of the image. It is characterized by having a gradation conversion unit for processing. Display means for displaying an image that has been subjected to gradation conversion processing in the gradation conversion section may be provided.

  In the gradation value conversion process, the gamma conversion process based on the gamma conversion characteristic may be performed after the gradation value conversion process based on the selected gradation conversion characteristic. The gradation conversion characteristic may be such that, in an area where the input gradation value of the pixel data is large, the output gradation value gradually changes compared to an area where the input gradation value of the pixel data is small. Alternatively, the image may be divided into a plurality of regions, a luminance average value is calculated in at least one of the regions, and a gradation conversion characteristic to be used may be determined from the maximum value of the luminance average value.

  The present invention has a plurality of gradation conversion characteristics in which the input / output characteristics of the gradation values of the pixel data of each pixel constituting the image change nonlinearly, and one gradation conversion characteristic selected according to the luminance of the image Since the tone value conversion process is performed based on the above, the optimal conversion process according to the captured image can be performed, the low-brightness part of the subject is bright, and the subject is reproduced with rich gradation. The high luminance part can reproduce the gradation, and an image with a wide dynamic range can be obtained.

A perspective view of the front side of the digital camera 10 embodying the present invention is shown in FIG. 1A, and a perspective view of the back side is shown in FIG. A photographing lens 13 is provided on the front surface of the camera body 11, and a shutter button 15 is provided on the upper surface.

  On the rear surface of the camera body 11, an operation unit 17 including a cursor button in which a zoom button, a mode selection button, a power button, and four buttons are arranged in a cross shape, and an LCD 19 for performing various displays are provided. A memory card 21 is detachably set on the side surface of the camera body 11. For example, image data compressed in the JPEG format is stored in the memory card 21.

  The digital camera 10 is provided with a shooting mode, a playback mode, and a menu mode for performing various settings. In addition to the normal shooting mode, the shooting mode includes a high-sensitivity shooting mode for shooting with high sensitivity, and a wide dynamic range mode for shooting subjects with high-brightness and low-brightness parts. Is provided. Each mode can be switched by operating the operation unit 17. While switching between these modes, the user performs shooting in the shooting mode, and confirms a shot image in the playback mode.

  As shown in FIG. 2, a CPU 23 is provided inside the digital camera 10. The CPU 23 controls each part of the digital camera 10 according to an operation signal input from the shutter button 15 or the operation unit 17.

  The photographing lens 13 is driven by a lens driving mechanism 25 including a motor, and the optical photographing magnification is changed and the focus is adjusted. The aperture 27 is driven by an aperture drive mechanism 29 including a motor, and the aperture diameter is switched. The lens driving mechanism 25 and the aperture driving mechanism 29 are driven by motor drivers 31 and 33 controlled by the CPU 23.

  Behind the diaphragm 27, a CCD image sensor (CCD) 35 is disposed as an image sensor. As is well known, the CCD 35 includes a photoelectric surface in which a large number of light receiving elements are arranged in a matrix. The CCD 35 passes through the photographing lens 13 and the diaphragm 27 and photoelectrically converts subject light imaged on the photoelectric surface. In front of the photocathode, a microlens array for condensing light on each pixel and a color filter array for each color are arranged so that each pixel corresponds to one of R, G, and B.

  The CCD 35 outputs the charge accumulated for each pixel as a serial imaging signal line by line in synchronization with the vertical transfer clock and horizontal transfer clock supplied from the CCD driver 37. The charge accumulation time (exposure time) of each pixel is determined by an electronic shutter drive signal given from the CCD driver 37.

  The capturing of the imaging signal by the CCD 35 is started when the digital camera 10 is set to the photographing mode. In the shooting mode, an image pickup signal is captured as a through image with a reduced number of pixels and is continuously displayed on the LCD 19. When the shutter button 15 is pressed and a main photographing instruction is issued, the CCD 35 once stops capturing a through image and captures an imaging signal as a main image. When the main photographing is completed, the imaging signal is captured again as a through image. The analog imaging signal thus captured is output to the analog signal processing circuit 41.

  The analog signal processing circuit 41 includes a correlated double sampling circuit (CDS) 43, a signal amplifier 45, and an A / D converter 47. The CDS 43 removes noise included in the imaging signal. When the digital camera 10 is set to the normal shooting mode or the wide dynamic range mode, the analog image pickup signal from which noise has been removed is amplified by the normal gain (Ga) in the signal amplifying unit 45 and is supplied to the A / D converter 47. Sent.

  On the other hand, when the digital camera 10 is set to the high-sensitivity shooting mode, the analog imaging signal from which noise has been removed is amplified by the signal amplifying unit 45 with a gain determined according to the set sensitivity. For example, in the ISO 400 mode, amplification is performed with a gain (4Ga) that is four times that in the normal photographing mode, and with a gain (2Ga) that is twice that in the ISO 200 mode.

  FIG. 3 is a graph showing how the analog imaging signal is amplified in the signal amplification unit 45. The horizontal axis of the figure represents the analog signal value output from the CCD 35 and input to the analog signal processing circuit 41, and the vertical axis represents the analog signal value output from the analog signal amplification unit 41. The solid line in the figure represents the normal shooting mode (gain Ga), and the dotted line represents the high-sensitivity shooting mode (gain 2Ga).

  In the high-sensitivity shooting mode, the high luminance portion of the subject, that is, the portion where the input value of the analog signal value is high exceeds the maximum output value and becomes saturated, and is reproduced as so-called whiteout. However, many main subjects have low to medium luminance, and this portion can be expressed with rich gradation. Thus, when gain increase is performed, the width of the subject luminance that can be photographed (dynamic range) is reduced, but the gradation range that reproduces the low-medium luminance portion of the subject widens. The analog imaging signal amplified by the signal amplification unit 45 is transmitted to the A / D converter 47.

  The A / D converter 47 digitally converts the transmitted analog imaging signal into a signal value (gradation value) having a 10-bit gradation width (1024 gradations) to generate image data. This image data is CCD-RAW data having gradation values indicating R, G, and B densities for each pixel, and this data is output to the frame memory 51 via the data bus 49.

  In addition to the frame memory 51 and the analog signal processing circuit 41, the digital signal processing circuit 61, the CPU 23, the LCD 19, and the memory card 21 are connected to the data bus 49, and these transmit and receive information via the data bus 49. It can be done. The frame memory 51 is a working memory used when the digital signal processing circuit 61 performs various signal processes on the image data, and temporarily stores the image data.

  The digital signal processing circuit 61 includes an image adjustment unit 63, an image correction unit 65, a gradation conversion unit 67, and a gamma correction unit 68. Further, the image adjusting unit 63 is provided with an AF circuit 69, an AE circuit 71, and an AWB circuit 73.

  The AF circuit 69 detects the focal position based on the image data output from the analog signal processing circuit 41 and performs focusing. The AE circuit 71 measures the subject luminance based on the image data output from the analog signal processing circuit 41 and performs exposure control. The AWB circuit 73 is an auto white balance circuit, and automatically adjusts the white balance at the time of shooting. The image correction unit 65 performs image quality correction processing such as sharpness correction and contrast correction on the captured image.

  When the digital camera 10 is set to the normal shooting mode or gain up mode, the image data captured by the CCD 35 as a through image or a main image and output from the analog signal processing circuit 41 is an image adjustment unit 63, an image correction unit. After various processing is performed by 65, it is output to the gamma conversion unit 68 to perform gamma conversion processing.

  As is well known, when image data is output to a monitor, a difference occurs between the gradation value input to the monitor and the gradation value output from the monitor due to a physical periodic effect. The gamma conversion process is performed to correct this deviation. As shown in FIG. 4, the gamma conversion unit 68 is provided with a gamma conversion table for correcting the deviation, and the gamma conversion unit 68 receives image data input based on the gamma conversion table. The tone value is converted and output.

  On the other hand, when the digital camera 10 is set to the wide dynamic range mode, the image data captured by the CCD 35 is processed by the gradation converting unit 67 in addition to various processes by the image adjusting unit 63 and the image correcting unit 65. A key conversion process is performed. Then, a gamma conversion process is performed on the image data after the gradation conversion process. Hereinafter, the gradation conversion process will be described.

  As shown in FIG. 5, the gradation conversion unit 67 is provided with three gradation conversion tables of 100% mode, 200% mode, and 400% mode. The gradation conversion unit 67 selects one of the gradation conversion tables according to the dynamic range of the input image, and for each pixel included in the image data based on the selected gradation conversion table. The tone value is converted and output. In the present embodiment, 10-bit (gradation value 0 to 1023) data that is amplified with the normal gain (Ga) by the signal amplification unit 45 and A / D converted is input to the gradation conversion unit 67. 10-bit data is output after gradation conversion.

  The gradation conversion table to be used in the gradation conversion process is determined by the following procedure. First, as shown in FIG. 6A, the input image 70 is divided into 16 parts, and the average value (brightness average value) of the gradation values of the pixels in the area is calculated in each of the areas A1 to A16. Then, the maximum value of the luminance average value is detected by comparing the luminance average values in all 16 regions.

  As shown in FIG. 6B, the gradation conversion unit 67 selects the 100% mode when the maximum luminance average value is 0 to 255, that is, when it is determined that the subject luminance is low. . When the maximum value is 256 to 511, the 200% mode is selected, and when it is 512 to 1023, the 400% mode is selected.

  When the 100% mode is selected, the gradation conversion unit 67 linearly converts the input gradation value so that the output gradation value is quadrupled. For example, when data with a gradation value of 255 is input, data with a gradation value of 1023 is output. This has input / output characteristics equivalent to those obtained when an analog signal is amplified with a gain (4Ga) of 4 times in the high-sensitivity shooting mode.

  When the 200% mode is selected, gradation conversion is performed with the same characteristics as in the 100% mode for a portion with a low input gradation value, and the slope of the input / output characteristics increases as the input gradation value increases. The input gradation value is converted with an output / input characteristic that changes gradually and draws a non-linear curve such that the gradation value of the output data becomes 1023 when the gradation value of the input data is 511.

  Further, when the 400% mode is selected, gradation conversion is performed with the same characteristics as in the 100% mode for a portion with a low input gradation value, and the input gradation value becomes high as in the 200% mode. The slope of the input / output characteristics changes gradually as the time elapses. Then, the input gradation value is converted by an input / output characteristic that draws a non-linear curve such that the gradation value of the output data becomes 1023 when the gradation value of the input data is 1023.

  As described above, the image data subjected to the gradation conversion process by the gradation conversion table is input to the gamma conversion unit 68, and further, the gamma conversion process is performed by the gamma conversion table (see FIG. 4). As a result, the gradation values of the image data that has been subjected to the gradation conversion processing by the gradation conversion tables in the 100% mode, the 200% mode, and the 400% mode are finally converted and output as shown in FIG.

  The image data output from the gamma conversion unit 68 is displayed on the LCD 19 when acquired as a through image, and is compressed and recorded on the memory card 21 when acquired as a main image.

  Hereinafter, the operation of the present invention will be described. When imaging a subject having a high-luminance portion and a low-luminance portion, the user operates the operation unit 17 to set the digital camera 10 to the wide dynamic range mode. In the wide dynamic range mode, an optical subject image is photoelectrically converted into an imaging signal by the CCD 35, amplified by a normal gain (Ga) in the signal amplifier 45, and then digitally converted. After that, it is sent to the gradation conversion unit 67 to perform gradation conversion processing.

  The gradation conversion unit 67 detects the maximum value of the average luminance value of the input image, and selects one of the three gradation conversion tables based on this. Then, the gradation value of the input image data is converted and output by the gradation conversion characteristic determined by the selected gradation conversion table.

  In the gradation conversion process, gradation conversion is performed with the same input / output characteristics as those shot in the high sensitivity mode in the part where the input gradation value is low. It is possible to express with rich gradation. In the 200% mode or the 400% mode, gradation conversion is performed so that the change in the bright part of the subject is moderate, so that the occurrence of so-called whiteout can be suppressed and an image with a wide dynamic range can be obtained. it can.

  Further, in the above configuration, the dynamic range is expanded only by gradation conversion, and it is not necessary to provide two types of image pickup devices or to perform shooting twice, so that the image quality can be improved with a simple configuration. It becomes possible. Furthermore, since the maximum value of the luminance average value of the image data is detected and the gradation conversion characteristic is determined, optimal gradation conversion according to the image data can be performed.

  In the above embodiment, two types of nonlinear gradation conversion tables are provided, but one or three or more nonlinear gradation conversion tables may be provided and appropriately selected according to the brightness of the input image. In addition, the conversion table is determined so that the output gradation value is quadrupled in the portion where the input gradation value is low, but this magnification can be changed as appropriate.

  Moreover, although both the input gradation value and the output gradation value of the gradation conversion table are 10 bits, the number of bits may be changed as appropriate. The number of bits of the input value and the output value may be different. For example, the output value may be 9 bits with respect to the input value of 10 bits.

  In the above embodiment, the gradation conversion table is selected by dividing the input image into 16 and calculating the luminance gradation value of each area. However, the number of divided areas can be changed as appropriate. . Since the main subject is usually located at the center of the screen, the gradation conversion table may be selected based on the average brightness value at the center of the screen (A6, A7, A10, A11 in FIG. 5). .

  In the above embodiment, the digital camera has been described as an example, but the present invention can also be applied to a digital video camera. Furthermore, the present invention can also be applied to a printer device that records an image on a recording material. As shown in FIG. 8, in the wide dynamic range mode, for example, conversion tables for 100% mode, 200% mode, and 400% mode, which are selected according to the subject reflectance of the image (image brightness), are prepared. . In the 200% mode and the 400% mode, the print density of the region where the subject reflectance of the image is high (bright region) is changed gently, and the print density is higher in this region than in the normal print mode. Determine the output characteristics of the printer device. Thereby, it is possible to prevent so-called whiteout from occurring in an image to be printed out.

It is an external appearance perspective view of the front side of a digital camera, and a back side. It is a block diagram of a digital camera. It is explanatory drawing showing the effect | action of a gain up. It is a graph showing the relationship between the input gradation value and output gradation value in a gamma conversion process. It is a graph showing the relationship between the input tone value and the output tone value in the tone conversion process. It is explanatory drawing explaining the selection procedure of a gradation conversion table. It is explanatory drawing showing the relationship between the input gradation value and output gradation value in wide dynamic range mode. 6 is a graph showing the relationship between subject reflectance and print density.

Explanation of symbols

10 Digital camera 23 CPU
35 CCD
41 Analog signal processing circuit 45 Signal amplification unit 61 Digital signal processing circuit 67 Gradation conversion unit

Claims (9)

In the image processing method for converting the gradation value of the pixel data of each pixel constituting the image and outputting it,
A plurality of gradation conversion characteristics in which the input / output characteristics of the gradation values change nonlinearly, and the gradation value conversion processing is performed based on the gradation conversion characteristics selected according to the luminance of the image; A featured image processing method.   2. The image according to claim 1, wherein in the gradation value conversion process, a gamma conversion process based on a gamma conversion characteristic is performed after a gradation value conversion process based on the selected gradation conversion characteristic. Processing method.   2. The gradation conversion characteristic according to claim 1, wherein in the region where the input gradation value of the pixel data is large, the output gradation value gradually changes compared to the region where the input gradation value of the pixel data is small. 3. The image processing method according to 2.   2. The image is divided into a plurality of areas, a luminance average value is calculated in at least one of the areas, and the gradation conversion characteristic to be used is determined from a maximum value of the luminance average values. 4. The image processing method according to any one of items 1 to 3. In a digital camera that performs image processing that captures an image of a subject, configures image data, and converts a gradation value of pixel data of each pixel that forms the subject image.
A gradation that includes a plurality of gradation conversion characteristics in which the input / output characteristics of the gradation values change nonlinearly, and performs gradation value conversion processing based on the gradation conversion characteristics selected according to the luminance of the image A digital camera comprising a conversion unit.   The digital gradation processing according to claim 5, wherein the gradation value conversion processing is performed by performing gamma conversion processing based on gamma conversion characteristics after gradation value conversion processing based on the selected gradation conversion characteristics. camera.   6. The gradation conversion characteristic according to claim 5, wherein in the region where the input gradation value of the pixel data is large, the output gradation value gradually changes compared to the region where the input gradation value of the pixel data is small. 6. The digital camera according to 6.   The gradation conversion unit divides the image into a plurality of regions, calculates a luminance average value in at least one of the regions, and determines the gradation conversion characteristic to be used from the maximum value of the luminance average value. The digital camera according to claim 5, wherein:   9. The digital camera according to claim 5, further comprising display means for displaying the image based on the image data that has been subjected to gradation conversion processing in the gradation conversion unit.

Images