JP2008113132A - Electronic camera and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide means capable of performing shading correction with higher accuracy according to photographing conditions.
An image sensor, a light shielding unit, a sensitivity adjustment unit, and a control unit are provided. The imaging element captures a subject image by photoelectric conversion. The light shielding unit switches between a non-light shielding state in which the light beam from the subject reaches the image sensor and a light shielding state in which the image sensor is shielded against the light beam. The sensitivity adjustment unit changes the setting of the imaging sensitivity related to the image sensor. The control unit generates data of a main image captured in a non-light-shielded state and data of a dark image captured in a light-shielded state when an imaging instruction is received in a state where the imaging sensitivity is equal to or higher than a threshold value.
[Selection] Figure 1

Description

  The present invention relates to an electronic camera and an image processing program that perform shading correction based on data of a main image and a dark image.

The output of the image sensor of the electronic camera includes a noise component due to a dark current of the image sensor and the signal component of the captured image. It is known that the level of the dark current varies depending on the exposure time of the image sensor. For this reason, some electronic cameras acquire dark current data from a dark image captured in a light-shielded state, and subtract the dark current noise from the signal level of the recording image to correct shading (the brightness of the image). Non-uniformity correction). For example, Patent Document 1 shows an example of the configuration of the electronic camera.
Japanese Patent No. 3800197

However, even with the above-described prior art, there are cases where shading correction is still insufficient depending on the photographing conditions, and an improvement thereof has been demanded.
The present invention solves the above-mentioned problems of the prior art. An object of the present invention is to provide means capable of performing shading correction with higher accuracy in accordance with shooting conditions.

  An electronic camera according to a first invention includes an image sensor, a light shielding unit, a sensitivity adjustment unit, and a control unit. The imaging element captures a subject image by photoelectric conversion. The light shielding unit switches between a non-light shielding state in which the light beam from the subject reaches the image sensor and a light shielding state in which the image sensor is shielded against the light beam. The sensitivity adjustment unit changes the setting of the imaging sensitivity related to the image sensor. The control unit generates data of a main image captured in a non-light-shielded state and data of a dark image captured in a light-shielded state when an imaging instruction is received in a state where the imaging sensitivity is equal to or higher than a threshold value.

According to a second aspect, in the first aspect, the electronic camera further includes a subtraction processing unit that subtracts the signal level value of the dark image data from the signal level value of the main image data.
In a third aspect based on the second aspect, the electronic camera further includes a clamp processing unit that fixes a black reference level of image data output from the image sensor to a value larger than 0. The subtraction processing unit performs subtraction on the main image and dark image data output from the clamp processing unit.

In a fourth aspect based on the second aspect, the control unit performs a smoothing process on the dark image data.
In a fifth aspect based on the first aspect, the electronic camera further includes an input unit for accepting an operation for changing the photographing mode from the user. Further, the control unit stops imaging of the dark image regardless of the imaging sensitivity value when the imaging mode suitable for continuous shooting is set.

In a sixth aspect based on the first aspect, the electronic camera further includes an input unit that receives manual designation of imaging sensitivity from a user. Further, the control unit generates the data of the main image and the data of the dark image when the imaging sensitivity is set to a high sensitivity value that can be selected only by manual designation.
In a seventh aspect based on the first aspect, the electronic camera further includes an input unit that receives a designation of a threshold value related to imaging sensitivity from a user.

  An electronic camera according to an eighth aspect includes an image sensor, a light shielding unit, a sensitivity adjustment unit, a temperature detection unit, and a control unit. The imaging element captures a subject image by photoelectric conversion. The light shielding unit switches between a non-light shielding state in which the light beam from the subject reaches the image sensor and a light shielding state in which the image sensor is shielded against the light beam. The sensitivity adjustment unit changes the setting of the imaging sensitivity related to the image sensor. The temperature detection unit detects the temperature state of the image sensor. When receiving an imaging instruction in at least one of a state where the imaging sensitivity is equal to or higher than the threshold and a state where the temperature of the imaging element is equal to or higher than the threshold, the control unit The captured dark image data is generated.

  An image processing program according to a ninth aspect causes a computer to execute the following steps. In the first step, main image data and dark image data are read. In the above-described main image data, a subject image is picked up by an image pickup device, and the black reference level is fixed to a value larger than zero. The dark image data is captured in a state where the image sensor is shielded from light, and the black reference level is fixed to a value larger than 0. In the second step, the signal level value of the dark image data is subtracted from the signal level value of the main image data.

  A configuration in which the configuration related to the above invention is converted into an image processing apparatus, an image processing method, a recording medium on which a computer program is recorded, and the like is also effective as a specific aspect of the present invention.

  According to the present invention, it is possible to perform shading correction with higher accuracy in accordance with shooting conditions.

FIG. 1 is a block diagram of the electronic camera of this embodiment. The electronic camera includes an image sensor 11, an image sensor drive unit 12, a mechanical shutter 13, a temperature detection unit 14, an analog signal processing unit 15, a digital signal processing unit 16, a CPU 17, a buffer memory 18, and an operation. A section 19 and a release button 20, an image processing section 21, and a recording I / F 22 are provided.
The image sensor 11 is disposed on the image space side of a photographing optical system (not shown). The image sensor 11 photoelectrically converts the image of the subject on the light receiving surface to generate an analog signal. The output of the image sensor 11 is connected to an analog signal processing unit 15.

  Here, a plurality of light receiving elements are arranged in a matrix on the light receiving surface of the image sensor 11 (illustration of the individual light receiving elements is omitted). Further, the light receiving surface of the image sensor 11 is divided into an effective pixel region and a light shielding region (optical black region) (see FIG. 2). In the effective pixel region of the image sensor 11, signal charges are accumulated in each light receiving element according to the brightness of the image of the subject. Then, the image sensor 11 generates an image signal constituting a captured image based on the output of the light receiving element in the effective pixel region.

  On the other hand, the light-shielding pixel region of the image sensor 11 is formed adjacent to the outer periphery of the effective pixel region. In this light shielding pixel region, the surface of each light receiving element is covered with a light shielding film. In this light-shielded pixel region, accumulated charges (dark current components) due to temperature changes and the like are accumulated in each light receiving element. In the electronic camera, a signal level (black reference level) corresponding to the black color of the subject image is determined based on the output of the light-shielding pixel region.

The image sensor driving unit 12 outputs a timing signal to the image sensor 11 in accordance with an instruction from the CPU 17 and controls the charge accumulation time of the image sensor 11 and signal reading.
The mechanical shutter 13 is installed so as to face the light receiving surface of the image sensor 11. The mechanical shutter 13 controls the light shielding state and the exposure state of the image sensor 11 by a front curtain and a rear curtain that are driven according to an instruction from the CPU 17. For example, the mechanical shutter 13 adjusts the exposure time of the image sensor 11 by changing the travel interval between the front curtain and the rear curtain during shooting. In addition, the electronic camera according to the present embodiment can acquire a dark image in which the noise component of the image sensor 11 is recorded by shooting in a light-shielded state with the mechanical shutter 13 closed.

The temperature detection unit 14 is a temperature sensor for detecting the temperature state of the image sensor 11. The temperature detection unit 14 is disposed, for example, in the vicinity of the image sensor 11. Note that the temperature detection unit 14 may be, for example, an observer that estimates the temperature state of the image sensor 11 from the magnitude of the dark current component.
The analog signal processing unit 15 is an analog front end circuit having a CDS unit 15a, a gain adjustment unit 15b, an analog clamp unit 15c, and an A / D conversion unit 15d. The CDS unit 15a reduces the noise component of the output signal from the image sensor 11 by correlated double sampling. The gain adjusting unit 15b amplifies the gain of the input signal according to an instruction from the CPU 17, and adjusts the imaging sensitivity corresponding to the ISO sensitivity for the image signal. The analog clamp unit 15c feeds back the signal output from the gain adjustment unit 15b and clamps the black reference level of the image signal to a constant voltage level larger than 0 value. The A / D conversion unit 15d performs A / D conversion on the output signal from the gain adjustment unit 15b. The output of the A / D conversion unit 15d is input to the digital signal processing unit 16.

  Here, FIG. 3A is a schematic diagram showing a signal output before the clamp processing in the analog clamp unit 15c. FIG. 3B is a schematic diagram showing a signal output after the clamp processing in the analog clamp unit 15c. In each drawing of FIG. 3, the vertical axis indicates the signal level value, and the horizontal axis indicates the time t when the signal is read. For simplicity, FIG. 3 shows the output of effective pixels and light-shielded pixels for a dark image.

In the signal output before the clamping process shown in FIG. 3A, both the black reference level and the effective pixel signal level tend to increase with time. At this time, the signal level fluctuates at a different ratio between the black reference level and the effective pixel level. FIG. 3 illustrates an example in which the black reference level of the image sensor 11 increases at a rate higher than the effective pixel level.
When the black reference level of the signal output is clamped by the analog clamp unit 15c, the value of the black reference level becomes constant as shown in FIG. 3B, and the DC fluctuation during signal transmission is canceled. In FIG. 3B, the signal level of the effective pixel tends to decrease with time as the black reference level is clamped.

  The digital signal processing unit 16 performs image processing such as defective pixel correction and black level adjustment on the digital image signal input in the reading order from the image sensor 11. The output of the digital signal processing unit 16 is input to the CPU 17. In the black level adjustment in the digital signal processing unit 16, the value of the black reference level clamped by the analog clamp unit 15c is shifted to 0 value (see FIG. 3C).

  The CPU 17 is a processor that performs overall control of the electronic camera. The CPU 17 controls the operation of each part of the electronic camera according to a predetermined sequence program, and executes various calculations (AF, AE, white balance, etc.) necessary for shooting. Further, the CPU 17 changes whether to take a dark image together with the main image for recording at the time of release, according to the value of the imaging sensitivity, the temperature of the imaging element 11, and the like. Further, the CPU 17 executes predetermined image processing in the shading correction of the main image using the dark image.

The buffer memory 18 is connected to the CPU 17. The buffer memory 18 has a capacity for recording image data for a plurality of frames, and temporarily stores image data in a pre-process or post-process of image processing by the CPU 17 or the image processing unit 21.
The operation unit 19 receives input operations for various settings from the user. The operation unit 19 includes, for example, a command dial or a cross-shaped cursor key. In the present embodiment, (1) setting of the shooting mode and recording mode, (2) setting of imaging sensitivity at the time of shooting, and (3) setting of a threshold value related to shooting of a dark image are performed via the operation unit 19. A user can input to the CPU 17.

  In the setting of the shooting mode in (1) above, the user can select a normal shooting mode and a continuous shooting mode (a mode in which continuous shooting speed is prioritized). In the recording mode setting, the first recording mode for recording normal photographed image data subjected to image processing and compression processing, and RAW data (image data without any image processing after A / D conversion) The user can select the second recording mode for recording.

  In the setting of the imaging sensitivity in (2) above, the user can manually specify the imaging sensitivity of the image sensor 11. In the electronic camera of this embodiment, a value on the low sensitivity side (for example, an imaging sensitivity up to ISO sensitivity 1600) of the imaging sensitivity can be used for automatic exposure correction. However, a high sensitivity value above a certain level (for example, an imaging sensitivity equivalent to ISO sensitivity 3200 or higher) can be used only when the user directly designates it manually.

In the setting of the threshold value related to the dark image shooting in (3) above, the user can manually specify the threshold value of the imaging sensitivity for shooting the dark image at the time of release. When shooting is performed with an imaging sensitivity equal to or higher than the threshold specified by the user, the main image and the dark image are shot at the time of release.
The release button 20 receives from the user an input of an AF operation instruction by a half-press operation and an input of a release instruction (exposure start instruction) by a full-press operation. The output of the release button 20 is connected to the CPU 17.

The image processing unit 21 performs various types of image processing (color interpolation, gradation conversion processing, contour enhancement processing, white balance adjustment, etc.) on the digital image signal for one frame to generate captured image data. The image processing unit 21 also executes processing for compressing the data of the main image in the JPEG format and processing for decompressing and restoring the compressed captured image data.
A connector for connecting the recording medium 23 is formed in the recording I / F 22. The recording I / F 22 executes data writing / reading with respect to the recording medium 23 connected to the connector. The recording medium 23 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the recording medium 23.

Hereinafter, an example of the photographing operation with the electronic camera of the present embodiment will be described. In the electronic camera according to the present embodiment, a main image and a dark image are captured at the time of release under a predetermined condition, and shading correction is performed using data of the dark image. FIG. 4 is a flowchart showing a determination process of the CPU 17 regarding whether or not a dark image needs to be taken.
Step 101: The CPU 17 determines whether or not the shooting mode is set to a mode other than the continuous shooting mode. In the case other than the continuous shooting mode (YES side), the CPU 17 proceeds to S102. On the other hand, in the continuous shooting mode (NO side), the CPU 17 captures only the main image without capturing a dark image at the time of release. This is because, in the continuous shooting mode, continuous shooting of the main image has the highest priority, and it is necessary to prevent the continuous shooting of the main image from being hindered by the shooting of the dark image.

  Step 102: The CPU 17 determines whether or not the current imaging sensitivity is set to a high sensitivity value that can be designated only manually. When the above condition is satisfied (YES side), the CPU 17 captures the main image and the dark image at the time of release. In this case, since noise tends to remain in the main image, it is preferable to perform shading correction with high accuracy using dark image data. On the other hand, if the above condition is not satisfied (NO side), the CPU 17 proceeds to S103.

  Step 103: The CPU 17 determines whether or not the value of the imaging sensitivity at the time of shooting is equal to or greater than a threshold value related to dark image shooting. When the above condition is satisfied (YES side), the CPU 17 captures the main image and the dark image at the time of release. This is because the user desires to perform shading correction using dark image data under this shooting condition. On the other hand, when the above condition is not satisfied (NO side), the CPU 17 captures only the main image without capturing a dark image at the time of release.

Next, a specific operation when the electronic camera captures the main image and the dark image will be described with reference to the flowchart of FIG. In the example of FIG. 5, the following description is given on the assumption that the recording mode of the CPU 17 is set to the first recording mode.
Step 201: The CPU 17 shoots the main image under a predetermined exposure condition in response to the full pressing operation of the release button 20. At this time, the CPU 17 drives the mechanical shutter 13 to bring the image sensor 11 into an exposure state, and causes the image sensor 11 to capture a subject image. The image signal of the main image read from the image sensor 11 passes through the analog signal processing unit 15 and the digital signal processing unit 16 in a pipeline manner and is buffered in the buffer memory 18.

Here, in photographing the main image in S201, the digital signal processing unit 16 omits black level adjustment for adjusting the black reference level to 0 value for the data of the main image. That is, the black reference level of the main image data in the buffer memory 18 is a value corresponding to the voltage level when clamped by the analog clamp unit 15c.
Step 202: The CPU 17 captures a dark image immediately after capturing the main image. At this time, the CPU 17 sets the mechanical shutter 13 in a light-shielded state and causes the image sensor 11 to capture a subject image. The exposure time and imaging sensitivity of the dark image are set to the same conditions as the main image.

  Then, the image signal of the dark image read from the image sensor 11 passes through the analog signal processing unit 15 and the digital signal processing unit 16 in a pipeline manner and is buffered in the buffer memory 18. Even in the dark image shooting in S202, the digital signal processing unit 16 omits the black level adjustment for adjusting the black reference level to 0 for the dark image data. That is, the black reference level of the dark image data in the buffer memory 18 is also a value according to the voltage level when clamped by the analog clamp unit 15c.

Step 203: The CPU 17 executes a smoothing process using a low-pass filter on the dark image data (S202). Thus, random noise is removed from the dark image data.
Here, consider a case where shading correction is performed using dark image data including random noise. In this case, random data of both the main image and the dark image is superimposed on the data after subtraction processing described later, and the random noise component increases in principle by a factor of √2 (about 1.41 times). However, in the present embodiment, random noise is removed from dark image data, so that random noise does not increase after shading correction as described above.

Step 204: The CPU 17 executes shading correction of the main image data using the dark image data. Specifically, the CPU 17 subtracts the signal level of the corresponding pixel in the dark image from the signal level of each pixel of the main image.
Here, in S204, data that is not subjected to black level adjustment in the digital signal processing unit 16 is subtracted, so that shading correction can be performed with high accuracy. Hereinafter, the reason will be described with reference to FIG.

  FIG. 6A is a schematic diagram illustrating main image data (S201) that is not subjected to black level adjustment in the digital signal processing unit 16. FIG. FIG. 6B is a schematic diagram showing dark image data (S202) in which the digital signal processing unit 16 does not perform black level adjustment. FIG. 6C is a schematic diagram illustrating data of the main image after shading correction according to the present embodiment. In each diagram of FIG. 6, the vertical axis indicates the signal level value, and the horizontal axis indicates the time t when the signal is read.

In the present embodiment, the black reference level is adjusted to 0 by subtracting the dark image data (FIG. 6B) from the main image data (FIG. 6A). Further, by subtracting the dark image data from the main image data, the inclination of the signal level of the effective pixel is almost completely removed (see FIG. 6C).
On the other hand, a case is considered in which black level adjustment is performed on each data in FIGS. 6A and 6B and each black reference level is shifted to 0 value. In this case, data below the black reference level is discarded. For this reason, in the dark image, there is a risk that data indicating the slope of the signal level of the effective pixel and noise component data in a range below the black reference level may be lost. That is, it can be seen that if shading correction is performed between data that has undergone black level adjustment, the signal level to be corrected cannot be subtracted from the data of the main image, and noise may remain in the corrected main image.

Step 205: In response to an instruction from the CPU 17, the image processing unit 21 executes various types of image processing and compression processing on the main image data after the subtraction processing (S204).
Step 206: The CPU 17 records the data of the main image after the compression processing on the recording medium 23. This is the end of the description of the photographing operation in FIG.
As described above, in the present embodiment, when the imaging sensitivity is set to a high sensitivity value, or when the imaging sensitivity is set to a threshold value or higher, shading correction of the main image is performed using the dark image (S102). , S103, S201 to S204). Therefore, even when the imaging sensitivity is set high, the user can acquire a high-quality main image in which noise is remarkably suppressed.

  Note that when the electronic camera captures the main image and the dark image while the electronic camera is set to the second recording mode, the CPU 17 associates the RAW data of the main image with the RAW data of the dark image and associates the image with the image. Generate a file. Then, the CPU 17 records the image file on the recording medium 23 and ends the photographing operation. In the case of the second recording mode, when the user performs development processing of the RAW data of the main image after shooting on the electronic camera or computer, the above-described shading correction may be performed using the RAW data of the dark image. It becomes possible.

(Supplementary items of the embodiment)
(1) The image processing program of the present invention is executed on the CPU 17 of the electronic camera, for example. Then, the image processing program executes shading correction on the main image and dark image data acquired at the time of shooting and the image file read from the recording medium 23. Of course, an image processing program (such as a personal computer) that can read the image file may be executed.

(2) In the above-described embodiment, the case has been described in which it is determined whether or not it is necessary to shoot a dark image according to the imaging sensitivity. As described above, when determining whether or not a dark image needs to be captured in the determination process of FIG. 4, the temperature detection unit 14 may be omitted from the configuration of the electronic camera.
Further, as shown in FIG. 7, the CPU 17 displays a dark image together with the main image when receiving an imaging instruction in at least one of a state where the imaging sensitivity is equal to or higher than the threshold and a temperature of the imaging element 11 is equal to or higher than the threshold. You may make it photograph.

Here, FIG. 7 is a diagram illustrating a modification of the determination process of FIG. Since S301 and S302 in FIG. 7 correspond to S101 and S102 in FIG. 4 respectively, redundant description is omitted.
Step 303: The CPU 17 determines whether or not the value of the imaging sensitivity at the time of shooting is equal to or greater than a threshold value related to dark image shooting. When the above condition is satisfied (YES side), the CPU 17 captures the main image and the dark image at the time of release. On the other hand, if the above condition is not satisfied (NO side), the CPU 17 proceeds to S304.

Step 304: The CPU 17 determines whether or not the temperature state of the image sensor 11 detected by the temperature detector 14 is equal to or greater than a threshold value. When the above condition is satisfied (YES side), the CPU 17 captures the main image and the dark image at the time of release. On the other hand, when the above condition is not satisfied (NO side), the CPU 17 captures only the main image without capturing a dark image at the time of release.
It should be noted that the present invention can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

Block diagram of the electronic camera of this embodiment View of the image sensor from the light-receiving surface side (A) Schematic diagram showing signal output before clamp processing in analog clamp unit, (b) Schematic diagram showing signal output after clamp processing in analog clamp unit, (c) Black level adjustment in digital signal processing unit Schematic diagram showing the signal output after Flow chart showing determination processing regarding necessity of photographing dark image Flow chart explaining the operation when shooting the main image and dark image (A) Schematic diagram showing data of main image in S201, (b) Schematic diagram showing data of dark image in S202, (c) Schematic diagram showing data of main image after shading correction. The figure which shows the modification of the determination process of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Imaging device, 13 ... Mechanical shutter, 14 ... Temperature detection part, 15 ... Analog signal processing part, 15b ... Gain adjustment part, 15c ... Analog clamp part, 17 ... CPU, 19 ... Operation part

Claims (9)

An image sensor that captures a subject image by photoelectric conversion;
A light-shielding unit that switches between a non-light-shielding state in which a light beam from a subject reaches the image sensor and a light-shielding state in which the image sensor is shielded against the light beam;
A sensitivity adjustment unit for changing the setting of the imaging sensitivity related to the imaging element;
A control unit that generates data of a main image captured in the non-light-shielded state and data of a dark image captured in the light-shielded state when an imaging instruction is received in a state where the imaging sensitivity is equal to or higher than a threshold;
An electronic camera comprising: The electronic camera according to claim 1,
An electronic camera, further comprising: a subtraction processing unit that subtracts the signal level value of the dark image data from the signal level value of the main image data. The electronic camera according to claim 2,
A clamp processing unit for fixing the black reference level of the image data output from the image sensor to a value larger than 0;
The electronic camera according to claim 1, wherein the subtraction processing unit performs the subtraction on the data of the main image and the dark image output from the clamp processing unit. The electronic camera according to claim 2,
The electronic camera according to claim 1, wherein the controller performs a smoothing process on the data of the dark image. The electronic camera according to claim 1,
It further includes an input unit for receiving an instruction to change the shooting mode from the user,
The electronic camera is characterized in that the control unit stops imaging of the dark image regardless of the value of the imaging sensitivity when a shooting mode suitable for continuous shooting is set. The electronic camera according to claim 1,
An input unit for receiving manual designation of the imaging sensitivity from a user;
The electronic camera generates the main image data and the dark image data when the imaging sensitivity is set to a high sensitivity value that can be selected only by the manual designation. The electronic camera according to claim 1,
An electronic camera, further comprising: an input unit that receives a designation of the threshold value related to the imaging sensitivity from a user. An image sensor that captures a subject image by photoelectric conversion;
A light-shielding unit that switches between a non-light-shielding state in which a light beam from a subject reaches the image sensor and a light-shielding state in which the image sensor is shielded against the light beam;
A sensitivity adjustment unit for changing the setting of the imaging sensitivity related to the imaging element;
A temperature detector for detecting a temperature state of the image sensor;
When the imaging instruction is received in at least one of the state where the imaging sensitivity is equal to or higher than the threshold and the temperature of the imaging element is equal to or higher than the threshold, data of the main image captured in the non-light-shielded state and the light-shielded state A control unit that generates data of a dark image captured in
An electronic camera comprising: A subject image is picked up by an image pickup device, the main image data in which the black reference level is fixed to a value larger than 0 value, and the image pickup device is picked up in a light-shielded state, and the black reference level is a value larger than 0 value. A first step of reading each of the dark image data fixed to
A second step of subtracting the signal level value of the dark image data from the signal level value of the main image data;
An image processing program for causing a computer to execute.

Images