JP2004056573A - Video signal processing method, processing circuit, and driving method of imaging apparatus - Google Patents

Abstract

An object of the present invention is to reduce deterioration of the S / N ratio at an intermediate input level when generating a wide dynamic range image by combining a short-time exposure image and a long-time exposure image.
A multiple adder + memory section includes a multiple adder section and a field memory, and uses the multiple adder section to input an input short-time exposure signal and a signal already stored in the field memory. The signals are added and output as a pseudo medium-time exposure signal. By using such a medium-time exposure image when generating a wide dynamic range image, a wide dynamic range image with an improved S / N ratio is obtained. In addition, motion detection within the screen is performed from image information for several fields, and for a moving part (dynamic area), the dynamic resolution is impaired by dynamically changing the multiple addition coefficient of the medium-time exposure image. A medium-time exposure image is generated without generating a wide dynamic image with an improved S / N ratio.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a video signal processing method, a processing circuit, and a driving method of an imaging device that combine a short-time exposure image and a long-time exposure image and compress the synthesized image to generate a wide dynamic range image.
[0002]
[Prior art]
Conventionally, based on the short-time exposure image and the long-time exposure image obtained from the imaging unit, the short-time exposure image is subjected to gain and offset processing, then synthesized with the long-time exposure image, and further combined with the synthesized image. 2. Description of the Related Art There is known a camera device that obtains a synthesized image having a wide dynamic range by performing level compression.
However, in the camera apparatus of such a method, an image having a low signal level of a short-time exposure image has a large gain increase, so that an image having a poor S / N ratio and a lot of noise has been generated. .
In particular, when the exposure ratio between the long-time exposure image and the short-time exposure image (that is, the ratio of the two exposure times) is set large, that is, when it is desired to widen the dynamic range of the image to be synthesized, depending on the subject, The problem is that the level of the short-time exposure image in the screen becomes too low, and the area with a poor S / N ratio may increase, and the exposure ratio cannot be set too high (that is, the dynamic range of the composite image cannot be increased). was there.
[0003]
FIG. 10 is a block diagram showing a configuration example of a video signal processing circuit provided in such a conventional camera device.
As shown in FIG. 10, the camera device includes an image sensor 11, a gain + offset unit 12, a combining unit 13, and a compression unit 14.
In such a circuit, a long exposure signal (hereinafter also referred to as a long signal) and a short exposure signal (hereinafter also referred to as a short signal) are output from the image sensor 11.
Here, FIG. 11 shows the respective exposure times, where the long-time exposure is as shown by a waveform 141 before, and the short-time exposure is as shown by a waveform 142 after.
Then, the short-time signal is input to the gain + offset unit 12 having the configuration shown in FIG. 12, multiplied by the gain by the multiplier 71, and added by the adder 72 to thereby obtain the short-time exposure before synthesis. (Hereinafter, also referred to as a short signal before synthesis).
FIG. 13 shows a graph of each waveform of the synthesizing process in the present example with respect to the amount of incident light. In the figure, a solid line 51 indicates a long exposure signal (L), a thin broken line 52 indicates a short exposure signal (S), and a thick broken line 53 indicates a composite signal (X). An arrow 54 indicates an intermediate input level described later.
[0004]
Next, in FIG. 10, the short-time signal before synthesis and the long-time signal are synthesized by the synthesis unit 13. FIG. 14 shows a configuration example of the synthesizing unit 13. As illustrated, the combining unit 13 includes a combined gain generating unit 81, multipliers 82 and 83, and an adder 84.
As shown in FIG. 14, the method of this synthesis is such that the input signal level obtained from the long-time signal or the short-time signal before synthesis is subjected to the synthesis gain generation unit 81 to determine the length of each of the long-time signal and the short-time signal before synthesis. Generating a time exposure gain signal (hereinafter also referred to as a long time gain) and a short time exposure gain signal (hereinafter also referred to as a short time gain), multiplying the long / short time signal and the long / short gain signal, and adding them. Do with.
Here, as shown in FIG. 15, when the input signal is below a certain level, the long-time gain 31 (GL) is set to 100%, and the ratio of the long-time gain is reduced from before the long-time signal is saturated, as shown in FIG. Conversely, the ratio of the short-time gain 32 (GS) is increased so that the short-time gain 32 (GS) is set to 100% immediately before the level at which the long-time signal is completely saturated.
This synthesized result has a waveform like the synthesized signal 53 shown in FIG.
[0005]
[Problems to be solved by the invention]
However, in the above-described conventional configuration, for a short-time signal, the exposure ratio of the long-time signal and the short-time signal should originally be multiplied as a gain. However, when the signal is at a level (intermediate input level 54) at which the long-time signal 51 begins to saturate in FIG. 13, the S / N ratio is particularly deteriorated.
Here, the S / N ratio is particularly degraded at the intermediate input level because the short-time signal is multiplied by a fixed value gain over the entire input level. When the signal is compressed to the standard level, as shown in an input / output curve 161 in FIG. This is because the ratio is relatively improved.
As described above, the conventional method has a problem that the S / N ratio at the intermediate input level is deteriorated, particularly when the exposure ratio is set to be high.
[0006]
For the purpose of improving the S / N ratio of the short-time exposure signal, a method of performing noise reduction processing on the short-time exposure signal using a digital filter has been proposed, for example, as disclosed in Japanese Patent Application Laid-Open No. 12-69355. However, there is a limit to the effect of improving the S / N ratio by the filtering process in the same field, and there is a situation in which a sufficient S / N ratio improving effect cannot be obtained.
[0007]
Therefore, an object of the present invention is to provide a video signal processing method capable of reducing the deterioration of the S / N ratio at an intermediate input level when generating a wide dynamic range image by combining a short exposure image and a long exposure image. An object of the present invention is to provide a processing circuit and a driving method of an imaging device.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, the present invention provides an image input step of inputting a short-time exposure image and a long-time exposure image, and a multiplex addition step of multiplex-adding the short-time exposure image input by the image input step for several fields. Combining the multi-added image generated by the multi-addition step with the long-time exposure image as a medium-time exposure image, and compressing the image synthesized by the image synthesis step to generate a wide dynamic range image. Compression step.
[0009]
The present invention also provides an image input unit for inputting a short-time exposure image and a long-time exposure image, a multiplex addition unit for multiplex-adding the short-time exposure image input from the image input unit for several fields, and the multiplex addition unit. An image synthesizing unit that synthesizes the multi-added image generated as described above with the long-time exposure image input from the image input unit as a medium-time exposure image, and compresses the image synthesized by the image synthesizing unit. And compression means for performing the compression.
[0010]
The present invention also provides an image input step of inputting a short-time exposure image and a long-time exposure image by an imaging unit, a multiplex addition step of multiplex-adding the short-time exposure image input in the image input step for several fields, An image synthesizing step of synthesizing the multi-added image generated by the multi-adding step as a medium-time exposure image with the long-time exposure image to generate a wide dynamic range image, and a compression step of compressing the image synthesized by the image synthesizing step And characterized in that:
[0011]
According to the present invention, when a short-time exposure image and a long-time exposure image are synthesized, and the synthesized image is compressed to generate a wide dynamic range image, the short-time exposure image is multiplex-added for several fields, and the multiplex addition is performed. Since an image is synthesized as a medium-time exposure image with a long-time exposure image to generate a wide dynamic range image, deterioration of the S / N ratio at an intermediate input level can be effectively reduced, and particularly, the exposure ratio is increased to shorten the S / N ratio. It is possible to remarkably improve the image quality when generating a wide dynamic range image by combining the time exposure image and the long exposure image.
It should be noted that the present invention provides a more effective S / N ratio improvement effect than the method of filtering the noise reduction of the short-time exposure signal as disclosed in the above-mentioned Japanese Patent Application Laid-Open No. 12-69355. It is.
[0012]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of a video signal processing method, a processing circuit, and a driving method of an imaging device according to the present invention will be described.
The present embodiment has the following features in a system that combines a short-time exposure image and a long-time exposure image and generates a wide dynamic range image by compressing the combined image.
(1) Multiple short-time exposure images are added by a memory for several fields to generate a pseudo medium-time exposure image (hereinafter also referred to as a medium-time signal), and generate a wide dynamic range image from the medium-time exposure image. In this case, a wide dynamic range image with an improved S / N ratio is obtained.
(2) In addition to the above (1), motion detection in the screen is performed based on image information of several fields of the short-time exposure image, the long-time exposure image, or both, and a portion (dynamic region) having this motion is By dynamically changing the multiple addition coefficient of the above-described medium-time exposure image, a medium-time exposure image is generated without deteriorating the dynamic resolution, and a wide dynamic image with an improved S / N ratio is obtained.
[0013]
Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of a video signal processing circuit provided in a camera device according to an embodiment of the present invention.
As shown in the figure, the camera device includes an image sensor 21, a gain + offset circuit 22, a multiple addition + memory unit 23, a synthesizing unit 24, and a compression unit 25. That is, the video signal processing circuit of the present example is provided with a multiplex addition + memory unit 23 for generating a medium time exposure signal in addition to the configuration shown in FIG. The input has been changed from two inputs to three inputs.
FIG. 2 is a block diagram illustrating a circuit example of the multiple addition + memory unit 23 in the present example.
The multiplex addition + memory unit 23 includes a multiplex addition unit 101 and a field memory 102. The multiplex addition unit 101 adds the input short-time signal and the signal already stored in the field memory 102, The configuration is such that it is output as a middle time signal and is also written to the field memory 102.
[0014]
FIG. 3 is a block diagram illustrating a configuration example of the multiplex addition unit 101.
The multiplex adder 101 is composed of two multipliers 111 and 112 and one adder 113, and multiplies the short-time signal (S) and the field memory output (MO) by different coefficients K1 and K2, respectively. After multiplication by the devices 111 and 112, these are added by the adder 113 to obtain a field memory input signal (MI).
This is represented by the following equation.
MI = S × K1 + MO × K2 (1)
[0015]
FIG. 4 is a block diagram illustrating a configuration example of a circuit that generates the coefficients K1 and K2.
This circuit has subtracters 121 and 123, a field memory 122, and a coefficient generator 124.
Then, first, a subtractor 121 generates a difference signal (D1) between the short-time signal (S) and the field memory output (MO). This difference signal (D1) is input to the coefficient generation unit 124, and the coefficient generation unit 124 generates coefficients (K1 and K2).
[0016]
FIG. 5 is an explanatory diagram showing a coefficient generation method in the coefficient generation unit 124. A solid line 131 indicates a coefficient 1 (K1), and a broken line 132 indicates a coefficient 2 (K2).
In FIG. 5, when the difference value (D1 or D2) is small, the input image (S) is close to the output signal (MO) of the field memory, which means that the image of that portion has little temporal change. .
In this case, as the coefficient, the coefficient K1 applied to the short-time signal (S) is reduced, and the coefficient K2 applied to the memory output (MO) is increased.
Conversely, when the difference value increases, it is considered that the image is moving. Therefore, in the dynamic region, by increasing K1 and decreasing K2, the output ratio of the short-time signal is increased. That is, the moving subject is not shaken, that is, the moving resolution is not reduced.
Note that the graph shown in FIG. 5 is an example of the relationship between the difference value and the coefficient, and it is necessary to set the curve of the graph to an optimum value while looking at the actual screen. Should be.
[0017]
In the above description, the case where the difference signal is obtained from the short-time signal and its field memory output has been described. However, when the intermediate input level is particularly low, the level of the short-time exposure signal is not sufficient. In some cases, the accuracy is insufficient for use as a difference signal. Therefore, in this case, as shown in FIG. 4, a method of using a difference (D2) between the long-time signal (L) and its field memory output (MOL) can be considered.
[0018]
FIG. 8 is a block diagram showing a configuration example of a synthesizing unit using the intermediate time signal generated by the above method. As illustrated, the synthesis unit 24 includes a synthesis gain generation unit 91, multipliers 92, 93, 94, and an adder 95.
That is, the number of signals input to the adder 95 is increased by one as compared with the conventional combining method shown in FIG. The gains (GL, GM, GS) for the long-time signal (L), the middle-time signal (M), and the short-time signal before synthesis (SG) are generated by the synthesis gain generation unit 91 as shown in FIG. That is, in FIG. 6, a thin broken line 41 indicates a long-time exposure signal gain (GL), a solid line 42 indicates a medium-time exposure signal gain (GM), and a thick broken line 41 indicates a short-time exposure signal gain (GS). I have.
At the intermediate input level, the gain 42 for the intermediate time signal is set to 100%, and the intermediate time exposure signal is synthesized at the intermediate input level.
[0019]
FIG. 7 shows the output of the combined signal with respect to the input signal, and is a graph of each waveform of the combining process with respect to the incident light amount.
In the figure, a solid line 61 indicates a long-time exposure signal (L), a thin broken line 62 indicates a short-time exposure signal (S), an alternate long and short dash line 63 indicates a medium-time exposure signal (M), and a thick broken line 64 indicates a composite signal (X). I have. An arrow 65 indicates an intermediate input level described later.
As shown in the figure, since the intermediate time exposure signal is output at the intermediate input level, the S / N ratio of the composite signal is also improved by the S / N ratio improvement effect of the field memory.
This improvement in the S / N ratio makes it possible to increase the exposure ratio and output an image with a higher dynamic range with higher contrast.
If a medium time signal is output from the image sensor, the exposure time ratio has a waveform as shown in FIG. 9, and the exposure time of long exposure is reduced as compared with the conventional timing shown in FIG. , The sensitivity is reduced.
However, in this example, since the exposure time of the long-time exposure is the same as the conventional one, the image sensitivity can be maintained without sacrificing the long-time exposure sensitivity.
[0020]
As described above, according to the present embodiment, the dynamic range of the composite image can be expanded without sacrificing the S / N ratio, so that a subject having a wider dynamic range is sacrificed for the S / N ratio. It becomes possible to image without.
Also, since the medium time exposure signal used to improve the S / N ratio is created by multiplexing the short time exposure signal with the memory, the exposure time of the long time exposure is shorter than when the same signal is obtained from the image sensor. , The image can be captured with high sensitivity, and a darker portion can be captured.
Further, since there is no new output from the image sensor, there is no need to drive the image sensor at a higher speed or to provide a new output unit, and there is an advantage that the conventional image sensor can be used as it is.
[0021]
In the above example, the output from the image sensor is output for a long time and a short time at the same time. However, a method in which the image sensor is driven at double speed and the time is adjusted externally is also applicable.
Further, in the above example, a field memory is used as a memory for multiplexing the short-time exposure images for several fields, but the same processing can be performed using a frame memory instead. In particular, when an image pickup device such as a VGA is used instead of the conventional TV signal system, a frame memory is indispensable.
Further, in the above example, a configuration using a solid-state imaging device such as a CCD image sensor or a CMOS image sensor is assumed as the imaging device. However, an imaging device using other imaging means may be used. A system in which the imaging unit and the camera body can be separated, or a single video signal processing circuit is also included in the scope of the present invention.
[0022]
【The invention's effect】
As described above, according to the video signal processing method, the processing circuit, and the driving method of the imaging device of the present invention, a short-time exposure image and a long-time exposure image are combined, and the combined image is compressed to obtain a wide dynamic range. When generating an image, the short-time exposure image is multiplex-added for several fields, and the multiplex-added image is combined with the long-time exposure image as a medium-time exposure image to generate a wide dynamic range image. In particular, when the exposure ratio is increased and the short-time exposure image and the long-time exposure image are combined to generate a wide dynamic range image, the image quality is significantly improved. There is an effect that can be achieved.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration example of a video signal processing circuit provided in a camera device according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a circuit example of a multiple addition + memory unit of the video signal processing circuit illustrated in FIG. 1;
FIG. 3 is a block diagram illustrating a configuration example of a multiplex addition unit of a multiplex addition + memory unit illustrated in FIG. 2;
4 is a block diagram illustrating a configuration example of a circuit that generates a coefficient to be supplied to a multiplex addition unit illustrated in FIG. 3;
FIG. 5 is an explanatory diagram showing a method of generating a coefficient in a coefficient generating unit of the circuit shown in FIG.
FIG. 6 is an explanatory diagram showing a long-time exposure signal gain, a medium-time exposure signal gain, and a short-time exposure signal gain in the video signal processing circuit shown in FIG.
FIG. 7 is an explanatory diagram showing a long-time exposure signal, a short-time exposure signal, a medium-time exposure signal, a composite signal, and an intermediate input level in the video signal processing circuit shown in FIG.
8 is a block diagram illustrating a configuration example of a synthesizing unit that uses a medium time signal of the video signal processing circuit illustrated in FIG. 1;
FIG. 9 is an explanatory diagram illustrating an exposure time when a medium-time exposure signal is directly input from an image sensor.
FIG. 10 is a block diagram illustrating a configuration example of a video signal processing circuit provided in a conventional camera device.
11 is an explanatory diagram showing an exposure time by the video signal processing circuit shown in FIG.
12 is a block diagram showing a gain + offset unit of the video signal processing circuit shown in FIG.
13 is an explanatory diagram showing a long exposure signal, a short exposure signal, a composite signal, and an intermediate input level in the video signal processing circuit shown in FIG.
14 is a block diagram illustrating a configuration example of a combining unit in the video signal processing circuit illustrated in FIG.
15 is an explanatory diagram showing a long-time exposure signal gain and a short-time exposure signal gain in the video signal processing circuit shown in FIG.
16 is an explanatory diagram showing a compression curve in the video signal processing circuit shown in FIG.
[Explanation of symbols]
Reference numeral 21: imaging element, 22: gain + offset circuit, 23: multiple addition + memory section, 24: synthesis section, 25: compression section, 101: multiple addition section, 102: field memory, 111 112: Multiplier, 113: Adder.

Claims (8)

An image input step of inputting a short exposure image and a long exposure image,
A multiplex addition step of multiplex addition of the short-time exposure image input by the image input step for several fields,
An image combining step of combining the multiple added image generated by the multiple addition step with the long exposure image as a medium time exposure image, and generating a wide dynamic range image,
A compression step of compressing the image synthesized by the image synthesis step;
A video signal processing method comprising: In the multiple addition step, a dynamic region in the image is detected from the short-time exposure image or the long-time exposure image or an image for several fields of both, and the medium-time exposure image 2. The video signal processing method according to claim 1, wherein the multiplex addition is performed by dynamically changing the multiplex addition coefficient. Image input means for inputting a short exposure image and a long exposure image,
Multiple addition means for multiple addition of the short-time exposure image input from the image input means for several fields,
Image combining means for combining the multiple added image generated by the multiple adding means with the long-time exposure image input from the image input means as a medium-time exposure image, and generating a wide dynamic range image,
Compression means for compressing the image synthesized by the image synthesis means;
A video signal processing circuit comprising: The multiple addition unit detects a dynamic region in the image from the short-time exposure image or the long-time exposure image or an image for several fields of both, and for the dynamic region, 4. The video signal processing circuit according to claim 3, wherein the multiplex addition is performed by dynamically changing the multiplex addition coefficient. 4. The video signal processing circuit according to claim 3, wherein said multiplex addition means includes a field memory or a frame memory for storing said short-time exposure image. 4. The video signal processing circuit according to claim 3, wherein said image input means inputs an image from an image sensor for imaging a subject. An image input step of inputting a short-time exposure image and a long-time exposure image by an imaging unit,
A multiplex addition step of multiplex addition of the short-time exposure image input by the image input step for several fields,
An image combining step of combining the multiple added image generated by the multiple addition step with the long exposure image as a medium time exposure image, and generating a wide dynamic range image,
A compression step of compressing the image synthesized by the image synthesis step;
A method for driving a camera device, comprising: In the multiple addition step, a dynamic region in the image is detected from the short-time exposure image or the long-time exposure image or an image for several fields of both, and the medium-time exposure image The method according to claim 7, wherein the multiple addition coefficient is dynamically changed to perform multiple addition.

Images