JP2010004367A - Imaging device - Google Patents

Abstract

An object of the present invention is to provide an imaging apparatus capable of performing appropriate simulation display even under sensor driving conditions in which external illumination is irradiated to a plurality of frames when displaying the effect of external illumination during live view.
An image pickup circuit for picking up a subject image, and illumination light emission image data and illumination non-light emission image data for simulating and displaying the effect of external illumination based on the subject image picked up by the image pickup circuit. And an image processing circuit 3 for generating image data. Further, a display circuit 9 for displaying the image data generated by the image processing circuit 3 is provided. Further, an image frame in which illumination is applied to all lines of the imaging circuit 3 is selected as illumination emission image data, and an image frame in which illumination is not applied to all lines of the imaging circuit 3 is selected as illumination non-emission image data. A system control circuit 5 is provided.
[Selection] Figure 1

Description

  The present invention relates to an imaging apparatus that captures (captures) a subject image by photoelectric conversion with an imaging element, and more particularly to an imaging apparatus that performs imaging using flash light.

  In recent years, electronic cameras using an image sensor using a photoelectric conversion element have been widely used. In the electronic camera, it is possible to convert an optical image into an electric signal and store the image in a memory card such as a semiconductor element.

  As a photoelectric conversion element, a CCD (Charge Coupled Device) image sensor is known. In addition, there is a CMOS image sensor as an image sensor that replaces a CCD image sensor.

  The CMOS image sensor is characterized in that its circuit is composed of MOS transistors, so that it consumes very little power and does not require multiple power supplies including negative voltages for charge transfer like the CCD sensor. Have.

  An electronic camera having a function called live view display or through image display is widely used. This is a function of updating and displaying image data generated from a subject image formed on an image sensor on an image display device such as an LCD (Liquid Crystal Display) at a predetermined cycle.

  In general, the image display device has an advantage that the area is larger than that of the optical viewfinder, so that the visibility is good and composition confirmation is easily performed.

  Image data used for live view display is generated by performing image processing on the image data generated by the imaging circuit. During this image processing, for example, if you change the WB (white balance) parameter or perform digital gain calculation, check the WB color tone and the effect of exposure compensation in advance before shooting. It becomes possible to do.

  On the other hand, a technique has been proposed in which the irradiation (illumination) effect of external illumination such as a strobe can be confirmed in advance before actual photographing.

  First, subject image data irradiated only with ambient light and subject image data irradiated with a predetermined amount of external illumination light and ambient light are acquired. Then, a region irradiated with external illumination light is extracted from the difference between the two subject image data.

  Next, a main light emission amount is calculated by calculation processing so that the region irradiated with the external illumination light has a predetermined exposure level. In addition, image processing is performed on the subject image data, image data is generated so that the area irradiated with the external illumination light corresponds to a predetermined exposure level, and displayed on the image display device.

  Accordingly, it is possible to perform a display in which a light emission amount equivalent to the main light emission is simulated based on image data captured with a light emission amount smaller than the main light emission.

  It is also possible to apply the above technique to a live view operation. That is, the light emission and non-light emission of the external illumination are repeated in synchronization with the imaging frame, and the simulation display corresponding to the main light emission is continuously updated.

  By the way, when performing live view using a CMOS image sensor, signals are read out in units of lines and the accumulation timing is controlled. For this reason, the time for starting the accumulation of photocharges in each line is shifted (see FIG. 5).

  In FIG. 5 (a), the portion drawn with rhombuses shows the timing of the start and end of accumulation of the imaging frame and each line. It should be noted that the charge read operation time after the completion of accumulation is omitted. Part of the rhombus and the dotted circle are enlarged, and FIG. 5B shows in detail that there is a difference in the accumulation start time of each line.

  As described above, when the accumulation start timing is different for each line, if the light emission time is too short when the external illumination is emitted, the illumination is not evenly applied to all the lines, resulting in luminance unevenness.

  An example is shown in FIG. Illumination is applied to the central portion of the rhombus, that is, near the center of the sensor surface, but no illumination is applied to the upper and lower sides.

  For such a problem, a technique for controlling the timing of sensor driving in accordance with the light emission time of external illumination has been proposed (see Patent Document 1).

According to the technique disclosed in Patent Document 1, external illumination is always applied to the entire region of the frame, and illumination can be applied to the entire region of the frame without unevenness.
JP 2006-21640 A

  However, the subject image is dark and the accumulation time cannot be shortened, or depending on the configuration of the imaging system, it may be unavoidable to operate at the timing shown in FIG.

  As indicated by the hatched portion in FIG. 7, the light is irradiated over a plurality of frames by external illumination. While the frame 1 is illuminated over all lines, the frame 2 is illuminated only on the lines above the sensor, and the amount of irradiation is not constant for each line.

  As described above, in order to display the effect of the external illumination in simulation during the live view operation, it is necessary to acquire the illumination light emission image and the illumination non-light emission image alternately and continuously.

  However, as shown in FIG. 7, under a driving condition in which the accumulation operation time is longer than the imaging frame period, it is not possible to simply switch between illumination emission and non-emission in synchronization with the imaging frame. There was a problem that images and non-illuminated images could not be acquired.

  An object of the present invention is to provide an imaging device capable of performing appropriate simulation display even under sensor driving conditions in which external illumination is irradiated to a plurality of frames when displaying the effect of external illumination during a live view. It is in.

  In order to achieve the above object, an imaging apparatus according to claim 1 is configured to simulate and display an effect of external illumination based on an imaging unit that captures a subject image and the subject image captured by the imaging unit. Image processing means for generating image data composed of illumination light emission image data and illumination non-light emission image data, display means for displaying the image data generated by the image processing means, and the imaging means as the illumination light emission image data System control means for selecting image frames in which illumination is applied to all lines of the image and selecting image frames in which illumination is not applied to all lines of the imaging means as the illumination non-emission image data. And

  The image pickup apparatus according to claim 2 starts and ends charge accumulation in a plurality of pixel lines from the first pixel line to the second pixel line in the image pickup device with a time difference for each predetermined number of pixel lines. An accumulation control means, an illumination means for illuminating the subject, an illumination control means for controlling illumination of the illumination means, image processing is performed on the charge-accumulated imaging data, and the first image data and the first image data in the accumulation control means are processed. Image processing means for generating display image data by combining two image data, and illumination by the illumination means as the first image data during charge accumulation of the first pixel line and the second pixel line. Continuously performed image frame data is selected and illumination by the illuminating means is performed during the charge accumulation of the first pixel line and the second pixel line as the second image data. It characterized in that it comprises a system control means for selecting the image frame data that has not been performed.

  According to the imaging apparatus of the present invention, when a simulation display of the effect of external illumination is performed during live view, an appropriate simulation display can be performed even under sensor driving conditions in which external illumination is applied to a plurality of frames.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram of an electronic camera as an imaging apparatus according to an embodiment of the present invention.

  Hereinafter, the configuration will be described together with the operation.

  The optical lens 1 forms a subject image on an image sensor. The optical lens 1 is also supplied with electric power for performing an autofocus (AF) operation and an aperture driving operation. The subject image formed by the optical lens 1 is AD converted by the image pickup circuit 2 including an image pickup element and an AD converter, and converted into digital data.

  The image processing circuit 3 is a circuit that performs image processing such as white balance (WB) calculation and image resizing processing on the digital data.

  The memory control circuit 4 controls data transmission / reception in the system control circuit 5, the volatile memory 8, the display circuit 9, and the recording memory 10.

  The system control circuit 5 is a circuit that controls the entire electronic camera, and also serves as an arithmetic processing circuit that performs a main operation. The nonvolatile memory 6 is a memory that records constants, variables, programs, and the like for operation of the system control circuit 5.

  The operation switch 7 is an input device for the user to perform various camera operations. The volatile memory 8 is a high-speed accessible memory such as an SDRAM.

  The display circuit 9 is a display device such as an LCD display, and can display a captured image in full color, display a captured image size capacity, a free space in the recording memory 10, and the like. The display circuit 9 can be used as a GUI operation input device that combines the operation switch 7 and display.

  The recording memory 10 is detachable from the electronic camera, and can be replaced when the memory capacity is insufficient. The light emission control circuit 11 controls the light emission unit 12 to emit light for a predetermined amount of light for a predetermined time.

  In a general live view operation, a subject image formed on the image sensor by the optical lens 1 is AD converted by the image pickup circuit 2 and converted into digital data. At this time, it is not always necessary to convert all pixels on the image sensor into digital data (reading and AD conversion).

  Generally, the number of display pixels of the display circuit 9 is smaller than the number of pixels of the image sensor, and even if all the pixels are converted to digital data, thinning processing such as resizing processing is performed at the time of display. This is because it is applied at 3.

  This is also because the time and power required for processing increase as the number of pixels converted into digital data increases. However, in the present embodiment, it is assumed that all pixels of the image sensor are converted into digital data in order to simplify the operation description.

  The image processing circuit 3 performs predetermined image processing on the digitized image data. For example, fixed pattern noise (FPN) removal processing of the image sensor, white balance optimization processing, and the like. In addition, as described above, resizing processing is also performed to generate display image data for display on the display circuit 9.

  The generated display image data is stored in a specific area of the volatile memory 8 via the memory control circuit 4. This area is generally called a VRAM area, and is an area for storing image data to be displayed on the display circuit 9.

  The display circuit 9 reads the image data from the VRAM area of the volatile memory 8 at a predetermined cycle and displays it.

  As described above, the live view can be realized by continuously performing the above-described operation.

  Here, the imaging circuit 2 functions as an imaging unit that captures a subject image.

  Further, the image processing circuit 3 generates image data including illumination light emission image data and illumination non-light emission image data for displaying the effect of external illumination on the basis of the subject image captured by the imaging unit. Functions as a means.

  The display circuit 9 functions as display means for displaying the image data generated by the image processing means.

  Further, the system control circuit 5 selects an image frame in which illumination is applied to all lines of the imaging unit as illumination light emission image data, and an image in which illumination is not applied to all lines of the imaging unit as illumination non-emission image data. It functions as a system control means for selecting a frame.

  Furthermore, the image pickup circuit 2 starts and ends charge accumulation in a plurality of pixel lines from the first pixel line to the second pixel line in the image pickup device with a time difference for each predetermined number of pixel lines. Functions as a means.

  The light emitting unit 12 functions as an illumination unit that illuminates the subject, and the light emission control circuit 11 functions as an illumination control unit that controls illumination of the illumination unit.

  The illumination means is controlled by the illumination control means at a predetermined timing in synchronization with the accumulation operation in the accumulation control means. The illumination control means can control the amount of illumination light within the illumination period of the illumination means to be constant.

  The image processing circuit 3 functions as an image processing unit that performs image processing on the captured image data that has been stored, and generates display image data by combining the first image data and the second image data in the storage control unit. To do.

  The display circuit 9 functions as display means for displaying the display image data generated by the image processing means.

  Further, the system control circuit 5 as the system control means uses, as the first image data, image frame data in which illumination by the illumination means is continuously performed during the charge accumulation of the first pixel line and the second pixel line. Select. Then, the system control circuit 5 selects, as the second image data, image frame data that is not illuminated by the illumination unit during the charge accumulation of the first pixel line and the second pixel line.

  The system control means calculates the main illumination light amount at the time of the main illumination photographing using the first image data and the second image data.

  Histogram data calculated from the display image data generated by the image processing means is displayed on the display means.

  FIG. 2 is a flowchart showing a procedure of photographing processing executed by the electronic camera of FIG.

  This process is executed under the control of the system control circuit 5 in FIG.

  The user operates the operation switch 7 to instruct to start the external illumination simulation display mode in the live view operation. Then, the system control circuit 5 executes a simulation display process (step S201).

  Hereinafter, the operation in step S201 will be described with reference to FIGS.

  FIG. 3 is a flowchart showing the procedure of the simulation display process executed in step S201 of FIG. FIG. 4 is a time chart showing the operation relationship between the operation timing of the imaging circuit 2, the light emission timing of the light emitting unit 12, and the display content displayed on the display circuit 9 in the simulation display process executed in step S <b> 201 of FIG. 2. is there.

  In FIG. 3, first, the system control circuit 5 sets 1 to an internal variable N corresponding to the imaging frame number (step S301).

  At timing t1 in FIG. 4, the system control circuit 5 communicates with the light emission control circuit 11, and causes the light emitting unit 12 to emit light with a predetermined light emission amount Y preliminary light emission (step S302).

  Here, the light emission amount Y preliminary light emission is set to 1/16 of the maximum light amount that the light emitting unit 12 can emit. This is because preliminary light emission is intermittently continuously performed as described later, so that power consumption is accelerated when the amount of light is too large.

  The preliminary light emission need only be able to detect the position of the subject and the amount of light reflected from the subject, and therefore need not be increased more than necessary. Conversely, if it is determined that the amount of emitted light is small because the amount of reflected light cannot be detected or the like, the amount of preliminary light emission is changed to 1/8 or the like.

  Next, the system control circuit 5 communicates with the imaging circuit 2 to start a photographing (imaging) operation and start a charge accumulation operation of the frame 1 (step S303). As shown in FIG. 4, since there is a difference in the accumulation start time of each line, the timing chart of the imaging frame is represented by a rhombus shape.

  Subsequently, the accumulation operation of frame 2 is started at timing t2 (step S304). At the timing t2, some lines of the frame 1 are being accumulated. That is, on the timing chart, adjacent frames are simultaneously accumulating.

  At the timing of t3, the accumulation operation and the read operation of the last line of frame 1 are completed (step S305). The read image data of frame 1 is subjected to predetermined image processing by the image processing circuit 3 and then stored in the volatile memory 8 through the memory control circuit 4.

  Similarly, the accumulation operation of frame 3 is started at timing t4 (step S306).

  At the timing of t5, the accumulation operation and the read operation of frame 2 are completed (step S307). The read image data of frame 2 is subjected to predetermined image processing by the image processing circuit 3 and then stored in the volatile memory 8 via the memory control circuit 4. The image data of the frame 2 is image data that is illuminated uniformly over the entire line.

  Further, in synchronization with the end of accumulation of the final line of frame 2, the system control circuit 5 communicates with the light emission control circuit 11 to stop the light emission of the light emitting unit 12 (step S308).

  The accumulation operation for frame 4 is started at timing t6 (step S309), and the accumulation operation for frame 3 and the read operation are completed at timing t7 (step S310). The read image data of the frame 3 is subjected to predetermined image processing by the image processing circuit 3 and then stored in the volatile memory 8 via the memory control circuit 4.

  The accumulation operation of frame 5 is started at timing t8 (step S311).

  At the timing t9, the accumulation operation and the read operation of frame 4 are completed (step S312). The read image data of the frame 4 is subjected to predetermined image processing by the image processing circuit 3 and then stored in the volatile memory 8 via the memory control circuit 4. The image data of the frame 4 is image data that is not irradiated with illumination over all lines.

  Further, in synchronization with the end of the accumulation operation of the last line of the frame 4, the system control circuit 5 communicates with the light emission control circuit 11, and causes the light emitting unit 12 to emit light again with the predetermined light amount Y preliminary light emission (step S313). .

  The system control circuit 5 selects the image data of the frame 2 as the illumination emission image and reads it from the volatile memory 8 (step S314). On the other hand, the image data of the frame 4 is selected as the non-illuminated image, and is similarly read from the volatile memory 8 (step S315).

  As described above, for the illumination emission image, the image data irradiated with the illumination over the entire line of the frame is selected, and the image data irradiated with the illumination only for a part of the lines of the frame is not selected.

  In addition, for the non-illuminated image, image data that is not illuminated over the entire line of the frame is selected, and image data that is illuminated only on some lines of the frame is not selected.

  The system control circuit 5 extracts an illumination irradiation area, that is, a subject area, using the illumination light emission image data and the illumination non-light emission image data (step S316). Further, the amount of reflected light of the illumination light applied to the subject is calculated (step S317).

  As a method for extracting the subject area, for example, a luminance difference is calculated for each pixel of the illumination light emission image data and the illumination non-issued image data, and it is determined that illumination is radiated to a pixel whose value exceeds a predetermined threshold value. Adopt the method to do.

  In addition, the amount of reflected light from the subject can also be obtained from the difference in luminance value of each pixel. Here, the luminance value of each pixel of the illumination light emission image data and the illumination non-light emission image data is L emission (x, y) and L non-light emission (x, y), respectively.

  In this case, the reflected light amount component L reflected light (x, y) of the subject can be expressed by Expression (1). Here, (x, y) is a coordinate value of each pixel in an image sensor (not shown) constituting the image pickup circuit 2.

次に、システム制御回路５は、ライブビュー時の蓄積時間条件に対して、露出が適正レベルとなる照明発光量の算出を行う。適正露光レベルの評価方法としては、例えば、画像の特定ブロック領域の輝度平均値が所定値Ｌ適正となるようにする方法を採用する。

Next, the system control circuit 5 calculates the amount of illumination light emission at which the exposure is at an appropriate level with respect to the accumulation time condition during live view. As a method for evaluating the appropriate exposure level, for example, a method is adopted in which the average luminance value of the specific block area of the image is appropriate to the predetermined value L.

  The upper left coordinates of the rectangular area including the subject area extracted in step S316 are (x1, y1), and the lower right coordinates are (x2, y2). At this time, the coefficient α satisfying the equation (2) is the light amount correction coefficient (step S318).

ステップＳ３１９では、以下の処理が行われる。即ち、このように得られた光量補正係数αと、非発光画像データＬ非発光（ｘ，ｙ）及び反射光データＬ反射光（ｘ，ｙ）を用いて、画像処理回路３は本発光相当の光量照射をシミュレーションした、表示用画像データＬ表示（ｘ，ｙ）を生成する。具体的には、式（３）の演算処理を行う。

In step S319, the following processing is performed. That is, the image processing circuit 3 corresponds to the main light emission by using the light amount correction coefficient α thus obtained, the non-light emission image data L non-light emission (x, y), and the reflected light data L reflected light (x, y). The display image data L display (x, y) is generated by simulating the irradiation of the light quantity. Specifically, the calculation process of Expression (3) is performed.

生成された表示用画像データは、メモリ制御回路４を介して、揮発性メモリ８のＶＲＡＭ領域に格納される。表示回路９は、揮発性メモリ８のＶＲＡＭ領域から表示用画像データを読み出して表示する（ステップＳ３２０）。

The generated display image data is stored in the VRAM area of the volatile memory 8 via the memory control circuit 4. The display circuit 9 reads the display image data from the VRAM area of the volatile memory 8 and displays it (step S320).

  Next, the system control circuit 5 adds 2 to the internal variable N (step S321).

  If the release operation by the operation switch 7 has not been performed by the user so far, the process returns to the sequence of step S309. When the release operation by the operation switch 7 is performed by the user, the process returns to the main flow (step S322).

  When the release operation is instructed, the system control circuit 5 calculates the main light emission amount Y main light emission using α calculated as described above. If the accumulation time of each line at the time of live view is t accumulation, and the emission time at the time of main flash photography is t main emission, it can be obtained by equation (4).

求めた光量にて本発光撮影を行い（ステップＳ２０３）、撮影された画像を記録用メモリ１０に記録する（ステップＳ２０４）。

The main flash photography is performed with the obtained light quantity (step S203), and the photographed image is recorded in the recording memory 10 (step S204).

  As described above, according to the present invention, even if the influence of illumination irradiation by the light emitting unit 12 extends over a plurality of frames, by appropriately selecting the light emission image data and the non-light emission image data, the simulation corresponding to the main light emission is performed. Display can be performed during live view operation.

  In the embodiment of the present invention, the light emitting unit 12 is preferably an illumination device such as an LED that has a stable light emission amount during light emission.

  In the present embodiment, the main light emission simulation display image displayed on the display circuit 9 is generated so that the subject image corresponds to the appropriate exposure, but the exposure level of the subject image can be changed intentionally. You may do it. With such a configuration, it is possible to check the dimming correction amount on the display circuit 9 in advance.

  Further, in the present embodiment, only the main light emission simulation display image is displayed on the display circuit 9, but a histogram diagram calculated based on the main light emission simulation display image may be displayed. This makes it possible to confirm the dimming effect in more detail.

It is a block diagram of the electronic camera as an imaging device concerning an embodiment of the invention. It is a flowchart which shows the procedure of the imaging | photography process performed with the electronic camera of FIG. It is a flowchart of the procedure of the simulation display process performed by step S201 of FIG. 3 is a time chart showing the operational relationship between the operation timing of the imaging circuit 2, the light emission timing of the light emitting unit 12, and the display content displayed on the display circuit 9 in the simulation display process executed in step S201 of FIG. It is a figure explaining that the difference of the accumulation start time arises for every line in the conventional imaging device. It is a figure explaining illumination light emission becoming nonuniform in the conventional imaging device. It is a figure explaining that illumination light emission influences several frames in the conventional imaging device because accumulation time is long.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens 2 Imaging circuit 3 Image processing circuit 4 Memory control circuit 5 System control circuit 6 Non-volatile memory 7 Operation switch 8 Volatile memory 9 Display circuit 10 Recording memory 11 Light emission control circuit 12 Light emission part

Claims (6)

Imaging means for capturing a subject image;
Image processing means for generating image data composed of illumination light emission image data and illumination non-light emission image data for simulating and displaying the effect of external illumination based on the subject image imaged by the imaging means;
Display means for displaying the image data generated by the image processing means;
An image frame in which illumination is applied to all lines of the imaging unit is selected as the illumination light emission image data, and an image frame in which illumination is not applied to all lines of the imaging unit is selected as the illumination non-emission image data. System control means;
An imaging apparatus comprising: Accumulation control means for starting and ending charge accumulation in a plurality of pixel lines from the first pixel line to the second pixel line in the image sensor with a time difference for each predetermined number of pixel lines;
Illumination means for illuminating the subject;
Illumination control means for controlling illumination of the illumination means;
Image processing means for image-processing the charge-accumulated imaging data and generating display image data by combining the first image data and the second image data in the accumulation control means;
As the first image data, image frame data in which illumination by the illuminating means is continuously performed during charge accumulation of the first pixel line and the second pixel line is selected, and the second image data System control means for selecting image frame data that is not illuminated by the illumination means during charge accumulation of the first pixel line and the second pixel line;
An imaging apparatus comprising:   The imaging apparatus according to claim 2, wherein the illumination unit is controlled to be illuminated by the illumination control unit at a predetermined timing in synchronization with the accumulation operation in the accumulation control unit.   The imaging apparatus according to claim 2, wherein the illumination control unit can control the amount of illumination light within an illumination period of the illumination unit to be constant.   The imaging apparatus according to claim 2, wherein the system control unit calculates a main illumination light amount at the time of main illumination photographing using the first image data and the second image data.   The imaging apparatus according to claim 2, further comprising display means for displaying histogram data calculated from the display image data generated by the image processing means.

Images