JP2015015622A - Imaging apparatus, and control method therefor - Google Patents

Abstract

[PROBLEMS] To capture a still image having a high dynamic range expansion effect during shooting of a moving image having a substantially expanded dynamic range. When a CPU 101 receives a SW1 signal indicating a half-pressed state of a shutter button, HDR still image exposure calculation is performed in parallel with moving image shooting and HDR moving image synthesis processing. Thereafter, when the SW2 signal indicating the fully pressed state of the shutter button is received, the proper exposure image and underexposure image obtained by moving image shooting are stored for HDR still image synthesis. Next, the CPU 101 performs shooting by changing the underexposure of the next frame to overexposure, stores the overexposed image in the memory 103, and synthesizes an HDR still image using each of the stored images. [Selection] Figure 1

Description

  In particular, the present invention relates to an imaging apparatus, an imaging apparatus control method, a program, and a storage medium that are suitable for capturing moving images and still images with an extended dynamic range.

  Conventionally, there has been known a technique for obtaining an image representing both characteristics even when there is an extreme difference in brightness between photographing targets. For example, Patent Document 1 discloses a technique of capturing a plurality of moving images with different exposures and combining them to obtain a moving image (hereinafter referred to as an HDR moving image) in which a substantial dynamic range is expanded. For example, Patent Document 2 discloses an imaging apparatus that can capture a still image without interrupting the capturing of the moving image.

JP-A-6-273354 JP 2005-57378 A

  Here, in order to obtain a still image (hereinafter referred to as an HDR still image) with an extended substantial dynamic range, three still images with different exposures are combined. That is, by capturing and combining a properly exposed image with exposure adjusted to the main subject, an underexposed image with exposure adjusted to the higher luminance side, and an overexposed image adjusted exposure to the lower luminance side. An HDR still image can be obtained. On the other hand, when obtaining an HDR video, it is necessary to maintain a constant frame rate. Therefore, when shooting HDR moving images, it is necessary to reduce the number of images used for composition compared to shooting HDR still images. For this reason, when trying to shoot an HDR still image during shooting of an HDR video, there is a problem that only a low dynamic range expansion effect image can be obtained because there are not enough types of exposure necessary to synthesize the HDR still image. is there.

  SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and an object of the present invention is to make it possible to shoot a still image having a high dynamic range expansion effect during shooting of a moving image with a substantial dynamic range expanded.

  The imaging device according to the present invention generates a first frame by shooting with a first exposure, and generates a second frame by shooting with a second exposure different from the first exposure, A moving image photographing means for generating a moving image composed of a frame obtained by combining the first frame and the second frame, a first image by photographing with the first exposure, and the second To generate a second image, and to generate a third image by shooting at a third exposure different from the first exposure and the second exposure, the first image, A still image capturing unit that generates a still image obtained by combining the second image and the third image; and an instruction that instructs the still image capturing unit to capture a still image while the moving image capturing unit captures a moving image. In response to a photographing instruction by the means and the instruction means. Controlling the moving image photographing means to generate a third image by the exposure of the first frame, a first frame generated by the first exposure, and a second frame generated by the second exposure; And a control means for controlling the still image photographing means so as to generate a still image by combining the third image generated by changing to the third exposure.

  According to the present invention, a still image having a dynamic range expansion effect equivalent to that of an HDR still image obtained by HDR still image shooting can be obtained during shooting of an HDR moving image.

It is a block diagram which shows the internal structural example of the digital camera which concerns on each embodiment. 6 is a flowchart illustrating an example of a processing procedure for shooting an HDR still image during shooting of an HDR video in the first embodiment. FIG. 6 is a diagram for describing processing timing when an HDR still image is shot during shooting of an HDR video in the first embodiment. 9 is a flowchart illustrating an example of a processing procedure for capturing an HDR still image while capturing an HDR moving image in the second embodiment. In 2nd Embodiment, it is a figure for demonstrating the timing of the process in the case of image | photographing a HDR still image during imaging | photography of a HDR moving image. 14 is a flowchart illustrating an example of a processing procedure for capturing an HDR still image while capturing an HDR moving image in the third embodiment. FIG. 10 is a diagram for describing processing timing in the case of shooting an HDR still image during shooting of an HDR video in the third embodiment.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of a digital camera 100 according to the present embodiment.
In FIG. 1, the digital camera 100 includes a CPU 101, a nonvolatile memory 102, a memory 103, an imaging unit 104, an image processing unit 105, a display 106, an input unit 107, a drive device 108, and a communication I / F 110. Each of these components is connected to the internal bus 150, and each unit connected to the internal bus 150 can exchange data with each other via the internal bus 150.

  The non-volatile memory 102 is composed of, for example, a ROM, and the non-volatile memory 102 stores image data, audio data, other data, various programs for operating the CPU 101, and the like. The memory 103 is composed of, for example, a RAM (a volatile memory using a semiconductor element). The CPU 101 controls each unit of the digital camera 100 using the memory 103 as a work memory, for example, according to a program stored in the nonvolatile memory 102.

  The imaging unit 104 includes a lens group including a zoom lens and a focus lens, a shutter, an imaging element such as a CCD or CMOS element, and the like. The image processing unit 105 performs various image processing on the image data stored in the nonvolatile memory 102 or the recording medium 109 based on the control of the CPU 101. The image processing performed by the image processing unit 105 includes A / D conversion processing, D / A conversion processing, image data encoding processing, compression processing, decoding processing, enlargement / reduction processing (resizing), noise reduction processing, and color conversion. Processing etc. are included. The image processing unit 105 may be configured by a dedicated circuit block for performing specific image processing. Depending on the type of image processing, the CPU 101 can perform image processing according to a program without using the image processing unit 105. is there.

  The display 106 displays an image or a GUI screen constituting a GUI (Graphical User Interface) based on the control of the CPU 101. The CPU 101 generates a display control signal according to a program, and controls each unit of the digital camera 100 so as to generate a video signal to be displayed on the display 106 and output it to the display 106. The display 106 displays a video based on the output video signal.

  The input unit 107 is an input device for receiving user operations including buttons, dials, touch panel joysticks, and the like. Note that the touch panel is an input device that is configured to be two-dimensionally superimposed on the display 106 so that coordinate information corresponding to the touched position is output. The buttons include a shutter button for instructing still image shooting, a button for starting and stopping moving image shooting, and a switch for switching between a normal shooting mode and a mode for shooting with an expanded dynamic range.

  The drive device 108 can be loaded with a recording medium 109 such as an SD card, and reads data from the loaded recording medium 109 or writes data to the recording medium 109 based on the control of the CPU 101. The communication I / F 110 is an interface for connecting to the external device 111 or the Internet and transmitting and receiving various data such as video signals, audio signals, files, and commands.

  In the digital camera 100 described above, it is possible to perform photographing using central one-point AF (autofocus) and face AF. The central one-point AF is to perform AF on one central position in the shooting screen, and the face AF is to perform AF on the face in the shooting screen detected by the face detection function. is there.

  Next, synthesis of an HDR (High Dynamic Range) image will be described. In the following description, the SW1 signal refers to a signal generated when the shutter button is operated halfway (so-called shooting preparation instruction). The SW2 signal is a signal generated when the operation of the shutter button is completed, that is, when the shutter button is fully pressed (shooting instruction).

  When an instruction to shoot a still image (HDR still image) whose dynamic range is expanded by the SW2 signal of the shutter operation is issued, the CPU 101 controls the image capturing unit 104 so that the main subject has proper exposure, thereby obtaining a proper exposure image. obtain. Next, the CPU 101 controls the imaging unit 104 so that a high-brightness subject such as a blue sky has proper exposure, and obtains an underexposed image. Further, the CPU 101 controls the imaging unit 104 so as to obtain an overexposed image so that a low-luminance subject that is blacked out at appropriate exposure is appropriately exposed.

  Based on the control of the CPU 101, the image processing unit 105 synthesizes the three types of exposure images obtained so that the high luminance part becomes an underexposed image and the low luminance part becomes an overexposed image. An image reproduced up to is generated. When shooting a moving image (HDR moving image) with an extended dynamic range, the CPU 101 captures and combines only images with appropriate exposure, underexposure, or overexposure, and maintains a constant frame rate. Control each part. In the present embodiment, it is assumed that a properly exposed image and an underexposed image are combined.

  Hereinafter, in the present embodiment, an example in which HDR still image shooting is performed during HDR movie shooting will be described. Hereinafter, the first exposure refers to proper exposure, the second exposure refers to underexposure, and the third exposure refers to overexposure.

FIG. 2 is a flowchart illustrating an example of a processing procedure for shooting an HDR still image during shooting of an HDR video in the present embodiment. In the example illustrated in FIG. 2, an example in which a moving image is captured and combined using the first exposure and the second exposure is illustrated.
First, in step S201, during HDR video shooting, the user waits until the shutter button is half-pressed by an operation of the user and the input unit 107 generates the SW1 signal. Then, when the CPU 101 receives the SW1 signal, in step S202, the third exposure necessary for photographing the HDR still image is calculated.

  In step S203, the process waits until the shutter button is fully pressed by the user operation and the input unit 107 generates the SW2 signal. Then, when the CPU 101 receives the SW2 signal, in step S204, the imaging unit 104 is controlled to perform shooting by switching the exposure to the third exposure, and the captured image is stored in the memory 103. When the shooting with the third exposure is completed, the CPU 101, in step S205, captured in the memory 103, the first and second exposure images (moving image frames) taken in the most recent time and the step S204. A captured image by the third exposure is read out. Then, the image processing unit 105 is controlled to perform HDR still image composition. Thereafter, in step S206, the CPU 101 generates a moving image frame corresponding to the frame whose exposure has been changed using a photographed image with appropriate exposure.

  FIG. 3A is a diagram for describing processing timing when an HDR still image is captured while capturing an HDR moving image of a combination of an appropriate luminance image and a low luminance image. In FIG. 3A, “M” indicates a proper exposure image generated by the first exposure, and “L” indicates an under-exposure image photographed so that the high-brightness subject is appropriate by the second exposure. Point to. In addition, “H” indicates an overexposed image captured so that a low-luminance subject is appropriate by the third exposure. Further, the horizontal axis in FIG. 3A indicates time.

  In HDR movie shooting, the time for one frame is divided into two, two types of images “M” and “L” are shot, and HDR movie combining processing is performed in parallel with the shooting of the next frame. When the CPU 101 receives the SW1 signal at time T1, the CPU 101 performs the exposure calculation for the HDR still image in parallel with the moving image shooting and the HDR moving image combining processing. Thereafter, when the SW2 signal is received at time T2, the CPU 101 stores the “M” and “L” images taken from time T3 in the memory 103 for HDR still image composition.

  Next, at time T <b> 4, the CPU 101 changes the “L” exposure of the next frame to “H”, performs shooting, and stores the “H” image in the memory 103. Next, the CPU 101 synthesizes an HDR still image using the obtained “M”, “L”, and “H” images. Further, since the “H” image whose exposure has been changed at time T4 cannot be used for synthesizing the HDR moving image, only the “M” image is used as the moving image frame at time T5.

FIG. 3B is a diagram for explaining the processing timing when the SW2 signal is obtained at a timing different from the example shown in FIG. 3A in the present embodiment.
For example, when the SW2 signal is input at time T2 ′ earlier than the time when the “L” image is captured, the CPU 101 stores the second exposure image captured at the time T3 ′ as an HDR still image composition image. Save to 103. Then, the “M” image of the next frame is stored in the memory 103, the exposure for “L” is changed to “H” at time T 4 ′, and the “H” image is stored in the memory 103. .

  As described above, according to the present embodiment, when a still image is shot during shooting of an HDR moving image, an image shot with more types of exposure than when shooting a moving image can be obtained. For this reason, an HDR still image with a high dynamic range expansion effect can be obtained.

(Second Embodiment)
In the present embodiment, a shooting process for further reducing the time lag of still image shooting when HDR still image shooting described in the first embodiment is performed will be described. The internal configuration of the digital camera according to this embodiment is the same as that shown in FIG. Hereinafter, differences from the first embodiment will be described.

  FIG. 4 is a flowchart illustrating an example of a processing procedure for shooting an HDR still image during shooting of an HDR video in the present embodiment. Note that steps S401 to S402 and steps S405 to S407 are the same as steps S201 to S202 and steps S204 to S206 of FIG. 2 described in the first embodiment, respectively, and thus description thereof is omitted.

  After receiving the SW1 signal and calculating the third exposure necessary to capture the HDR still image, in step S403, the CPU 101 stores the image captured at the second exposure in the memory 103 for the HDR still image. To do. Each time the next frame is shot, in step S404, the CPU 101 determines whether or not the SW2 signal is received. As a result of the determination, if the SW2 signal is received, the process proceeds to step S405. On the other hand, if the result of determination in step S404 is that the SW2 signal has not been received, processing returns to step S403, and the image captured with the second exposure already stored in the memory 103 is updated to a new image.

FIG. 5 is a diagram for describing processing timing in the case where an HDR still image is shot during shooting of an HDR moving image in the present embodiment.
When the CPU 101 receives the SW1 signal at time T1, the CPU 101 saves or updates the “L” image to be captured thereafter in the memory 103 for HDR still image composition. Then, when the SW2 signal is received at time T2, as in the first embodiment, the CPU 101 causes the imaging unit 104 to capture an “H” image at time T4 in capturing the next frame starting from time T3. Control. The CPU 101 uses the “L” image stored in the memory 103, the “H” image obtained by the exposure change process, and the “M” image started to be captured at the time T3, to generate an HDR still image at the time T5. The synthesis of

  As described above, according to the present embodiment, after receiving the SW1 signal, the image by the second exposure is stored as an image for HDR still image composition. Thereby, it is possible to further shorten the time lag from the shutter operation to the acquisition of the image for synthesis than when the image for HDR still image synthesis is taken after receiving the SW2 signal.

(Third embodiment)
In the processing procedure illustrated in FIG. 3A, an image “M” is captured between the time when the “L” image used to synthesize the HDR still image and the time when the “H” image is acquired. For this reason, there is a possibility that the composition process may fail when shooting a fast-moving subject. Therefore, in the present embodiment, a process of acquiring images for HDR still image synthesis by continuous shooting in a procedure similar to the HDR still image shooting in the first embodiment will be described. The internal configuration of the digital camera according to this embodiment is the same as that shown in FIG. Hereinafter, differences from the first embodiment will be described.

  FIG. 6 is a flowchart illustrating an example of a processing procedure for shooting an HDR still image during shooting of an HDR moving image according to the present embodiment. Note that steps S601 to S603 and steps S605 to S607 are the same as steps S201 to S203 and steps S204 to S206 of FIG. 2 described in the first embodiment, respectively, and thus description thereof is omitted.

  When the CPU 101 receives the SW2 signal, in step S604, the CPU 101 determines whether or not an “L” image, that is, a third exposure image is continuously obtained. As a result of this determination, if images of the third exposure are obtained continuously as in the example shown in FIG. 3B, the process proceeds to step S605 as it is. On the other hand, as a result of the determination in step S604, if images of the third exposure cannot be obtained continuously as in the example shown in FIG. 3A, in step S608, the CPU 101 performs shooting with the first exposure, The order is switched between shooting with the second exposure. Thereafter, the process proceeds to step S605.

FIG. 7 is a diagram for explaining processing timing in the case where HDR still images are captured during HDR movie capturing in the present embodiment. Hereinafter, differences from the example shown in FIG.
When the CPU 101 receives the SW2 signal at time T2, the CPU 101 determines whether or not the HDR still image composition image can be continuously shot from the next shooting process. Since the storage of the HDR still image starts from time T3, the CPU 101 changes the shooting order of moving image shooting of frames after time T4, and then changes the shooting of the “L” image to the shooting of the “H” image. Switch. As a result, images for HDR still image composition can be obtained by continuous shooting, and the success rate of combining HDR still images in shooting a fast-moving subject can be increased.

  As described above, according to the present embodiment, it is determined whether HDR still image composition images can be continuously shot after the shutter is fully pressed. When continuous shooting is not possible, the order of shooting with the first exposure and shooting with the second exposure is switched, and shooting with the second exposure is changed to shooting with the third exposure. Thereby, the time lag can be further reduced and images for HDR still image composition can be continuously taken.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed. In this case, the program and a computer-readable storage medium storing the program constitute the present invention.

101 CPU
Reference Signs List 103 Memory 104 Imaging unit 105 Image processing unit 107 Input unit

Claims (9)

Shooting with a first exposure to generate a first frame and shooting with a second exposure different from the first exposure to generate a second frame, the first frame and the first frame Moving image capturing means for generating a moving image composed of frames obtained by combining the two frames;
The first exposure is taken to generate a first image, the second exposure is taken to produce a second image, and the second exposure is different from the first exposure and the second exposure. A still image capturing means for generating a third image by shooting at an exposure of 3, and generating a still image obtained by combining the first image, the second image, and the third image;
Instructing means for instructing shooting of a still image by the still image shooting means during shooting of a moving image by the moving image shooting means;
In response to a shooting instruction by the instruction unit, the moving image shooting unit is controlled to generate a third image by the third exposure, and the first frame generated by the first exposure; Control the still image photographing means to generate a still image by combining the second frame generated by the second exposure and the third image generated by changing to the third exposure. And an imaging device.   The control means includes a first frame and a second frame generated by the moving image shooting means after the shooting instruction is given by the instruction means, and a third frame generated by changing to the third exposure. The imaging apparatus according to claim 1, wherein the still image capturing unit is controlled to generate a still image by combining the image. Storage means for storing a second frame generated by photographing with the second exposure;
In response to a shooting instruction from the instruction unit, the control unit converts the first frame generated after the instruction, the second frame stored in the storage unit, and the second exposure into the first frame. The image pickup apparatus according to claim 1, wherein the still image capturing unit is controlled to generate a still image by combining the third image generated by changing to an exposure of 3.   The control means generates a second frame based on the second exposure after the moving image photographing means generates a first frame based on the first exposure after an instruction for photographing is made by the instruction means. In this case, the moving image photographing unit is controlled so that the second exposure is changed to the third exposure after the order of photographing with the first exposure and photographing with the second exposure is changed. The imaging apparatus according to claim 1, wherein:   5. The imaging apparatus according to claim 1, wherein the second exposure is underexposure and the third exposure is overexposure. 6.   6. The moving image photographing means according to claim 1, wherein the first frame is a moving image frame while the second exposure is changed to the third exposure. The imaging device according to item. Shooting with a first exposure to generate a first frame and shooting with a second exposure different from the first exposure to generate a second frame, the first frame and the first frame A moving image shooting process for generating a moving image composed of frames obtained by combining the two frames;
The first exposure is taken to generate a first image, the second exposure is taken to produce a second image, and the second exposure is different from the first exposure and the second exposure. A still image capturing step of generating a third image by shooting at an exposure of 3, and generating a still image obtained by combining the first image, the second image, and the third image;
An instruction step for instructing shooting of a still image by the still image shooting step during shooting of the moving image in the moving image shooting step;
In response to a shooting instruction by the instruction step, the moving image shooting step is controlled to generate a third image by the third exposure, and the first frame generated by the first exposure; The still image shooting process is controlled so as to generate a still image by combining the second frame generated by the second exposure and the third image generated by changing to the third exposure. And a control step for controlling the imaging apparatus. Shooting with a first exposure to generate a first frame and shooting with a second exposure different from the first exposure to generate a second frame, the first frame and the first frame A moving image shooting process for generating a moving image composed of frames obtained by combining the two frames;
The first exposure is taken to generate a first image, the second exposure is taken to produce a second image, and the second exposure is different from the first exposure and the second exposure. A still image capturing step of generating a third image by shooting at an exposure of 3, and generating a still image obtained by combining the first image, the second image, and the third image;
An instruction step for instructing shooting of a still image by the still image shooting step during shooting of the moving image in the moving image shooting step;
In response to a shooting instruction by the instruction step, the moving image shooting step is controlled to generate a third image by the third exposure, and the first frame generated by the first exposure; The still image shooting process is controlled so as to generate a still image by combining the second frame generated by the second exposure and the third image generated by changing to the third exposure. A control program for causing a computer to execute a control process.   A computer-readable storage medium storing the program according to claim 8.

Images