JP2018182678A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of acquiring a proper captured image without regard to the luminance of a subject.SOLUTION: The imaging apparatus which performs imaging with a rolling shutter method, comprises: imaging means which acquires a stroboscope light emission image and a stroboscope light non-emission image; pixel unit value acquisition means which acquires a pixel unit value of each pixel unit with respect to at least one of the stroboscope light emission image and the stroboscope light non-emission image; and synthesizing means which synthesizes the stroboscope light emission image and the stroboscope light non-emission image on the basis of the pixel unit values.SELECTED DRAWING: Figure 1

Description

  BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to an imaging apparatus having a function of performing flash photography using a rolling shutter method.

  Conventionally, in a CMOS imaging device, it is known that an exposure period is shifted for each pixel line because shooting is performed by a rolling shutter method. In addition, it is known that when the shutter speed is high, pixel lines on which strobe light is reflected and pixel lines not to be reflected are generated due to the deviation of the exposure period.

  Therefore, in Patent Document 1, the subject is obtained by obtaining the difference between the first captured image in which the electronic shutter is opened with all the pixel lines open simultaneously and the second captured image in which the flash is not emitted. An imaging device (corresponding to the imaging device of the present application) that generates an image signal representing the image signal has been proposed.

JP, 2009-81808, A

  However, in the imaging device (imaging device) of Patent Document 1, the shutter open time differs for each pixel line in the first imaging. As a result, there is a possibility that the luminance of a relatively bright part in the subject is saturated particularly in a pixel line where the shutter open time is long. As a result, the value of the non-flash emission image is subtracted from the value of the saturated image, which may result in failure to obtain a normal image.

  In view of the above problems, the main object of the present invention is to eliminate the difference in luminance between pixel lines and to acquire an appropriate photographed image regardless of the luminance of the subject even when stroboscopic imaging is performed by the rolling shutter method. Providing an imaging device capable of

  An imaging apparatus according to a first aspect of the present invention includes a strobe and performs shooting using a rolling shutter method. The imaging device irradiates the subject with the strobe light and obtains at least one strobe light emission image in which the irradiation light of the strobe light reaches all pixel lines, and at least one of the subject without irradiating the subject with the strobe light. Image pickup means for acquiring a non-flash-emission image for each time, and pixel unit value acquisition means for acquiring a pixel unit value for each pixel unit including at least one pixel for at least one of the strobe-emission image and the non-flash-emission image; And image combining means for combining the strobe light emission image and the non-strobe light emission image based on the pixel unit value.

  Here, the “pixel unit value” includes, for example, a luminance value based on at least one pixel included in the pixel unit, and a pixel value based on at least one pixel included in the pixel unit.

  According to the above aspect, the image combining unit obtains pixel unit values for evaluating each pixel unit for each pixel unit, and performs image combining processing based on the pixel unit values. As a result, image combining processing can be performed according to the state of the pixel unit (for example, the degree of suitability for luminance), and an appropriate image can be obtained. Specifically, for example, when there is a pixel unit including a pixel deviating from a suitable luminance such as a saturated pixel in the strobe light emission image, the deviation state of the luminance is eliminated based on the pixel unit value. Image synthesis processing can be performed.

  The combining unit may perform image combining processing for each of the pixel units.

  As a result, since the composition based on the pixel unit value can be performed for each pixel unit, it is possible to perform the image composition process reflecting the state of the pixel unit more accurately.

  The apparatus further comprises an exposure time calculation unit that calculates an exposure time, and the image pickup unit acquires an image by an exposure time longer than the exposure time calculated by the exposure time calculation unit when acquiring a strobe light emission image. You may

  With such a configuration, even when the exposure time calculated by the exposure time calculation means is short, it is possible to prevent the occurrence of the pixel line on which the strobe light is reflected and the pixel line not to be reflected.

  The image combining unit may combine corresponding pixel units of the strobe light emission image and the strobe non-light emission image at a ratio according to the pixel unit value.

  As described above, by performing the image combining process at a ratio according to the pixel unit value, it is possible to obtain an appropriate image and to make it difficult to cause a step between pixel units of the combined image. Specifically, for example, as composition at a ratio according to the pixel unit value, composition according to the pixel unit value for each of the pixel unit of the strobe light emission image at the corresponding position and the pixel unit of the strobe non-light emission image After multiplying the coefficients, they may be added.

  The apparatus further comprises luminance information acquisition means for acquiring luminance information of a subject, and based on the luminance information of the subject, at least one of a first photographing mode for acquiring the strobe light emission image, the strobe light emission image and the strobe non-light emission image. Control means for selecting a shooting mode to be applied from the second shooting mode for obtaining each time or the third shooting mode for obtaining the non-flash image and controlling the flash and the imaging means based on the selected shooting mode The image combining process may be performed when the second imaging mode is selected.

  In this configuration, when the appropriate shutter open time determined based on the luminance evaluation result is ensured only in a period in which the strobe light is reflected on all pixel lines (the subject is sufficiently dark), the strobe is used in the first photographing mode. Shoot the flash. In addition, if the appropriate shutter opening time determined based on the luminance evaluation results does not ensure a period in which the flash is reflected in all pixel lines (if the subject is somewhat bright but is photographed dark when the flash is not emitted), 1 In the second shooting mode, shooting is performed in the second shooting mode, which includes stroboscopic shooting with a shutter opening time longer than normal. Then, when the subject is brighter, the flash non-flash shooting is performed in the third shooting mode. Thereby, the photographing means for acquiring an appropriate photographed image can be optimized.

  According to the present invention, even when stroboscopic imaging is performed by the rolling shutter method, an appropriate captured image can be acquired regardless of the luminance of the subject.

It is a block diagram showing a schematic structure of an imaging device concerning an embodiment. It is a figure which shows an example of the allocation state of the evaluation area in an imaging surface. It is a flowchart which shows a part of operation | movement of an imaging device. It is a flowchart which shows a part of operation | movement of an imaging device. It is a flowchart which shows operation | movement of the taking-in process of a still image. It is a figure for demonstrating the relationship between shutter speed and imaging | photography mode. It is a figure for demonstrating the imaging | photography operation | movement of strobe light emission imaging | photography. It is a figure which shows an example of the synthetic | combination coefficient used for an image synthetic | combination process. It is a figure for demonstrating an image synthetic | combination process. It is a figure which shows an example of the pixel value before (a) compositing, the pixel value after compositing, and the pixel value after (c) compositing about an image compositing process.

  Hereinafter, embodiments of the present invention will be described in detail based on the drawings. The following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the invention, its application or its application.

<Configuration of Imaging Device>
FIG. 1 is a block diagram showing the configuration of an imaging device according to the embodiment. As shown in FIG. 1, the imaging device 10 includes a focus lens 12 and an aperture unit 14 which are driven by a driver 18a and a driver 18b, respectively. The optical image that has passed through these members is irradiated onto the imaging surface of the imager 16 and subjected to photoelectric conversion. Thereby, charges corresponding to the optical image are generated.

  As shown in FIG. 2, an evaluation area EVA is assigned to the center of the imaging surface of the imager 16. The evaluation area EVA includes 256 divided areas divided into 16 in each of the horizontal direction and the vertical direction.

  Returning to FIG. 1, the driver 18c exposes the imaging surface of the imager 16 under the control of the CPU 26, which will be described later, in response to a vertical synchronization signal Vsync periodically generated from an SG (signal generator) not shown. The charges generated on the imaging surface are read out in a raster scan manner. Raw image data based on the read charge is periodically output from the imager 16. In FIG. 1, the imaging means is constituted by the focus lens 12, the aperture unit 14, the imager 16 and the drivers 18a, 18b and 18c. However, the configuration of the imaging means is not limited to the configuration of FIG. Specifically, if the imaging system is a rolling shutter system and is configured to be able to perform both flash non-flash shooting and flash flash photography described later, configurations other than FIG. 1 (for example, FIG. A configuration in which a plurality of functional blocks are combined into one may be used.

  The preprocessing circuit 20 subjects the raw image data output from the imager 16 to processing such as digital clamping, pixel defect correction, gain control and the like. The raw image data subjected to these processes is written to the SDRAM 32 through the memory control circuit 30. Further, the preprocessing circuit 20 executes a simple RGB conversion process for converting raw image data into RGB data in a simple manner.

 The AE evaluation circuit 22 integrates RGB data belonging to the divided area (see FIG. 2) of the evaluation area EVA among the RGB data generated by the preprocessing circuit 20 every time the vertical synchronization signal Vsync is generated. As a result, 256 integral values, that is, 256 luminance evaluation values Ey are output from the AE evaluation circuit 22 in response to the vertical synchronization signal Vsync.

  The AF evaluation circuit 24 integrates high frequency components of RGB data belonging to the divided area of the area EVA (see FIG. 2) among the RGB data generated by the preprocessing circuit 20 every time the vertical synchronization signal Vsync is generated. As a result, 256 integrated values, that is, 256 focus evaluation values Ef are output from the AF evaluation circuit 24 in response to the vertical synchronization signal Vsync.

  The CPU 26 has a function of controlling each component of the imaging device 10 including the drivers 18a, 18b, and 18c. The operation of the CPU 26 will be described in detail in "Operation of the Imaging Device" described later. The control program executed by the CPU 26 is stored in the flash memory 46.

  The key input device 28 is provided with a shutter button 28 sh, a mode change switch 28 md for switching the photographing mode, and an OK button 28 k for performing an input operation as to whether or not the user may proceed with the process.

  The post-processing circuit 34 reads the raw image data stored in the SDRAM 32 through the memory control circuit 30, and performs color separation processing, white balance adjustment processing, and YUV conversion processing on the read raw image data. The memory control circuit 30 writes the image data of YUV format generated thereby to the SDRAM 32. Furthermore, the image data of the YUV format is converted into a JPEG image file under the control of the CPU 26.

  The LCD driver 36 repeatedly reads out YUV format image data stored in the SDRAM 32 via the memory control circuit 30, and drives the LCD monitor 38 based on the read out image data. As a result, a through image representing a scene captured on the imaging surface is displayed on the monitor screen.

  The memory I / F 42 receives control of the CPU 26 and records the JPEG image file read from the SDRAM 32 via the memory control circuit 30 in the recording medium 44.

  Furthermore, the imaging device 10 includes a flash 62 and a flash driver 61 that drives the flash 62 under the control of the CPU 26. When shooting is performed by causing the flash 62 to emit light (hereinafter referred to as flash shooting), the CPU 26 supplies a trigger signal to the flash driver 61. As a result, the flash 62 emits light in synchronization with the main exposure of the imager 16, and the JPEG image file of the subject generated by the flash shooting is recorded on the recording medium 44 by the memory I / F 42 under the control of the CPU 26. .

<Operation of Imaging Device>
Hereinafter, the operation of the imaging device 10 will be described in detail with reference to the flowcharts of FIGS. 3 to 5. 3 and 4 are flowcharts showing a part of the operation of the imaging device 10. FIG. 5 is a flowchart showing the operation of the still image capturing process described later.

  First, when the shutter button 28 sh is half-depressed, the CPU 26 takes in the brightness evaluation value Ey from the AE evaluation circuit 22 and takes in the focus evaluation value Ef from the AF evaluation circuit 24 (step S 11).

  In step S12, the CPU 26 calculates an optimal shutter speed (optimum exposure period) and an optimal aperture value of the aperture unit 14 based on the taken-in luminance evaluation value Ey. Thereafter, the CPU 26 controls the aperture driver 18 b so that the aperture value of the aperture unit 14 becomes the above-mentioned optimum aperture value. Similarly, the driver 18c is set based on the calculated optimal shutter speed.

  In step S13, the CPU 26 performs focus adjustment based on the acquired focus evaluation value Ef. Specifically, the CPU 26 controls the driver 18a so that the focus evaluation value Ef becomes large, and sets the focus lens 12 at the in-focus position.

  In step S14, the CPU 26 determines whether or not the focus adjustment has been completed, that is, whether or not the focus lens 12 has been set to the in-focus position, and determines that the focus adjustment has not been completed yet. The process proceeds to step S15.

  In step S15, the CPU 26 determines whether the operation of the shutter button 28sh is released, specifically, whether the user's finger is released from the shutter button 28sh. Here, if it is determined that the operation of the shutter button 28 sh has been released, the CPU 26 ends (discontinues) the processing shown in this flowchart. That is, when the user's finger is released from the shutter button 28 sh before the focus adjustment is completed, shooting is not performed.

  When the operation of the shutter button 28 sh is not released in step S15, the CPU 26 returns to step S14. Then, if it is determined that the focus adjustment is completed in this step S14, the flow proceeds to step S21 of FIG.

  Also in step S21, the CPU 26 determines whether or not the operation of the shutter button 28sh is canceled as in step S15 described above, and when the operation of the shutter button 28sh is canceled, the process shown in the flowchart Quit (discontinue). On the other hand, when the operation of the shutter button 28 sh is not released, the CPU 26 determines whether or not the shutter button 28 sh in step S 22 is full-pressed. Then, the CPU 26 repeatedly executes the determination of step S21 and step S22 until the operation of the shutter button 28sh is canceled or the shutter button 28sh is full-pressed.

  Then, if it is determined that the shutter button 28 sh is full-pressed, the flow proceeds from step S 22 to step S 23, and the CPU 26 proceeds to the still image capturing process.

-Still image capture process-
Hereinafter, the still image capturing process of FIG. 5 will be described in detail.

  In step S30 of FIG. 5, in response to full depression of the shutter button 28sh, the CPU 26 enters selection processing of the imaging mode to be applied. Specifically, the CPU 26 performs, as the shooting mode, a single-flash-flash shooting mode (steps S41 to S42) for performing flash-flash shooting as the first shooting mode (steps S41 to S42), and flash-flash shooting and non-strobe shooting as the second shooting mode. Select one of the two-flash flash shooting mode (steps S51 to S54) for shooting twice for flash shooting and the non-flash photography mode (steps S61 to S62) for non-flash photography as the third shooting mode Do.

= Selection of shooting mode =
In the following, selection of the shooting mode will be specifically described with reference to FIGS. 6 and 7. FIG. 6 exemplifies the relationship between the optimum shutter speed SS (horizontal axis) calculated in the above-mentioned step S12 (see FIG. 3) and the shutter speed SS (vertical axis) which is actually photographed. Further, the relationship between the optimal shutter speed SS (horizontal axis) and the imaging mode (imaging method) to be selected is illustrated. FIG. 7 shows the photographing operation of the strobe light emission photographing at each shutter speed in the strobe light emission one time photographing mode and the strobe light emission second time photographing mode. In FIG. 6, it is assumed that the shutter speed SS2 is the limit value (threshold value) of the shutter speed capable of causing the irradiation light of the strobe to reach all the pixel lines (see FIG. 7B described later). In FIG. 6, Ey1, Ey2, Ey3 and Ey4 described in parentheses below the shutter speeds SS1, SS2, SS3 and SS4 are luminance evaluation values corresponding to the respective shutter speeds SS1, SS2, SS3 and SS4. It shall be Ey.

(1 flash shooting mode)
In step S30 of FIG. 5, when the shutter speed is SS2 or less, that is, when the luminance evaluation value Ey is Ey2 or less, the CPU 26 performs control relating to the single flash light emission mode.

  Specifically, in step S41, when the vertical synchronization signal Vsync is input, the CPU 20 instructs the driver 18c to perform one frame worth of main exposure and all pixel readout. Further, the CPU 26 supplies the strobe driver 61 with a trigger signal for causing the strobe 62 to emit light. FIGS. 7A and 7B respectively show the photographing operation of the strobe light emission photographing when the shutter speed is SS1 (here, SS1 <SS2) and the shutter speed is SS2. As shown in FIG. 7A, since the shutter speed SS1 is slower than the shutter speed SS2, the exposure time T1 is sufficiently secured, and the irradiation light of the strobe 62 can reach all pixel lines.

  Returning to FIG. 5, after the process of step S41, the CPU 20 causes the LCD monitor 38 to display a preview image in step S42. Thereafter, the flow proceeds to the recording process of FIG. 4 (step S27), and the image file of the subject is recorded on the recording medium 44 to complete a series of photographing processes.

(Strobe 2 times shooting mode)
In step S30 in FIG. 5, when the shutter speed is faster than SS2 and slower than SS4, that is, when the luminance evaluation value Ey is larger than Ey2 and smaller than Ey4, the CPU 26 performs control relating to the flash light emission twice shooting mode. Do.

  Specifically, in step S51, when the vertical synchronization signal Vsync is input, the CPU 20 sets the shutter speed to SS2 and instructs the driver 18c to perform one frame worth of main exposure and all pixel readout. Further, the CPU 26 supplies a trigger signal for causing the strobe driver 62 to emit light in addition to the above command. The first captured image captured here is temporarily saved in the image storage area of the SDRAM 32.

  After the photographing processing in step S51, in step S52, the CPU 26 sets the exposure (the shutter speed SS and the aperture amount to the shutter speed (optimum exposure period) and the aperture value reset under the flash non-emission condition. Change at least one).

  In FIG. 7C, the first shooting (strobe flash shooting) is performed at the shutter speed SS2, and then the second shooting (strobe non-flash shooting) is performed at the reset shutter speed SS3 (here, SS3> SS2). Shows the shooting operation in the case of. As shown in FIG. 7C, the strobe illumination light is made to reach all pixel lines at a shutter speed SS2 to obtain a strobe light emission photographed image, and then a strobe non-light emission photographed image photographed at a shutter speed SS3 (Hereafter, it is abbreviated and it is called a non-light emission photography picture). The second shot image is saved in the image storage area of the SDRAM 32 in the same manner as the first shot image.

  Note that the shutter speed for flash non-flash shooting may be the optimum shutter speed (SS3 in FIG. 6) calculated in step S12 (see FIG. 3), or may be a shutter speed SS different from that. Good. Specifically, the shutter speed SS for non-flash shooting may be changed according to the subject. For example, based on the distance between the subject and the imaging device 10, the shutter speed SS for non-flash shooting may be changed. More specifically, when the subject is in a position where the light of the flash 62 can reach the limit, the shutter speed is decreased (for example, the shutter speed in FIG. 6 is set to SS31). On the other hand, when the subject is close to the imaging device 10, the shutter speed is increased (for example, the shutter speed in FIG. 6 is set to SS32).

  When the second shooting is completed, in step S54, an image combining process is performed on the image data acquired in steps S51 and S53. The image composition processing is executed, for example, in a work area in the SDRAM 32 under the control of the CPU 26 and the memory control circuit 30, and composite image data created thereby is stored in another image storage area in the SDRAM 32.

= Image composition processing =
Hereinafter, the image combining process in step S54 (FIG. 5) will be described in detail with reference to FIGS. 8 to 10.

  The image combining process is performed based on the pixel unit value for each pixel unit in at least one of the strobe light emission photographed image and the non-light emission photographed image photographed in the strobe light emission twice photographing mode.

  The pixel unit indicates a minimum unit for at least one of acquisition of pixel unit values obtained by dividing all pixels constituting an image and image combining processing. And the division number can be set arbitrarily. For example, all pixels may be divided such that one pixel is included in the pixel unit, or all pixels are divided such that a plurality of adjacent pixels (for example, about 2 to a dozen or so pixels) are included. You may do it. For example, the above division is made to include at least one R pixel (hereinafter simply referred to as R pixel), at least one G pixel (hereinafter referred to simply as G pixel), and at least one B pixel (hereinafter simply referred to as B pixel) May be performed. In addition, the sizes of the pixel units to be divided, that is, the number of pixels included in each pixel unit may be different among the pixel units. That is, the number of pixels included in the pixel unit may be set in advance, or may be determined based on a process of obtaining a pixel unit value, which will be described later, or other operations and evaluations.

  The pixel unit value indicates a value that serves as an index of the image combining process obtained from at least one of the strobe light emission photographed image and the non-light emission photographed image. For example, as the pixel unit value, a value related to the brightness of the pixel unit (for example, a brightness value or a pixel value) can be used. More specifically, as the pixel unit value, a luminance value calculated based on any R pixel, any G pixel, and any B pixel in the pixel unit can be used. When a plurality of RGB pixels are included in a pixel unit as a pixel unit value, an average value of luminance values calculated for each RGB pixel may be determined, and the average value may be used. You may make it use the luminance value calculated from RGB pixel. Further, as the pixel unit value, the pixel value of any G pixel contained in the pixel unit may be used, or an operation value calculated based on the pixels of RGB pixels may be used. Further, in the case where a plurality of G pixels are included in a pixel unit, the average of the pixel values of the plurality of G pixels may be obtained, and the average value may be used. In addition, pixel unit values of other adjacent pixel units may be referred to as pixel unit values. By referring to the pixel unit value of the adjacent pixel unit, it is possible to reduce the amount of calculation when calculating the pixel unit value.

  In the following, image combining processing using combining coefficients will be described as a specific example of the image combining processing. In the following description, an example is shown in which the pixel value of any pixel in the pixel unit is used as the pixel unit value. Further, it is assumed that the number of pixels included in the pixel unit for obtaining the pixel unit value is the same as the number of pixels included in the pixel unit for performing the image combining process. However, as described above, the pixel unit value is not limited to the pixel value.

  FIG. 8 is a diagram showing an example of the relationship between pixel values and synthesis coefficients. The data of the synthesis coefficient as shown in FIG. 8 is stored in advance in, for example, the flash memory 46 or the like.

  First, the CPU 26 aligns the strobe light emission photographed image and the non-light emission photographed image. The alignment can use the same method as the prior art. For example, a method of aligning the positions of edges of characteristic shapes and colors can be used.

  Next, the CPU 26 determines the pixel value to be applied to the image combining process for each pixel unit in the strobe light emission photographed image and the non-light emission photographed image after the alignment, that is, for each pixel unit in the image after the image synthesis processing. Do. In other words, the CPU 26 determines what ratio (composition coefficient) to combine the pixel unit of the strobe light emission photographed image and the pixel unit of the non-light emission photographed image.

  In FIG. 8, as the pixel value applied to the pixel unit (hereinafter simply referred to as the pixel value) decreases, the composition ratio of the pixel unit of the strobe light emission photographed image is increased, while the pixel unit of the strobe light non-light emission photographed image The synthesis ratio of is to be low. Conversely, as the pixel value increases, the composition ratio of the pixel units of the non-flash-emission image is increased, while the composition ratio of the pixel units of the flash-emission image is decreased.

  In FIG. 10, (a) pixel values before combining, (b) combining coefficients used in combining, and (c) pixels after combining when image combining processing is performed on the arrow portion in the image of FIG. An example of the value is shown.

  As shown in FIG. 10A, the pixel values of the non-emission captured image and the strobe emission captured image are relatively high for the white portion in the image of FIG. 9 before combining. On the other hand, for dark colored portions of the image, the pixel values of the non-emission captured image and the strobe emission captured image are relatively low. In addition, throughout the whole, the flash-flashed photographed image is higher than the non-flashed photographed image.

  When composition is performed on this image using the composition coefficient shown in FIG. 8, as shown in FIG. 10B, for the white part of the image, the composition coefficient of the non-emission captured image is high, The composition coefficient of the flash photography image is low. That is, for pixel units where the brightness of the strobe light emission photographed image may be saturated, the composition ratio of the pixel units of the non-light emission photographed image is increased. On the other hand, with regard to dark colored portions of the image, the composition ratio of the non-emission captured image is low while the composition ratio of the strobe emission captured image is high. As described above, with regard to a pixel unit having a low pixel value, an appropriate image can be acquired even in a case where the luminance of the subject is low, by increasing the composition ratio of the pixel units of the stroboscopic light emission photographed image.

  Referring back to FIG. 5, after the combining process in step S54, the flow proceeds to step S55, and the CPU 20 causes the LCD monitor 38 to display a preview image as in step S42. Thereafter, the flow proceeds to the recording process (step S27) of FIG. 4, and the recording process similar to the above-described operation is performed to complete a series of photographing processes.

  The image combining process may be performed as follows.

  For example, in FIG. 8, the combination coefficient changes linearly with respect to the pixel value, but is not limited thereto. For example, as the combination coefficient, a combination coefficient based on an arbitrary expression, a measurement result or the like can be applied. For example, a coefficient to which gamma correction or the like has been applied, or a coefficient based on a simulation or a measurement result by a real machine can be used as a combination coefficient.

  Further, FIG. 8 shows an example in which the pixel units of the flash light emission photographed image and the non-light emission photographed image are combined at a predetermined ratio, but one of the strobe light emission photographed image and the non-light emission photographed image is 0% and the other is You may use a synthetic | combination coefficient synthesize | combined by 100%. In this case, a pixel unit with a composition ratio of 100% is adopted for the image after the image composition processing.

  For example, of the pixel units of the non-emission captured image, the composition ratio may be set to 100% for a pixel unit whose pixel value is sufficiently high (above a predetermined threshold). Similarly, for example, among the pixels of the non-emission captured image, the pixel unit of the non-emission captured image as data of the pixel unit after the combining processing without performing the image combining processing on the pixel unit having a sufficiently high pixel value May be adopted. This is because pixel units with sufficiently high pixel values are often appropriate pixel units as they are in the non-emission image. As a result, the amount of processing related to the image combining processing can be reduced, and the processing can be speeded up.

  Further, for example, among pixel units of a strobe light emission photographed image, a pixel unit whose brightness (pixel value) is clearly saturated may not be used for the image combining process. In that case, for example, the non-emission image may be adopted as the pixel unit after the combining process. This makes it possible to prevent the composite image from being affected by the pixel unit saturated with luminance.

  Also, for example, with respect to each of the strobe light emission photographed image and the non-light emission photographed image, the pixel value of each pixel unit is classified into a plurality of stages, and the image combining processing is performed based on the reference or the combination coefficient based on the class value. May be performed. In addition, the image combining method may be changed based on whether at least one predetermined threshold value is provided for the pixel value of each pixel unit and whether the threshold value is exceeded or exceeded.

(No flash mode)
The CPU 26 performs still image capture without light emission of the flash 62 under the condition that it is clear that the subject can be photographed well without the flash 62 being emitted.

  Specifically, if the shutter speed is SS4 or higher in step S31 in FIG. 5, that is, if the luminance evaluation value Ey is Ey4 or higher, the flow proceeds to step S61, and the CPU 26 selects the strobe non-emission shooting mode ( In FIG. 5, the control related to the non-emission shooting mode is performed.

  In step S61, when the vertical synchronization signal Vsync is input, the CPU 20 instructs the driver 18c to perform one frame worth of main exposure and all pixel readout. At this time, the CPU 26 does not cause the strobe 62 to emit light, that is, does not supply a trigger signal to the strobe driver 61. After the process of step S61, the CPU 20 causes the LCD monitor 38 to display a preview image in step S62. Thereafter, the flow proceeds to the recording process (step S27) of FIG. 4, and the same recording process as described above is performed to complete a series of photographing processes.

  In the non-flash photography mode, two pictures with different luminances are taken, and the two pictures are the same as the process described in the "image combining process" of the "two-flash photography mode" described above. Image composition processing may be performed. In the case of this double shooting, for example, the shutter speed can be as indicated by the broken lines indicated by P1 and P2 in FIG.

  As described above, according to the present embodiment, when the stroboscopic light emission twice photographing mode is selected, the pixel unit value is acquired for each pixel unit of the stroboscopic light emission photographed image and the non-light emission photographed image, and the pixel unit value is acquired. Accordingly, the composition processing for each pixel unit is performed. As a result, even when strobe shooting is performed by the rolling shutter method, it is possible to obtain an appropriate composite processed image (captured image) regardless of the brightness of the subject.

<Other Embodiments>
Although the embodiments of the present invention have been described above, various modifications are possible.

  In the above-described embodiment, the CPU 26 sets the shutter speed relating to the flash emission shooting to SS2 in the flash emission twice shooting mode, but the present invention is not limited to this. For example, the shutter speed may be slower than SS2.

  Further, the order of shooting in the two-flash-flash shooting mode of the above embodiment may be either flash non-flash shooting or flash-flash shooting first. For example, when the subject is a person, it is possible to prevent the subject from moving after the first shooting by mistaking that the shooting is completed by performing the flash shooting after performing the non-flash shooting. it can. Also, for example, by performing non-flash shooting after performing flash shooting, it is possible to improve the tactile response to the shooting operation. Further, in the two-flash flash photographing mode, the number of times of shooting with non-flash and non-flash photography is not limited to one each. For example, a plurality of non-flash flash photography and a plurality of flash flash photography may be combined, and the above-described image combining process may be performed using an optimal image (for example, an image with less blurring) from among the photographed images. You may

  The CPU 26 may supply a trigger signal for causing the strobe driver 61 to emit light after the second photographing in the two-step strobe emission mode of the above embodiment. By performing such an operation, it is possible to prevent the subject from moving due to misunderstanding that the photographing has been completed in the first flash emission photographing.

  According to the present invention, even when the rolling shutter method and the flash light emission photographing are combined, an appropriate image can be obtained regardless of the brightness of the subject, which is extremely useful.

10 imaging device 22 AE evaluation circuit (brightness information acquisition means)
26 CPU (control means)

Claims (7)

An imaging device that has a strobe and performs shooting with a rolling shutter method,
The subject is illuminated with the strobe, and at least one strobe light emission image in which the illumination light of the strobe reaches all pixel lines is acquired, and the strobe is not illuminated at least once without the strobe. Imaging means for acquiring an image;
Pixel unit value acquisition means for acquiring a pixel unit value for each pixel unit including at least one pixel for at least one of the strobe light emission image and the strobe non-light emission image;
An image pickup apparatus comprising: image combining means for combining the strobe light emission image and the non-flash light emission image based on the pixel unit value. In the imaging device according to claim 1,
An image pickup apparatus characterized in that the image combining unit performs an image combining process for each of the pixel units. In the imaging device according to claim 1,
The image pickup apparatus, wherein the pixel unit value is a luminance value based on at least one pixel included in the pixel unit. In the imaging device according to claim 1,
The image pickup apparatus, wherein the pixel unit value is a pixel value based on at least one pixel included in the pixel unit. In the imaging device according to claim 1,
It further comprises exposure time calculation means for calculating the exposure time,
An imaging unit configured to acquire an image with an exposure time longer than an exposure time calculated by the exposure time calculation unit when acquiring the strobe light emission image; In the imaging device according to claim 1,
An image combining unit configured to combine corresponding pixel units of the strobe light emission image and the strobe non-light emission image at a ratio according to the pixel unit value. The imaging device according to any one of claims 1 to 6.
It further comprises luminance information acquisition means for acquiring luminance information of the subject,
A first photographing mode for acquiring the strobe light emission image based on luminance information of the subject; a second photographing mode for acquiring the strobe light emission image and the strobe non-light emission image at least once each; A control unit is further provided which selects a shooting mode to be applied from a third shooting mode for obtaining an image, and controls the strobe and the imaging unit based on the selected shooting mode.
An image pickup apparatus characterized in that the image combining unit performs the image combining process when the second shooting mode is selected.

Images