JP2016018033A - IMAGING DEVICE, ITS CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM - Google Patents

Abstract

An image pickup apparatus having an image pickup device having a focus detection pixel provides a mechanism that enables high-speed continuous shooting while performing focus adjustment and exposure processing during continuous shooting. An imaging apparatus includes a focus detection pixel that photoelectrically converts light beams passing through different pairs of pupil regions of an exit pupil region of a photographing optical system and outputs a pair of image signals, and an exit pupil of the photographing optical system. An image sensor 107 having an image pickup pixel that photoelectrically converts all passing light beams and outputs an image signal is provided. During continuous shooting in which the image sensor 107 is sequentially exposed, the focus detection unit 121b performs focus detection from the image signal of the focus detection pixel generated for each exposure, and the focus adjustment unit 126 determines the detection result of the focus detection unit 121b. Based on the exposure conditions set by the exposure processing unit 121d, the exposure processing unit 121d sets the exposure condition from the image signal generated for each exposure. Aperture adjustment is performed during exposure. [Selection] Figure 8

Description

  The present invention relates to an imaging apparatus having an imaging optical system such as a digital single-lens reflex camera.

  A single-lens reflex camera or the like has a mirror part composed of a main mirror and a sub mirror that can be driven. When the mirror is down, the subject light flux that has passed through the imaging optical system enters the phase difference detection AF unit and optical viewfinder, and when the mirror is up Subject light flux enters the imaging surface.

  In the case of continuous shooting with such a single-lens reflex camera, it is possible to perform phase difference AF using an AF unit of a phase difference detection method by raising and lowering the mirror portion between exposures to the image sensor. .

  However, in phase difference AF during continuous shooting, the mirror unit must be driven up and down to expose the AF unit and the image sensor. For this reason, even when the readout time of the image sensor is high, the drive time of the mirror unit is required, and high-speed continuous shooting is difficult.

  As means for solving the above problem, there has been proposed an imaging apparatus that performs continuous shooting by performing phase difference AF on an imaging surface using an imaging element having focus detection pixels (Patent Document 1). This imaging apparatus includes an imaging element having a focus detection pixel row in which a pair of pixels that receive each of the subject light fluxes that have passed through a pair of partial areas in the exit pupil of the imaging optical system are arranged in the horizontal direction. .

  At the time of continuous shooting, a phase difference AF calculation process (AF calculation) is performed using a signal generated by the focus detection pixel array by the main exposure to the image sensor, and a focus shift amount (defocus amount) is calculated. . After that, by moving the focus lens to the in-focus position based on the calculated defocus amount, phase difference AF can be performed during continuous shooting without performing up-down driving of the mirror unit.

JP 2009-109631 A

  However, Patent Document 1 does not disclose the exposure process. For this reason, AF during continuous shooting is possible, but since the exposure process is not performed, the exposure condition is fixed. Therefore, if the subject or shooting environment changes during continuous shooting, continuous shooting cannot be performed under suitable exposure conditions.

  Accordingly, an object of the present invention is to provide a mechanism that enables high-speed continuous shooting while performing focus adjustment and exposure processing during continuous shooting in an imaging device including an imaging device having focus detection pixels. And

  In order to achieve the above object, the present invention provides a focus detection pixel that photoelectrically converts light beams passing through different pairs of pupil regions of an exit pupil region of a photographing optical system and outputs a pair of image signals, and the photographing optical An image pickup apparatus including an image pickup device having an image pickup pixel that photoelectrically converts a light beam passing through the exit pupil of a system and outputs an image signal, and performs focus detection using an output signal from the image pickup device A focus adjusting means for driving the focus lens to adjust the focus based on a detection result of the focus detecting means, an exposure processing means for setting an exposure condition at the time of photographing, and the exposure set by the exposure processing means An aperture adjustment unit that adjusts the aperture based on conditions, a determination unit that determines whether or not continuous shooting is performed to sequentially expose the image sensor, and the focus detection when the determination unit determines that continuous shooting is performed. The stage performs focus detection from the image signal of the focus detection pixel generated for each exposure, and the focus adjustment unit performs focus adjustment during the exposure based on the detection result of the focus detection unit, and the exposure processing Control means for setting an exposure condition from an image signal generated for each exposure, and for controlling the aperture adjustment means to adjust the aperture during the exposure based on the exposure condition set by the exposure processing means And means.

  According to the present invention, in an imaging apparatus including an imaging device having focus detection pixels, high-speed continuous shooting can be performed while performing focus adjustment and exposure processing during continuous shooting.

1 is a block diagram illustrating a system configuration example of a digital camera that is an example of an embodiment of an imaging apparatus of the present invention. It is a block diagram which shows the structural example of an image pick-up element (solid-state image sensor). (A) is a top view of an imaging pixel of 2 rows x 2 columns, (b) is an AA line sectional view of (a). (A) is a top view of the pixel of 2 rows x 2 columns containing a focus detection pixel, (b) is the sectional view on the AA line of (a). (A) is a figure which shows a mode that a light beam is restrict | limited by the aperture_diaphragm | restriction of the imaging optical system in the position of an exit pupil surface with respect to the pixel of the center vicinity of an image pick-up element, (b) is an image pick-up element in (a). It is a figure explaining the change of the gravity center position by the vignetting in the exit pupil plane of the focus detection pixel in the center of. FIG. 5A is a diagram showing sensitivity distributions of the A image pixel, the B image pixel, and the imaging pixel in the cross section along the line AA in FIG. 5B when the aperture is opened, and FIG. It is a figure which shows the sensitivity distribution of A image pixel, B image pixel, and imaging pixel in the AA line cross section of FIG.5 (b) in the case. It is a figure which shows the example of arrangement | positioning of the imaging pixel and focus detection pixel in an image sensor. It is a figure which shows the operation | movement sequence at the time of the continuous shooting photography of a digital camera. It is a figure which shows the operation | movement sequence at the time of the continuous shooting of a digital camera when the range which performs an exposure process has limited image height with respect to the range of an imaging pixel. It is a figure which shows the operation | movement sequence at the time of the continuous shooting of the digital camera at the time of adding a subject face detection process. It is a flowchart figure explaining the continuous shooting imaging | photography operation | movement of a digital camera. It is a flowchart figure explaining the exposure process of step S1100 of FIG.

  Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a system configuration example of a digital camera which is an example of an embodiment of an imaging apparatus of the present invention. In the present embodiment, a digital camera in which a camera body and a photographing optical system are integrally formed is exemplified. However, a digital camera in which a lens barrel constituting the photographing optical system is mounted on the camera body in a replaceable manner. It can also be applied to cameras.

  As shown in FIG. 1, the digital camera 100 of the present embodiment includes a first lens group 101, a second lens group 103, and a third lens group 105 that are arranged in order from the subject side to the image plane side. The first lens group 101, the second lens group 103, and the third lens group 105 constitute a photographing optical system, and are respectively held by a lens holding unit (not shown) so as to be movable in the optical axis direction.

  A diaphragm / shutter 102 is provided on the subject side of the second lens group 103, and the diaphragm / shutter 102 adjusts the light amount during photographing by adjusting the aperture diameter. The aperture / shutter 102 and the second lens group 103 integrally move forward and backward in the optical axis direction, and perform a zooming operation in conjunction with the forward and backward movement of the first lens group 101.

  The subject luminous flux that has passed through the photographing optical system forms an image on the image sensor 107 via the optical low-pass filter 106. The optical low-pass filter 106 is an optical element for reducing false colors and moire in the captured image.

  The image sensor 107 includes a CMOS sensor and its peripheral circuits, and photoelectrically converts a subject image formed on the image pickup surface to output an image signal. The image sensor 107 is a two-dimensional single-plate color sensor in which a Bayer array primary color mosaic filter is formed on-chip on light receiving pixels of m pixels in the horizontal direction and n pixels in the vertical direction.

  As will be described later, the image sensor 107 includes a plurality of focus detection pixels and a plurality of imaging pixels. The plurality of focus detection pixels receive light beams that have passed through different pupil regions of the photographing optical system and output a first pixel signal. The plurality of imaging pixels receive a light beam that has passed through the same pupil region of the photographing optical system and output a second pixel signal.

  The zoom actuator 111 rotates the cam cylinder (not shown) to drive the first lens group 101, the second lens group 103, and the third lens group 105 forward and backward in the optical axis direction, thereby performing a zooming operation. The aperture shutter actuator 112 controls the aperture diameter of the aperture / shutter 102 to adjust the amount of photographing light (aperture adjustment), and performs exposure time control during still image shooting. The focus actuator 114 performs focus adjustment by driving the third lens group 105 as a focus lens back and forth in the optical axis direction.

  The electronic flash 115 is used for subject illumination during photographing. As the electronic flash 115, a flash illumination device using a xenon tube is suitable, but an illumination device including LEDs that emit light continuously may be used. The AF auxiliary light means 116 projects a mask image having a predetermined aperture pattern onto the object field via the light projection lens, and improves the focus detection capability for a dark subject or a low-contrast subject.

  The camera control circuit 121 includes a calculation unit, a ROM, a RAM, an A / D converter, a D / A converter, a communication interface circuit, and the like in addition to a CPU that controls the entire camera. A program stored in a ROM or the like of the camera control circuit 121 is expanded in the RAM, and a series of processing such as AF, shooting, image processing, and recording is executed by the CPU. The camera control circuit 121 includes a focus detection unit 121b, a subject detection unit 121c, and an exposure processing unit 121d. The subject detection unit 121c performs a process of detecting a subject, for example, a face, based on the image signal obtained from the image sensor 107. The exposure processing unit 121d performs processing for determining an aperture value, a digital gain, a shutter speed, and the like, which are exposure conditions set in the main exposure, based on the image signal obtained from the image sensor 107.

  The electronic flash control circuit 122 controls the light emission of the electronic flash 115 in synchronization with the photographing operation. The auxiliary light drive circuit 123 controls the AF auxiliary light unit 116 to emit light in synchronization with the focus detection operation. The image sensor driving circuit 124 controls the image capturing operation of the image sensor 107 and A / D converts the image signal output from the image sensor 107 and transmits the image signal to the camera control circuit 121. The image processing circuit 125 performs image processing such as γ conversion, color interpolation, and JPEG compression on the image signal obtained from the image sensor 107.

  The focus drive circuit 126 controls the focus actuator 114 based on the focus detection result, and adjusts the focus by driving the third lens group 105 back and forth in the optical axis direction. The aperture shutter drive circuit 128 controls the aperture shutter actuator 112 and controls the aperture of the aperture / shutter 102. The zoom drive circuit 129 drives the zoom actuator 111 according to the zoom operation of the photographer.

  The display 131 is configured by an LCD or the like, and displays information related to the shooting mode of the camera, a preview image before shooting and a confirmation image after shooting, a focus state display image at the time of focus detection, and the like. The operation switch group 132 includes a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. The flash memory 133 is detachable and records captured images.

  FIG. 2 is a block diagram illustrating a configuration example of the image sensor 107. Note that FIG. 2 shows the minimum configuration necessary for the description of the image signal readout operation, and the pixel reset signal and the like are not shown.

  In FIG. 2, the photoelectric conversion unit 201 (PDmn) includes a photodiode, a pixel amplifier, a reset switch, and the like. M of the photoelectric conversion unit 201 (PDmn) is an X-direction address, and m = 0, 1,. N is a Y-direction address, and n = 0, 1,..., N−1. The image sensor 107 of the present embodiment is configured by arranging m × n photoelectric conversion units 201 in two dimensions. In addition, the code | symbol of m of the photoelectric conversion part 201 (PDmn) is attached | subjected only to the photoelectric conversion part 201 (PD00) periphery of the upper left of a figure for convenience of explanation.

  The switch 202 is a switch that selects the output of the photoelectric conversion unit 201 (PDmn), and is selected for each row by the vertical scanning circuit 208. The line memory (MEM) 203 is a memory that temporarily stores the output of the photoelectric conversion unit 201 (PDmn), and stores the output of the photoelectric conversion unit 201 (PDmn) for one row selected by the vertical scanning circuit 208. To do. As the line memory 203, a capacitor is usually used.

  The switch 204 is controlled by the signal HRST, is connected to the horizontal output line, and resets the horizontal output line to a predetermined potential VHRST. The switch 205 (H0 to Hm-1) sequentially outputs the output of the photoelectric conversion unit 201 (PDmn) stored in the line memory 203 to the horizontal output line. By sequentially scanning the switch 205 (H0 to Hm-1) by a horizontal scanning circuit 206 described later, the output of the photoelectric conversion unit 201 (PDmn) for one row is read.

  The horizontal scanning circuit 206 sequentially operates the output of the photoelectric conversion unit 201 (PDmn) stored in the line memory 203 and outputs it to the horizontal output line. The signal PHST is a data input of the horizontal scanning circuit 206, PH1 and PH2 are shift clock inputs, data is set when PH1 = H, and data is latched at PH2. By inputting the shift clock to the shift clock inputs PH1 and PH2, the signal PHST can be sequentially shifted and the switches 205 (H0 to Hm-1) can be sequentially turned on. The signal SKIP is a control input signal for performing setting during thinning readout. By setting the signal SKIP to H level, the horizontal scanning circuit 206 can be skipped at predetermined intervals. The amplifier AMP207 amplifies the signal on the horizontal output line and outputs it to the terminal VOUT.

  The vertical scanning circuit 208 can select the selection switch 202 of the photoelectric conversion unit 201 (PDmn) by sequentially scanning and outputting the control signals V0 to Vn-1. As in the case of the horizontal scanning circuit 206, the control signals V0 to Vn-1 are controlled by a signal PVST, shift clocks PV1 and PV2, which are data inputs, and a signal SKIP for performing thinning-out reading settings. Note that the details of the operation of the vertical scanning circuit 208 are the same as those of the horizontal scanning circuit 206, and thus description thereof is omitted.

  Next, with reference to FIGS. 3 to 6, the structure of the imaging pixels and focus detection pixels of the imaging element 107 will be described. In this embodiment, out of 4 pixels of 2 × 2, pixels having G (green) spectral sensitivity are arranged in two diagonal pixels, and R (red) and B (blue) spectral sensitivity are arranged in the other two pixels. A Bayer array in which one pixel having each of the above is arranged is employed. In addition, focus detection pixels having a structure to be described later are distributed and arranged according to a predetermined rule between the Bayer arrays.

  FIG. 3A is a plan view of 2 × 2 imaging pixels. As is well known, in the Bayer array, G (Green) pixels are arranged in a diagonal direction, and R (Red) and B (Blue) pixels are arranged in the other two pixels. The 2 rows × 2 columns structure is repeatedly arranged.

  FIG.3 (b) is the sectional view on the AA line of Fig.3 (a). In FIG. 3B, ML is an on-chip microlens disposed on the forefront of each pixel, CFR is an R color filter, and CFG is a G color filter. PD schematically shows a photoelectric conversion unit of the CMOS sensor, and CL is a wiring layer for forming signal lines for transmitting various signals in the CMOS sensor. TL schematically shows a photographing optical system.

  The on-chip microlens ML and the photoelectric conversion unit PD of the imaging pixel are configured to capture the light flux that has passed through the photographing optical system TL as effectively as possible. In other words, the exit pupil EP of the photographic optical system TL and the photoelectric conversion unit PD are designed to have a conjugate relationship by the microlens ML, and the effective area of the photoelectric conversion unit PD is large. In FIG. 3B, the incident light beam of the G pixel has been described, but the R pixel and the B pixel also have the same structure. Accordingly, the exit pupil EP corresponding to each of the RGB imaging pixels has a large diameter, and the S / N of the image signal is improved by efficiently capturing the light flux from the subject.

  FIG. 4 is a diagram for explaining the arrangement and structure of focus detection pixels for performing pupil division in the horizontal direction (left-right direction in FIG. 4) of the photographing optical system. Here, the horizontal direction refers to a direction along a straight line that is orthogonal to the optical axis and extends in the horizontal direction when the camera is held so that the optical axis of the photographing optical system is horizontal.

  FIG. 4A is a plan view of pixels of 2 rows × 2 columns including focus detection pixels. When obtaining an image signal for recording or viewing, the main component of luminance information is acquired by G pixels. Since human image recognition characteristics are sensitive to luminance information, image quality degradation is easily recognized when G pixels are lost. On the other hand, the R pixel and the B pixel are pixels that acquire color information (color difference information), but human visual characteristics are insensitive to color information. For this reason, it is difficult for pixels for obtaining color information to recognize image quality degradation even if some defects occur. Therefore, in the present embodiment, among the pixels of 2 rows × 2 columns, the G pixel is left as an imaging pixel, and the R and B pixels are replaced with focus detection pixels. This is indicated by SA and SB in FIG.

  FIG. 4B is a cross-sectional view taken along line AA in FIG. The micro lens ML and the photoelectric conversion unit PD of the focus detection pixel illustrated in FIG. 4B have the same structure as the imaging pixel illustrated in FIG. In the present embodiment, the focus detection pixel is configured by arranging a color filter CFG (Green) of G pixel at the pixel position of B pixel. Further, since pupil division is performed by the image sensor 107, the opening of the wiring layer CL is deviated in one direction with respect to the center line of the microlens ML.

  Specifically, since the opening OPHA of the pixel SA is biased to the right side, the light beam that has passed through the left exit pupil EPHA of the imaging optical system TL is received. Similarly, since the opening OPHB of the pixel SB is biased to the left side, the light beam that has passed through the right exit pupil EPHB of the imaging optical system TL is received. The pixels SA are regularly arranged in the horizontal direction, and a subject image acquired by these pixel groups is defined as an A image. The pixels SB are also regularly arranged in the horizontal direction, and the subject image acquired by these pixel groups is defined as a B image.

  Then, by detecting the relative positions of the A image and the B image and multiplying the shift amount of the image by a conversion coefficient, the focus shift amount (defocus amount) of the subject image can be calculated.

  Next, a method for obtaining a conversion coefficient for calculating the defocus amount from the image shift amount will be described. The conversion coefficient is calculated based on the aperture information of the imaging optical system and the sensitivity distribution of the focus detection pixels. The image sensor 107 (image sensor) receives a light beam limited by several constituent members such as the lens holding frame of the photographing lens TL and the diaphragm 102.

  FIG. 5 and FIG. 6 are diagrams illustrating a state in which the focus detection light beam is limited by the vignetting of the photographing optical system. FIG. 5A is a diagram showing a state in which the light beam is restricted by the stop 102 of the imaging optical system at the position of the exit pupil plane 501 with respect to the pixel near the center of the image sensor 107.

  In FIG. 5A, an image sensor 107a indicated by a solid line is an image sensor at the planned image plane position. The position 506 is the position of the optical axis 505 on the image sensor 107a at the planned imaging plane position. Light beams 507 and 508 are light beams when the diaphragm 102 is restricted by the diaphragm 102, respectively. The luminous fluxes 509 and 510 are luminous fluxes that are not limited by the diaphragm 102 when the diaphragm 102 is opened. Light beams 511 and 512 are focus detection light beams for the light beams 507 and 508, respectively. The barycentric positions 515 and 516 are barycentric positions of the focus detection light beams 511 and 512, respectively.

  Similarly, light beams 513 and 514 are focus detection light beams for light beams 509 and 510, respectively. The gravity center positions 517 and 518 are the gravity center positions of the focus detection light beams 513 and 514, respectively. Reference numeral 530 denotes a lens holding frame on the side closest to the image sensor, and 531 denotes a lens holding frame on the side closest to the subject.

  FIG. 5B is a diagram for explaining a change in the center of gravity due to vignetting on the exit pupil plane 501 of the focus detection pixel at the center of the image sensor 107b indicated by the broken line in FIG. In FIG. 5B, pupil areas 523 and 524 are pupils of light beams 507 and 508 restricted by the diaphragm 102 with respect to the central pixel of the image sensor 107b, and light beams 509 and 510 not restricted by the diaphragm 102, respectively. Indicates the area. Incident angle characteristics 525 and 526 indicate the incident angle characteristics of the focus detection pixels SA and SB, respectively.

  Light beams that have passed through the inside of the pupil regions 523 and 524 are incident on the focus detection pixels SA and SB with a sensitivity distribution indicated by the incident angle characteristics 525 and 526. Therefore, by obtaining the distribution centroids of the focus detection light fluxes that have passed through the inside of the pupil regions 523 and 524, the focus detection light flux is restricted by the stop 102, and the focus detection light flux is not restricted by the stop 102. Can be calculated.

  And each base line length can be calculated | required from the calculated center-of-gravity space | interval. Obtaining the conversion coefficient for calculating the defocus amount from the image shift amount by obtaining the sensitivity distribution information of the focus detection pixel and the aperture information of the imaging optical system from the measurement and calculation and storing them in a memory in advance. Can do.

  In FIG. 5A, the defocus amount 519 is DEF, and the distance 520 from the image sensor 107a to the exit pupil plane 501 is L. In addition, the base lengths (center-of-gravity distances) when the focus detection light beam is restricted by the diaphragm 102 and when it is not restricted are G1 (distance between centroid positions 515 and 516) and G2 (distance between centroid positions 517 and 518), respectively. ). In addition, the image shift amounts 521 and 522 are set to PRED1 and PRED2, respectively, and conversion coefficients for converting the image shift amounts 521 and 522 to the defocus amount DEF are set to K1 and K2, respectively.

  At this time, the defocus amount DEF can be obtained by the following equation (1).

DEF = K1 × PRED1 = K2 × PRED2 (1)
Also, conversion coefficients K1 and K2 for converting the image shift amounts 521 and 522 into the defocus amount DEF are obtained by the following equations (2) and (3), respectively.

K1 = L / G1 (2)
K2 = L / G2 (3)
Here, K1 <K2. Therefore, comparing the case where the aperture 102 is opened and the case where the aperture 102 is stopped, an error equivalent to the image misalignment amount occurs when the image misalignment amount is calculated, and the aperture 102 is reduced. In the case where there is an error, an error of K2 / K1 times occurs as the defocus amount.

  FIG. 6A is a diagram illustrating sensitivity distributions of the A image pixel 530, the B image pixel 531, and the imaging pixel 532 in the cross section along the line AA in FIG. 5B when the diaphragm 102 is opened. FIG. 6B is a diagram illustrating sensitivity distributions of the A image pixel 530, the B image pixel 531, and the imaging pixel 532 in the cross section along the line AA in FIG.

  In FIG. 6, the horizontal axis indicates the light incident angle, and the vertical axis indicates the sensitivity distribution. Comparing FIG. 6 (a) and FIG. 6 (b), the base length G1 is longer and the incident angle width with sensitivity is wider when the aperture is opened as shown in FIG. 6 (a). Since the base line length is proportional to the image shift amounts of the A and B images with respect to the defocus amount, the sensitivity of the image shift amount to the defocus amount increases as the base line length increases. Further, when the incident angle width having sensitivity is increased, the blur amount and image vignetting of the A image and the B image with respect to the defocus amount are increased. In general, as an image signal by a focus detection pixel, an image signal having a high sensitivity of an image shift amount with respect to a defocus amount and a small blur amount and image vignetting is desired.

  FIG. 7 is a diagram illustrating an arrangement example of imaging pixels and focus detection pixels in the imaging element 107. In FIG. 7, pixels G, GA, GB have a green color filter (green filter), pixel R has a red color filter (red filter), and pixel B has a blue color filter (blue filter). Filter).

  The focus detection pixel SA is a reference pixel group that is formed by biasing the opening of the pixel portion in the horizontal direction and detects the image shift amount in the horizontal direction with respect to the focus detection pixel SB. The focus detection pixel SB is a reference pixel group that is formed by biasing an opening of a pixel in a direction opposite to the focus detection pixel SA, and for detecting an image shift amount in the horizontal direction with respect to the focus detection pixel SA. The white portions of the focus detection pixels SA and SB indicate the opening positions of the biased pixels. The focus detection pixels SA and SB are arranged from the AF line 1 to the AF line 4.

  In order to detect the focus state near the in-focus position with high accuracy, it is necessary to arrange the focus detection pixels densely. On the other hand, the image signal (pixel signal) at the position of the focus detection pixel needs to be generated by interpolation processing using the output signal of the surrounding imaging pixels and the output signal of the focus detection pixel. For this reason, when the influence of image quality degradation is taken into consideration, it is desirable to arrange focus detection pixels sparsely.

  In FIG. 7A, the focus detection pixels SA and SB are arranged at the position of the B pixel where the image quality deterioration due to the defect is difficult to be recognized in consideration of the influence of the image quality deterioration. On the other hand, in FIG. 7B, since the focus detection pixels SA and SB are provided with a large number of imaging pixels of the same color in the periphery, the G pixel that makes it relatively easy to perform interpolation of the image signal at the position of the focus detection pixel It is arranged at the position. In the present embodiment, the focus detection process and the correction process of the focus detection result are performed using the signal obtained from the focus detection pixel. The arrangement of the focus detection pixels at that time is shown in FIGS. It is not limited to the two examples shown in b).

  Next, with reference to FIGS. 8 to 10, an example of operation of the digital camera 100 during continuous shooting will be described. 8 to 10 show the operation sequence for each of the release switch (SW2) ON, the image sensor 107, the AF process, the focus lens (third lens group) 105, the exposure process, and the aperture drive. Further, when the photographer presses a release button (not shown) halfway to turn on the release switch (SW1), AF processing is performed, and when the release button (SW2) is turned on by further pressing the release button, a falling signal is generated. This is shown as the start position. In the present embodiment, the main exposure is accumulated and read once after the start position, and the second shooting operation will be described. The exposure processing unit 121d sets an exposure condition for accumulating the next main exposure using the imaging signal read in the previous shooting.

  FIG. 8 is a diagram illustrating an operation sequence during continuous shooting in which the image sensor 107 of the digital camera 100 is sequentially exposed.

  The image sensor 107 accumulates as the main exposure and starts reading of the imaging pixels, and also reads the focus detection pixels in parallel. As described above, since the focus detection pixel is arranged in a range in which the image height is limited with respect to the range of the imaging pixel, the readout of the focus detection pixel is completed before the readout of the imaging pixel. The range in which focus detection is performed is determined by performing detection processing from the range-finding range at the time of AF processing or the captured image of the previous continuous shooting.

  After completing the readout of the focus detection pixels, AF calculation is performed. A correlation calculation is performed on the focus detection pixels, and a defocus amount is obtained from the calculated image shift amount. After the AF operation, the focus lens 114 is moved to the in-focus position by the focus actuator 114 based on the calculated defocus amount.

  The aperture drive is driven to open simultaneously with the start of reading. The reason for driving to open is to set the initial position of the diaphragm / shutter 102. Once the aperture / shutter 102 is driven to the open position and the aperture / shutter 102 is initially positioned and then driven to a predetermined aperture value, the aperture aperture accuracy can be maintained. However, as long as the aperture / shutter can satisfy the aperture aperture accuracy with a predetermined aperture value without opening the aperture, the aperture need not be driven to open. After the aperture is opened, the aperture / shutter 102 is driven to the aperture set in the exposure process described above.

  After confirming that the focus lens drive and aperture drive described above have been completed and reading has been completed, accumulation as main exposure is started, and the shutter speed is controlled by the shutter of the diaphragm / shutter 102 by the aperture shutter actuator 112. And complete the accumulation. With the above continuous shooting sequence, AF processing and exposure processing can be performed between the main exposure (accumulation) and readout of the image sensor 107, and high-speed continuous shooting is possible.

  FIG. 9 is a diagram illustrating an operation sequence at the time of continuous shooting of the digital camera 100 in a case where the image processing range is limited with respect to the imaging pixel range. The AF processing and focus lens driving in FIG. 9 are the same as those in FIG.

  In FIG. 9, accumulation of the main exposure is performed, and reading of the image pickup pixel signal is started, and at the same time, the aperture / shutter 102 is driven to open. When the readout of the imaging pixel signal in the range where the exposure processing is performed is completed, the above-described exposure processing is performed, and the diaphragm / shutter 102 is driven to narrow down to the set aperture value. After the aperture drive and the focus lens drive are completed and the reading is completed, accumulation as the main exposure is started.

  By performing the above processing, in FIG. 8, there is a time difference corresponding to two continuous shooting processes (two frames) in the main exposure (accumulation) used for setting the exposure condition and the main exposure reflecting the setting of the exposure condition. In FIG. 9, the setting of the exposure condition can be reflected with a time difference of one time (one frame). As a result, the exposure conditions can be set quickly without delaying the time for continuous shooting, so that high-speed continuous shooting is possible.

  FIG. 10 is a diagram showing an operation sequence at the time of continuous shooting of the digital camera 100 when subject face detection processing is added. The subject detection process is a process for specifying the position and range of the main subject using a known technique such as pattern matching. In the present embodiment, the face detection process is described, but a process using another detection technique may be performed.

  In FIG. 10, the subject detection means 121c is used to perform face detection processing from the imaging signal after completion of reading, and detect the position and range. Information on the detected position and range is reflected in AF processing and exposure processing. In the AF process, correlation calculation is performed with the focus detection pixels in the detected range, the defocus amount is calculated, and the focus lens is driven. The exposure process is performed within the detected range, and an aperture value is set. The aperture / shutter 102 is driven to a set aperture value. After the aperture driving and the focus lens driving are completed and the reading is completed, accumulation as the main exposure is started.

  In this way, by performing face detection processing and performing AF processing and exposure processing at the detected position and range, exposure condition setting and AF processing are performed on the main subject without delaying the time for continuous shooting. Therefore, high-speed continuous shooting can be performed.

  Next, the continuous shooting operation of the digital camera 100 will be described with reference to FIG. Each process in FIG. 11 is executed by the CPU after the program stored in the ROM or the like of the camera control circuit 121 is expanded in the RAM.

  In FIG. 11, in step S1001, the camera control circuit 121 performs an AF operation. Specifically, when the photographer presses the release button halfway and the release switch (SW1) is turned on, the camera control circuit 121 performs phase difference detection AF using focus detection pixels and focuses the focus lens 105. The position is moved to step S1002.

  In step S1002, the camera control circuit 121 proceeds to step S1003 when the photographer fully presses the release button to turn on the release switch (SW2).

  In step S1003, the camera control circuit 121 performs shooting preparation such as exposure condition setting, and the process advances to step S1004.

  In step S1004, the camera control circuit 121 performs accumulation, which is the main exposure, in the image sensor 107, and proceeds to step S1005.

  In step S1005, the camera control circuit 121 starts reading the signals of the imaging pixels and focus detection pixels, and the process proceeds to step S1006.

  In step S1006, the camera control circuit 121 determines whether or not the release switch (SW2) is kept on, that is, whether or not to perform continuous shooting, and if the release switch (SW2) is on. The process proceeds to step S1007 and step S1010. The camera control circuit 121 ends the photographing operation when the release switch (SW2) is not on.

  In step S1007, the camera control circuit 121 reads out the focus detection pixel from the image sensor 107. When the reading is completed, the process proceeds to step S1008.

  In step S1008, the camera control circuit 121 performs a correlation calculation using the focus detection pixels read from the image sensor 107 in step S1007, calculates a defocus amount, and proceeds to step S1009.

  In step S1009, the camera control circuit 121 moves the focus lens 105 to the in-focus position based on the defocus amount calculated in step S1008, and the process proceeds to step S1012.

  On the other hand, in step S1010, the camera control circuit 121 drives the diaphragm / shutter 102 to open, and the process proceeds to step S1100.

  In step S1100, the camera control circuit 121 performs an exposure process and proceeds to step S1011. Details of the exposure process here will be described later with reference to FIG.

  In step S1011, the camera control circuit 121 performs aperture stop driving of the aperture / shutter 102 to the aperture value of the exposure processing result in step S1100, and the process proceeds to step S1012.

  In step S1012, the camera control circuit 121 determines whether reading of the imaging pixel signal is completed. If completed, the process returns to step S1004 and starts accumulation as main exposure again.

  Next, the exposure process in step S1100 of FIG. 11 will be described with reference to FIG.

  In FIG. 12, in step S1101, the camera control circuit 121 determines whether or not to set exposure within the face detection range. If exposure setting is to be performed, the process proceeds to step S1102, and if exposure setting is not to be performed, the process proceeds to step S1104.

  In step S1102, the camera control circuit 121 determines whether the face detection process has been completed. If it has been completed, the process proceeds to step S1103.

  In step S1103, the camera control circuit 121 sets exposure conditions within the face detection range, and returns to the main flow in FIG.

  In step S1104, the camera control circuit 121 determines whether or not the exposure processing range is image height limited. If the image processing range is limited, the process proceeds to step S1105. In this embodiment, the exposure processing range is determined based on the image height limitation. However, the exposure processing range may be determined based on the aperture value of the exposure processing result instead of the image height limitation. This is because when the aperture is fully open, the aperture drive is not performed, so that no time is required for the aperture drive, so that there is no time delay for continuous shooting.

  In step S1105, the camera control circuit 121 determines whether reading of the imaging pixel signal in the exposure processing range is completed. If completed, the process proceeds to step S1106.

  In step S1106, the camera control circuit 121 sets exposure conditions in a limited image height range, and returns to the main flow in FIG.

  In step S1107, the camera control circuit 121 sets exposure conditions based on the previous accumulation result, and returns to the main flow in FIG.

  As described above, in the present embodiment, high-speed continuous shooting can be performed while performing focus adjustment and exposure processing during continuous shooting in an imaging device including an imaging element having a focus detection pixel and a captured image. In addition, the exposure condition setting can be changed during continuous shooting.

  In addition, this invention is not limited to what was illustrated to the said embodiment, In the range which does not deviate from the summary of this invention, it can change suitably.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed. It can also be realized by executing software (program) acquired via a network or various storage media on a personal computer (CPU, processor).

DESCRIPTION OF SYMBOLS 100 Digital camera 102 Aperture / shutter 107 Imaging element 105 Focus lens 121 Camera control circuit 121b Focus detection means 121c Subject detection means 121d Exposure processing means 126 Focus drive circuit 128 Aperture shutter drive circuit

Claims (7)

Focus detection pixels that photoelectrically convert light beams that pass through different pairs of pupil areas in the exit pupil area of the imaging optical system and output a pair of image signals, and photoelectric conversion of light beams that pass through the exit pupil of the imaging optical system An image pickup apparatus including an image pickup element having an image pickup pixel that outputs an image signal,
Focus detection means for performing focus detection using an output signal from the image sensor;
A focus adjusting means for adjusting the focus by driving a focus lens based on the detection result of the focus detecting means;
Exposure processing means for setting exposure conditions at the time of shooting;
A diaphragm adjusting means for adjusting a diaphragm based on the exposure condition set by the exposure processing means;
Determining means for determining whether or not continuous shooting for sequentially exposing the image sensor;
When the determination unit determines that continuous shooting is performed, the focus detection unit performs focus detection from the image signal of the focus detection pixel generated for each exposure, and the focus adjustment unit detects the focus detection unit. Based on the result, focus adjustment is performed during the exposure, the exposure processing unit sets an exposure condition from the image signal generated for each exposure, and the aperture adjustment unit sets the exposure condition set by the exposure processing unit. And a control means for controlling to adjust the aperture during the exposure based on the image pickup apparatus.   The imaging apparatus according to claim 1, wherein the exposure processing unit sets an exposure condition from image signals generated by exposure at different times according to a range in which the exposure process is performed.   The imaging apparatus according to claim 1, wherein the aperture adjustment unit performs aperture adjustment based on an exposure condition set by the exposure processing unit before an image signal is generated by the exposure. Subject detection means for detecting the position and range of the subject from the image signal;
The control means exposes the image signal generated for each exposure at the position and range of the subject detected by the subject detection means when the exposure processing means determines that continuous shooting is performed by the determination means. The imaging apparatus according to any one of claims 1 to 3, wherein control is performed so as to perform condition setting. Focus detection pixels that photoelectrically convert light beams that pass through different pairs of pupil areas in the exit pupil area of the imaging optical system and output a pair of image signals, and photoelectric conversion of light beams that pass through the exit pupil of the imaging optical system A method for controlling an image pickup apparatus including an image pickup element having an image pickup pixel that outputs an image signal,
A focus detection step of performing focus detection using an output signal from the image sensor;
A focus adjustment step of performing focus adjustment by driving a focus lens based on the detection result in the focus detection step;
An exposure processing step for setting exposure conditions during shooting,
An aperture adjustment step for adjusting the aperture based on the exposure condition set in the exposure processing step;
A determination step of determining whether or not continuous shooting is performed to sequentially expose the image sensor,
When it is determined that continuous shooting is performed in the determination step, the focus detection step performs focus detection from an image signal of the focus detection pixel generated for each exposure, and the focus adjustment step includes the focus detection step. Focus adjustment is performed during the exposure based on the detection result in the step, the exposure processing step sets an exposure condition from an image signal generated for each exposure, and the aperture adjustment step is the exposure processing step. An image pickup apparatus control method, comprising: adjusting an aperture during the exposure based on a set exposure condition.   A program for causing a computer to execute each step of the method for controlling an imaging apparatus according to claim 5.   A computer-readable storage medium, wherein the program according to claim 6 is stored.