JP2015108759A - IMAGING DEVICE, IMAGING SYSTEM, IMAGING DEVICE CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device that enables a highly accurate focus detection.SOLUTION: An imaging device comprises: an image pickup element that has a pixel including a first photoelectric conversion unit and a second photoelectric conversion unit with respect to one microlens and has the pixels arrayed two-dimensionally; focus detection means for performing a focus detection in any mode including a first mode and a second mode; and control means for setting the mode to be executed by the focus detection means. The first mode is configured to be a mode that performs the focus detection by a phase difference method on the basis of output signals of the first photoelectric conversion unit and the second photoelectric conversion unit, and the second mode is configured to be a mode that performs the focus detection by a light travel time method on the basis of an output signal of a plurality of pixels.

Description

  The present invention relates to an image pickup apparatus including an image pickup element that performs phase difference type focus detection.

  Japanese Patent Application Laid-Open No. 2005-228561 discloses an image pickup device configured such that a photodiode included in one pixel is divided into two, and each photodiode receives a light beam that passes through different pupil regions of the image pickup lens. ing. In such a configuration, phase difference type focus detection can be performed by comparing the outputs of two photodiodes.

  As described above, at the time of focus detection, the signals of the two photodiodes can be independently read from the image sensor, and the focus detection can be performed by detecting the phase difference. Further, by adding the divided photodiode signals, an output signal of one pixel can be obtained and used to generate a photographed image.

  By the way, when performing the phase difference type focus detection of the one-pixel two-photodiode configuration, there is a problem that the detection performance for a dark place or a subject with little contrast is low. In order to solve such a problem, it is conceivable to use focus detection of a different system in addition to the phase difference system. For example, there is a TOF (Time-Of-Flight) method using an image sensor. The TOF method is a method of measuring the distance to the object by measuring the delay time of the pulsed light from the time when the pulsed light is projected toward the object and the time when the reflected light from the object is received. is there.

  Patent Document 2 discloses a configuration in which at least two transfer switches and two charge storage units corresponding thereto are provided in one pixel, and the TOF method is applied. In such a configuration, the charge information generated by the reflected pulse light is distributed to the respective charge storage units by the two transfer switches, and distance information for each pixel is obtained by calculating the ratio of the charge amounts of the two charge storage units. Can do.

Japanese Patent No. 3774597 JP 2004-294420 A

  However, an increase in area becomes a problem in the configuration using two charge storage portions when applying the TOF method. In order to cope with this problem, a TOF method using adjacent pixels can be considered. However, in the case of an image pickup device having a configuration in which two photodiodes are included in one pixel, a specific problem may occur. For example, parallax occurs when the A and B images, which are the outputs of two photodiodes in the same microlens, are not in focus. For this reason, a difference occurs in signal strength (photodiode output level), and correct distance information may not be calculated.

  Therefore, the present invention provides an imaging apparatus, an imaging system, an imaging apparatus control method, a program, and a storage medium that can perform focus detection with high accuracy.

  An imaging device according to one aspect of the present invention includes a pixel including a first photoelectric conversion unit and a second photoelectric conversion unit for one microlens, and the pixels are two-dimensionally arranged. An imaging device, a focus detection unit that performs focus detection in any mode including the first mode and the second mode, and a control unit that sets the mode executed by the focus detection unit, The first mode is a mode for performing focus detection by a phase difference method based on output signals of the first photoelectric conversion unit and the second photoelectric conversion unit, and the second mode is a plurality of the pixels. In this mode, focus detection is performed by the light travel time method based on the output signal.

  An imaging device according to another aspect of the present invention includes an imaging device including a plurality of pixels that photoelectrically convert light beams that have passed through different pupil regions of the imaging optical system and output a pair of signals, and a first mode And a focus detection means for performing focus detection in any mode including the second mode, and a control means for setting the mode executed by the focus detection means. In this mode, focus detection is performed by a phase difference method based on a pair of signals output from the image sensor, and in the second mode, focus detection is performed by an optical transit time method based on output signals of the plurality of pixels. Mode.

  An imaging system according to another aspect of the present invention includes a lens device including a photographing optical system and the imaging device.

  According to another aspect of the present invention, there is provided a method for controlling an imaging apparatus including a pixel including a first photoelectric conversion unit and a second photoelectric conversion unit for one microlens, and the pixel is two-dimensionally formed. A method for controlling an imaging apparatus including an arrayed imaging device, the step of performing focus detection in a first mode to obtain a first focus detection result, and the second in which focus detection is performed in a second mode. The first mode is a mode for performing focus detection by a phase difference method based on output signals of the first photoelectric conversion unit and the second photoelectric conversion unit. The second mode is a mode in which focus detection is performed by the light travel time method based on the output signals of the plurality of pixels.

  According to another aspect of the present invention, there is provided a method for controlling an imaging apparatus including an imaging device including a plurality of pixels that photoelectrically convert light beams that have passed through different pupil regions of a photographing optical system and output a pair of signals. A method for controlling an imaging apparatus, the step of performing focus detection in a first mode and acquiring a first focus detection result, the step of performing focus detection in a second mode and acquiring a second focus detection result, The first mode is a mode for performing focus detection by a phase difference method based on output signals of the first photoelectric conversion unit and the second photoelectric conversion unit, and the second mode. Is a mode in which focus detection is performed by the light travel time method based on the output signals of the plurality of pixels.

  A program according to another aspect of the present invention is configured to execute a method for controlling the imaging apparatus on a computer.

  A storage medium according to another aspect of the present invention stores the program.

  Other objects and features of the present invention are illustrated in the following examples.

  According to the present invention, it is possible to provide an imaging apparatus, an imaging system, an imaging apparatus control method, a program, and a storage medium that can perform focus detection with high accuracy.

It is a block diagram of the imaging device in each embodiment. It is a pixel arrangement | positioning figure of the image pick-up element in each Example. It is a conceptual diagram which shows the condition where the light beam which came out of the exit pupil of the imaging lens in each Example injects into a unit pixel. 3 is a circuit configuration diagram of a unit pixel in Embodiment 1. FIG. 3 is an example of a readout circuit of the image sensor in Embodiment 1. 3 is a timing chart illustrating a first driving method according to the first exemplary embodiment. FIG. 6 is a schematic diagram of an output signal from an image sensor at the time of focus detection when the first driving method in Embodiment 1 is used. 6 is a timing chart illustrating a second driving method according to the first exemplary embodiment. 6 is a selection diagram of photoelectric conversion units used in the second driving method in Embodiment 1. FIG. It is a flowchart which shows the operation | movement of the focus detection in each Example. 12 is a timing chart illustrating a second driving method according to the second embodiment. 6 is a selection diagram of photoelectric conversion units used in the second driving method in Embodiment 2. FIG. FIG. 6 is a circuit configuration diagram of a unit pixel in Embodiment 3. 10 is an example of a readout circuit of an image sensor in Embodiment 3. 10 is a timing chart illustrating a first driving method according to Embodiment 3. 12 is a timing chart illustrating a second driving method according to the third embodiment. FIG. 10 is a selection diagram of photoelectric conversion units used in the second driving method in Example 3.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the same members are denoted by the same reference numerals, and redundant description is omitted.

  First, with reference to FIG. 1, an imaging apparatus according to Embodiment 1 of the present invention will be described. FIG. 1 is a block diagram of an imaging apparatus 100 in the present embodiment.

  In FIG. 1, reference numeral 101 denotes a photographic lens (photographic optical system), 204 denotes a diaphragm, 102 denotes a microlens array, and 103 denotes an image sensor. The light that has passed through the photographic lens 101 forms an image near the focal position of the photographic lens 101. The microlens array 102 includes a plurality of microlenses 1020 and is disposed in the vicinity of the focal position of the photographing lens 101. Accordingly, the microlens array 102 has a function of dividing and emitting light that has passed through different pupil regions of the photographing lens 101 for each pupil region.

  The image sensor 103 is a solid-state image sensor such as a CMOS image sensor and photoelectrically converts a subject image (optical image) obtained via the photographing lens 101 and outputs an image signal. In the imaging element 103, a plurality of pixels are two-dimensionally arranged so that a plurality of pixels correspond to one microlens 1020. As a result, the light emitted after being divided for each pupil region by the microlens 1020 can be received while retaining the division information and converted into an image signal that can be processed. The imaging element 103 includes a plurality of pixels including a first photoelectric conversion unit and a second photoelectric conversion unit (a plurality of divided pixels) that receive light beams that have passed through different pupil regions of the photographing lens 101. Each of the plurality of pixels included in the image sensor 103 has one microlens 1020 shared by the first photoelectric conversion unit and the second photoelectric conversion unit. The photographic lens 101 includes a focus driving mechanism (not shown) that performs an operation of focusing according to a focus detection result (range measurement result) described later.

  Reference numeral 104 denotes an analog signal processing circuit (AFE), and 105 denotes a digital signal processing circuit (DFE). The analog signal processing circuit 104 performs signal processing such as correlated double sampling processing, signal amplification, reference level adjustment, and A / D conversion processing on the image signal (analog signal) output from the image sensor 103. The digital signal processing circuit 105 performs digital image processing such as reference level adjustment on the image signal (digital signal) output from the analog signal processing circuit 104.

  Reference numeral 106 denotes an image processing circuit, 107 denotes a memory circuit, and 108 denotes a recording circuit. The image processing circuit 106 performs correlation calculation and focus detection of A and B images, which will be described later, and predetermined image processing and defect correction, for the image signal output from the digital signal processing circuit 105. The image processing circuit 106 (focus detection means) performs focus detection in any mode including the first mode and the second mode. The first mode is a mode for performing focus detection by a phase difference method based on output signals of the first photoelectric conversion unit and the second photoelectric conversion unit. The second mode is a mode in which focus detection is performed by the TOF method (light travel time method) based on the output signals of a plurality of pixels. Each of the memory circuit 107 and the recording circuit 108 is a recording medium such as a nonvolatile memory or a memory card that records and holds the image signal output from the image processing circuit 106.

  Reference numeral 109 denotes a control circuit (control means), 110 denotes an operation circuit, and 111 denotes a display circuit. The control circuit 109 generally drives and controls the entire imaging apparatus 100 such as the imaging element 103 and the image processing circuit 106. The operation circuit 110 receives a signal from an operation member provided in the imaging apparatus 100 and reflects a command from the user to the control circuit 109. The control circuit 109 sets a mode (first mode or second mode) executed by the image processing circuit 106 (focus detection unit). The display circuit 111 displays an image after shooting, a live view image, various setting screens, and the like. Reference numeral 112 denotes a light emitting device. The light emitting device 112 emits light in response to a PLIGHT signal output from the control circuit 109 in synchronization with a command from the control circuit 109.

  In the present embodiment, the imaging apparatus 100 is configured integrally with an imaging optical system (imaging lens 101) and an imaging apparatus main body including the imaging element 103. However, the present embodiment is not limited to this, and can also be applied to an imaging system including an imaging apparatus main body and a lens device (imaging optical system) that can be attached to and detached from the imaging apparatus main body.

  Next, the configuration of the image sensor 103 and the definition of the pixels of the image sensor 103 will be described with reference to FIG. FIG. 2 is a pixel arrangement diagram of the image sensor 103, and shows the image sensor 103 and the microlens array 102 observed from the direction of the optical axis Z (optical axis direction) in FIG.

  In this embodiment, each microlens 1020 constituting the microlens array 102 is defined as one pixel, and this is defined as a unit pixel 200. In addition, a plurality of divided pixels 201 (a plurality of photoelectric conversion units) are arranged so as to correspond to one microlens 1020. In this embodiment, the unit pixel 200 is provided with two divided pixels 201 (two photoelectric conversion units) in the horizontal direction (X-axis direction). The two divided pixels 201 are defined as divided pixels 201A and 201B (first photoelectric conversion unit and second photoelectric conversion unit), respectively.

  Next, with reference to FIG. 3, the relationship between the photographing lens 101, the microlens array 102 and the image sensor 103 (unit pixel 200) of the image capturing apparatus 100, and the principle of focus detection by the pupil division method will be described. FIG. 3 is a conceptual diagram showing a situation in which a light beam emitted from the exit pupil of the photographing lens 101 enters a unit pixel. FIG. 3 shows a view in which the light emitted from the photographing lens 101 passes through one microlens 1020 and is received by the image sensor 103 from the direction perpendicular to the optical axis Z (Y-axis direction). ing.

  In FIG. 3, 202 and 203 are exit pupils (pupil regions) of the photographing lens 101. The light beams that have passed through the exit pupils 202 and 203 (different pupil regions) enter the unit pixel 200 with the optical axis Z as the center. As shown in FIG. 3, the light beam passing through the exit pupil 202 is received by the divided pixel 201 </ b> A via the microlens 1020. On the other hand, the light beam that has passed through the exit pupil 203 is received by the divided pixel 201 </ b> B via the microlens 1020. As described above, the divided pixels 201 </ b> A and 201 </ b> B receive light beams from different exit pupils (pupil regions) of the photographing lens 101.

  The output signal of the divided pixel 201A divided in this way is acquired from the plurality of unit pixels 200 arranged in the X-axis direction, and a subject image composed of these output signal groups (first focus detection signals) is obtained. Let it be A image (A image signal). Similarly, an output signal of the divided pixel 201B obtained by pupil division is acquired from a plurality of unit pixels 200 arranged in the X-axis direction, and a subject image composed of these output signal groups (second focus detection signals) is obtained. Let it be B image (B image signal).

  The image processing circuit 106 performs a correlation operation on the A image and the B image, and detects an image shift amount (pupil division phase difference) between the A image and the B image. Further, the image processing circuit 106 multiplies the image shift amount by the focal position of the photographing lens 101 and a conversion coefficient determined from the optical system to thereby obtain a focal position (focal position information corresponding to an arbitrary subject position on the screen). ) Can be calculated. The control circuit 109 performs the focus control of the photographing lens 101 based on the focal position information calculated by the image processing circuit 106, thereby enabling the imaging surface phase difference AF. The image processing circuit 106 can obtain an A + B image signal that is an addition signal of the A image signal and the B image signal. The A + B image signal is used as a normal captured image (captured image signal).

  Next, the circuit configuration of the unit pixel 200 will be described with reference to FIG. FIG. 4 is a circuit configuration diagram of the unit pixel 200 of the image sensor 103. The unit pixel 200 includes photodiodes 301A and 301B, transfer switches 302A and 302B, a floating diffusion region 303, an amplifying unit 304, a reset switch 305, and a selection switch 306.

  The photodiodes 301 </ b> A and 301 </ b> B are photoelectric conversion units that receive light that has passed through the same microlens and generate signal charges corresponding to the amount of light received. The transfer switches 302A and 302B transfer the charges generated in the photodiodes 301A and 301B to the common floating diffusion region 303, respectively. The transfer switches 302A and 302B are controlled by transfer pulse signals PTXA1, PTXA2, PTXB1, and PTXB2, respectively. Specifically, the pixels in the odd columns are controlled by the transfer pulse signals PTXA1 and PTXB1, and the pixels in the even columns are controlled by the transfer pulse signals PTXA2 and PTXB2.

  The floating diffusion region 303 is a charge-voltage converter that temporarily holds the charges transferred from the photodiodes 301A and 301B and converts the held charges into a voltage signal. The amplifying unit 304 is a source follower MOS transistor, amplifies a voltage signal based on the charge held in the floating diffusion region 303, and outputs it as a pixel signal. The reset switch 305 is controlled by the reset pulse signal PRES, and resets the potential of the floating diffusion region 303 to the reference potential VDD. The selection switch 306 is controlled by the vertical selection pulse signal PSEL, and outputs the pixel signal amplified by the amplification unit 304 to the vertical output line 307. Reference numeral 308 denotes a common power supply VDD.

  Next, the configuration of the readout circuit of the image sensor 103 will be described with reference to FIG. FIG. 5 is an example of a readout circuit of the image sensor 103. Reference numeral 1200 denotes a pixel area. In the pixel region 1200, a plurality of unit pixels 200 are arranged in a matrix. For simplification of description, the pixel region 1200 is shown to include 8 × 4 pixels, but in reality, a larger number of pixels are arranged.

  Reference numeral 1201 denotes a vertical shift register, and a drive pulse is transmitted via a common drive signal line 1222 for each pixel in each row. Note that only the transfer pulse signals PTXA1, PTXA2, PTXB1, and PTXB2 are shown as the drive signal lines 1222 for simplification, but actually, a plurality of drive signal lines other than the transfer pulses are connected for each row. The In this embodiment, the pixels in the odd columns are controlled by the transfer pulse signals PTXA1 and PTXB1, and the pixels in the even columns are controlled by the transfer pulse signals PTXA2 and PTXB2. For this reason, the photodiode 301A or the photodiode 302B arranged in the same quadrant (the first side or the second side) of the unit pixel 200 is arranged between the odd-numbered unit pixels and the even-numbered unit pixels. It is possible to control at different timings.

  The unit pixels 200 in the same column are connected to a common vertical output line 307, and a signal from each pixel is input to a common readout circuit 1203 via the vertical output line 307. The signals processed by the reading circuit 1203 are sequentially output to the output amplifier 1221 by the horizontal shift register 1220. Reference numeral 1202 denotes a current source load connected to the vertical output line 307.

  Reference numeral 1203 denotes a reading circuit. Hereinafter, a specific circuit configuration of the reading circuit 1203 will be described. Reference numeral 1204 denotes a clamp capacitor C0, 1205 denotes a feedback capacitor Cf, 1206 denotes an operational amplifier, 1207 denotes a reference voltage source for supplying a reference voltage Vref, and 1223 denotes a switch for short-circuiting both ends of the feedback capacitor Cf. The switch 1223 is controlled by the PC0R signal. Reference numerals 1208, 1209, 1210, and 1211 denote capacitors for holding a signal voltage. Reference numeral 1208 denotes a capacitor CTSAB, 1210 denotes a capacitor CTSA, and 1209 and 1211 denote a capacitor CTN.

  Reference numerals 1212, 1213, 1214, and 1215 denote switches that control writing to the capacitors 1208, 1209, 1210, and 1211, respectively. The switch 1212 is controlled by the PTSAB signal, and the switch 1210 is controlled by the PTSA signal. The switches 1213 and 1215 are controlled by the PTN signal. Reference numerals 1216, 1217, 1218, and 1219 are switches for receiving a signal from the horizontal shift register 1220 and outputting a signal to the output amplifier 1221. The switches 1216 and 1217 are controlled by the HAB (n) signal of the horizontal shift register 1220, and the switches 1218 and 1219 are controlled by the HA (n) signal. Here, n represents the number of columns of the readout circuit to which the control signal line is connected. Signals written in the capacitors CTSAB 1208 and CTSA 1218 are output to the output amplifier 1221 through the common output line 1223. Further, signals written in the capacitors CTN 1209 and 1211 are output to the output amplifier 1221 via the common output line 1224.

  Next, a driving method (first driving method) in which the control circuit 109 reads pixel signals from the image sensor 103 when performing focus detection by the phase difference method will be described with reference to FIG. FIG. 6 is a timing chart showing the first driving method. Each signal in FIG. 6 is mainly output based on a command from the control circuit 109.

  First, at time T = t1, transfer pulse signals PTXA1, PTXA2, PTXB1, and PTXB2 that are control signals of the transfer switches 302A and 302B are set to H (high), and the photodiodes 301A and 301B are reset. Subsequently, at time T = t2, transfer pulse signals PTXA1, PTXA2, PTXB1, and PTXB2 are set to L (low), and accumulation of photocharges in the photodiodes 301A and 301B is started.

  After accumulation for the necessary time, the vertical selection pulse signal PSEL, which is the control signal of the selection switch 306, is set to H at time T = t3 to turn on the amplifier 304. Subsequently, at time T = t4, the reset pulse signal PRES, which is a control signal for the reset switch 305, is set to L to release the reset of the floating diffusion region 303. At this time, the potential of the floating diffusion region 303 is read as a reset signal level to the vertical output line 307 via the amplifier 304 and is input to the reading circuit 1203. In the readout circuit 1203, the reset signal level is input in a state where the operational amplifier 1206 buffers the output of the reference voltage Vref (PC0R signal is H and switch 1223 is on).

  Thereafter, the PC0R signal is set to L at T = t5, and the output of Vref at that time is written into the capacitors CTN 1209 and 1211. Therefore, at time T = t6, the PTN signal is set to H and the switches 1213 and 1215 are turned on. At time T = t7, the PTN signal is set to H, the switches 1213 and 1215 are turned off, and the writing is completed.

  Next, at time T = t8, the transfer pulse signals PTXA1 and PTXA2 are set to H, and the photoelectric charge of the photodiode 301A is transferred to the floating diffusion region 303. At time T = t9, the transfer pulse signals PTXA1 and PTXA2 are set to L. With this operation, the charge accumulated in the photodiode 301 </ b> A is read out to the floating diffusion region 303. Then, an output corresponding to the change is supplied to the reading circuit 1203 via the amplification unit 304 and the vertical output line 307.

  In the readout circuit 1203, an inversion gain is applied to the voltage change at a ratio of the clamp capacitor 1204 (C 0) and the feedback capacitor 1205 (Cf), and output. In order to write this voltage to the capacitor CTSA 1210, PTSA is switched from L to H at time T = t10, and the switch 1214 is turned on. At time T = t11, the PTSA is switched from H to L, the switch 1214 is turned off, and the writing is completed.

  Next, at time T = t12, the transfer pulse signals PTXA1 and PTXA2 are again set to H, and the transfer pulse signals PTXB1 and PTXB2 are also set to H again. With this operation, the photoelectric charges of both the photodiodes 301 </ b> A and 301 </ b> B can be simultaneously read out to the floating diffusion region 303. At time T = t13, the transfer pulse signals PTXA1, PTXA2, PTXB1, and PTXB2 are turned off. The read charge is supplied to the read circuit 1203 as in the case of reading only the photodiode 301A, and is output via the operational amplifier. In order to write this voltage to the capacitor CTSAB, PTSAB is switched from L to H at time T = t14, and the switch 1212 is turned on. At time T = t15, PTSAB is switched from H to L, the switch 1212 is turned off, and the writing is completed.

  By this operation, a differential voltage between the capacitor CTSAB 1208 and the capacitor CTN 1213 is calculated, and an A + B image signal (addition signal) that is the sum of output signals (pixel signals) from the photodiodes 301A and 301B is obtained. The A + B image signal is a captured image signal and is used for generating a captured image. Further, by calculating the voltage difference between the capacitor CTSA 1214 and the capacitor CTN 1215, an A image signal (first focus detection signal) that is an output signal (pixel signal) from the photodiode 301A is obtained. Information on the light beam passing through a part of the pupil region of the photographing lens 101 is obtained by the A image signal. Also, by calculating the difference between the A + B image signal and the A image signal, a B image signal (second focus detection signal) that is an output signal (pixel signal) from the photodiode 301B is obtained. Information on the light beam passing through a pupil region different from that of the A image signal is obtained by the B image signal. The control circuit 109 can obtain distance information based on the information of these two light beams.

  Next, at time T = t16, the reset pulse signal PRES is set to H, and the floating diffusion region 303 is reset. After that, in the signals held in the capacitors CTSA 1210 and CTN 1215, the drive pulse HA (n) of the horizontal shift register 1220 is sequentially changed from L → H → L for each readout circuit 1203 between times T = t17 and t18. Accordingly, the switches 1218 and 1219 are changed from OFF to ON to OFF. The signals held in the capacitors CTSA 1210 and CTN 1211 in the column in which the switches 1218 and 1219 are changed from OFF → ON → OFF are read to the common output lines 1223 and 1224, respectively, and output as a differential voltage by the output amplifier 1221. This difference voltage becomes the A image signal.

  Next, during time T = t18 to t19, the drive pulse HAB (n) of the horizontal shift register 1220 is sequentially changed from L → H → L for each readout circuit 1203, and accordingly, the switches 1216 and 1217 are turned OFF → ON → It becomes OFF. The signals held in the capacitors CTSAB 1208 and CTN 1209 in the column in which the switches 1216 and 1217 are changed from OFF to ON to OFF are read out to the common output lines 1223 and 1224, respectively, and output as a differential voltage by the output amplifier 1221. This difference voltage becomes an A + B image signal. The above operation is sequentially performed for each row, and reading of the A image signal (first focus detection signal) and the A + B image signal (captured image signal) is completed.

  Next, an output signal from the image sensor 103 at the time of focus detection by the phase difference method will be described with reference to FIG. FIG. 7 is a schematic diagram of an output signal from the image sensor 103 at the time of focus detection by the phase difference method that is the first driving (a schematic diagram of the output signal in the defocused state). In the large defocus state, the subject images a and b are greatly distorted as shown in FIG. In the middle defocus state, as shown in FIG. Further, in the small defocus state, as shown in FIG. As described above, as shown in FIGS. 7A to 7C, the manner in which the subject image is collapsed becomes smaller in accordance with the defocus amount. Here, an aperture (F number) of the optical system is generally considered as an element that affects the defocus amount.

  For example, when the aperture is close to the open state, a light beam that passes through a wider pupil region of the photographing lens 101 is captured, so that generally large defocusing tends to occur when the focus lens is moved away from the focused state. On the other hand, at the time of small-aperture shooting, the light passing through the pupil region of the taking lens 101 is limited, so that the depth of field becomes deep, and generally it is difficult for large defocusing to occur when moving away from the focused state. For this reason, there exists a difficulty that focusing performance deteriorates. Also, in the focus detection using the phase difference method, in the case of a dark place or a low-contrast subject, it is difficult to separate subject signals from the viewpoint of S / N, so that focusing performance often deteriorates. Thus, with the phase difference method, focus detection may be difficult depending on conditions. Therefore, in this embodiment, focus detection is performed by the TOF method (Time-of-Fight) as described below.

  With reference to FIG. 8, a driving method (second driving method) for reading out pixel signals when performing focus detection by the TOF method will be described. FIG. 8 is a timing chart showing the second driving method. In this embodiment, as shown in FIG. 9, the TOF method is performed on the addition signal (A + B image) of the adjacent pixels in the odd and even columns.

  FIG. 9 is a selection diagram of photoelectric conversion units used in the second driving method. As shown in FIG. 9, a colored region (A image + B image) of a pair of pixels surrounded by a thick frame is used as an output signal at the time of focus detection by the TOF method.

  In FIG. 8, first, at time T = t21, transfer pulse signals PTXA1, PTXA2, PTXB1, and PTXB2 that are control signals of the transfer switches 302A and 302B are set to H, and the photodiodes 301A and 301B are reset. Subsequently, at time T = t22, transfer pulse signals PTXA1, PTXA2, PTXB1, and PTXB2 are set to L, and photocharge accumulation in the photodiodes 301A and 301B is started.

  After accumulation for the necessary time, the vertical selection pulse signal PSEL, which is the control signal of the selection switch 306, is set to H at time T = t23, and the amplifying unit 304 is turned on. Subsequently, at time T = t24, the reset of the floating diffusion region 303 is released by setting the reset pulse signal PRES that is the control signal of the reset switch 305 to L. At this time, the potential of the floating diffusion region 303 is read as a reset signal level to the vertical output line 307 via the amplifier 304 and is input to the reading circuit 1203. In the readout circuit 1203, the reset signal level is input in a state where the operational amplifier 1206 buffers the output of the reference voltage Vref (the PC0R signal is H and the switch 1223 is on).

  Thereafter, the PC0R signal is set to L at time T = t25, and the output of Vref at that time is written to the capacitors CTN 1209 and 1211. Therefore, the PTN signal is set to H at time T = t26 and the switches 1213 and 1215 are turned on. At time T = t27, the PTN signal is set to L, the switches 1213 and 1215 are turned off, and the writing is completed.

  Next, at time T = t28, the transfer pulse signals PTXA1 and PTXB1 are set to H, and transfer of the photoelectric charges of the odd-numbered photodiodes 301A and 301B to the floating diffusion region 303 is started. Subsequently, the PLIGHT signal is set to H at time T = t29, and light is projected from the light emitting device 112.

  Next, at time T = t30, the transfer pulse signals PTXA1 and PTXB1 are set to L, and the transfer of the photoelectric charges of the odd-numbered photodiodes 301A and 301B to the floating diffusion region 303 is completed. At the same time, the transfer pulse signals PTXA2 and PTXB2 are set to H, and the photocharges of the photodiodes 301A and 301B in the even columns are started to be transferred to the floating diffusion region 303.

  After that, at time T = t31, PLIGHT is set to L, and the projection of light from the light emitting device 112 is finished. At time T = t32, the transfer pulse signals PTXA2 and PTXB2 are set to L, and the transfer of the photocharges of the photodiodes 301A and 301B in the even-numbered columns to the floating diffusion region 303 is completed. At time T = t32, the PTSAB signal is switched from L to H, and the switch 1212 is turned on. At time T = t33, the PTSAB signal is switched from H to L, and the switch 1212 is turned off.

  Next, at time T = t34, the reset pulse signal PRES is set to H, and the floating diffusion region 303 is reset. Then, during time T = t35 to t36, the drive pulse HAB (n) of the horizontal shift register 1220 is sequentially changed from L → H → L for each readout circuit. Accordingly, the switches 1216 and 1217 are changed from OFF to ON to OFF. The signals held in the capacitors CTSAB 1208 and CTN 1209 in the column in which the switches 1216 and 1217 are changed from OFF to ON to OFF are read out to the common output lines 1223 and 1224, respectively, and output as a differential voltage by the output amplifier 1221.

  With the above driving, it is possible to obtain outputs with different influences of light projection from the light emitting device 112 to the subject for adjacent pixels in the odd and even rows, and thereby focus detection is possible. Specifically, when the distance to the subject is zero, the reflected light is received simultaneously with the PLIGHT signal, and the outputs from the odd-numbered photodiodes 301A and 301B and the even-numbered photodiodes 301A and 301B are equal.

  When the distance to the subject is not zero by the driving shown in FIG. 8, the reflected light is received with a delay. As a result, the pixel output signals in the odd columns and the pixel output signals in the even columns are biased. The delay time of the reflected light with respect to the projection light can be estimated based on the ratio of the signals, and the distance to the subject can be calculated from the product of the delay time and the speed of light.

  Next, with reference to FIG. 10, the focus detection operation of the imaging apparatus 100 in the present embodiment will be described. FIG. 10 is a flowchart illustrating the focus detection operation of the imaging apparatus 100. Each step of FIG. 10 is mainly executed based on a command from the control circuit 109.

  First, in step S1301, the control circuit 109 checks mode settings such as still image shooting and moving image shooting, and shooting modes such as sensitivity and aperture value. Information obtained from the control circuit 109 is detected by the operation circuit 110 from the user or automatically by the imaging device 100.

  Subsequently, in step S1302, the control circuit 109 determines whether or not the shooting mode checked in step S1301 is a mode involving a focus detection operation. When the focus detection operation is not performed, this sequence ends. On the other hand, if the focus detection mode is set, the process proceeds to step S1303. In step S1303, the image processing circuit 106 performs phase difference focus detection by the above-described pupil division (first mode). Subsequently, in step S1304, the control circuit 109 determines the reliability of the focus detection result based on the phase difference detection information obtained in step S1303 (reliability determination). For the reliability determination, the same method as that used in the conventional phase difference method can be applied, and the details thereof are omitted. For example, the signal output value, the contrast determination, the luminance determination, and the correlation between the two images This is done using relationships.

  As a result of the reliability determination, if it is determined that the reliability is high, the process proceeds to step S1306. On the other hand, if it is determined that the reliability is low, the process proceeds to step S1305. In step S1305, focus detection (ranging calculation) is performed by the TOF method (light travel time method) as described above (second mode). In step S1306, the control circuit 109 performs focus control (focusing operation of the photographing lens 101) based on the focus detection result performed in step S1303 or step S1305, and ends this sequence.

  Preferably, the control circuit 109 changes the first mode to the second mode based on the first focus detection result of the image processing circuit 106 obtained in the first mode. More preferably, the control circuit 109 changes the first mode to the second mode when the first focus detection result obtained in the first mode does not satisfy the predetermined reliability. Preferably, the control circuit 109 performs focus control based on the first focus detection result when the first focus detection result obtained in the first mode satisfies a predetermined reliability. In addition, preferably, when the first mode is changed to the second mode, the image processing circuit 106 performs focus control based on the second focus detection result of the focus detection unit obtained in the second mode. .

  Preferably, the image processing circuit 106 performs focus detection by the TOF method (light travel time method) based on output signals of pixels adjacent to each other among the plurality of pixels. More preferably, the image processing circuit 106 performs focus detection by the TOF method (light travel time method) based on the addition signal of the first photoelectric conversion unit and the second photoelectric conversion unit.

  Next, a second embodiment of the present invention will be described. In the first embodiment, driving is performed using the combined signal (addition signal) of the photodiodes 301A and 301B without using the split pupil. However, in the case of using an imaging device having a divided pupil configuration, the present invention is not limited to this, and for example, either an A image or a B image can be used. Hereinafter, an example of a driving method of the TOF method using a part of the divided pixels will be described.

  With reference to FIG. 11, a driving method (second driving method) for reading a pixel signal when performing focus detection by the TOF method using a part of the divided pixels will be described. FIG. 11 is a timing chart showing a second driving method in the present embodiment. When a part of the divided pixels is used, the position of the pixel (the optical axis of the microlens array 102) is not constant with respect to the optical axis of the photographing lens 101. Therefore, the output of the A image and the B image depends on the pixel position. Dependency occurs. In order to reduce the influence, when a part of the divided pixels is used, as shown in FIG. 12, a fixed quadrant (for example, only the A image) is used in the pair of pixels. FIG. 12 shows an example in which a colored region (A image only) of a pair of pixels surrounded by a thick frame is used as an output signal in the operation by the TOF method. The pixel configuration is the same as that of the image sensor 103 described above, and the readout driving method is basically the same as the method described with reference to FIG. For this reason, only a different part is demonstrated here.

  In the driving method of the present embodiment, there is no change in the transfer pulse signals PTXB1 and PTXB2 from time T = t28 to t32. As a result, the B image signal is not output. For this reason, even when a part of the divided pixels is used, it is possible to realize the TOF method which is not easily affected by the output difference due to the difference in the entrance pupil. In the present embodiment, the phase difference detection has been described for the case where focus detection is performed from two types of images, that is, A image and A + B image detection. However, the present embodiment is not limited to this. For example, focus detection can be performed from two types of images, an A image and a B image.

  As described above, in this embodiment, the image processing circuit 106 uses the TOF method (light traveling) based on the output signal of the first photoelectric conversion unit or the second photoelectric conversion unit located in the same quadrant of each of the plurality of pixels. Focus detection by time method.

  Next, Embodiment 3 of the present invention will be described. In the second embodiment, the quadrant used by the pair of pixels is fixed so as not to be affected by the divided pupil of the image sensor. For example, depending on the position of the screen, the influence of incidence from the photographing lens 101 is affected. Is different. As a specific example, the output of the A image on the right side of the screen is high, but the output on the left side of the screen is reduced due to the relationship between the optical axis of the photographing lens 101 and the position of the pixel (optical axis of the microlens array 102). To do. Therefore, in this embodiment, the quadrant used by the pair of pixels in the TOF method is changed depending on the position of the screen. Specifically, the image sensor 103 is divided from the vicinity of the center of the screen, the A image is used on the left side, and the B image is used on the right side.

  FIG. 13 is a circuit configuration diagram of the unit pixel 200 of the image sensor 103 in the present embodiment. 13A shows a configuration belonging to the left side with respect to the optical axis of the photographing lens 101 (center of the photographing screen) in the plurality of pixel arrays, and FIG. 13B shows the optical axis of the photographing lens 101 (center of the photographing screen). ) On the right side. These basic configurations are the same.

  The unit pixel 200 includes photodiodes 301A and 301B, transfer switches 302A and 302B, a floating diffusion region 303, an amplifying unit 304, a reset switch 305, and a selection switch 306. The photodiodes 301 </ b> A and 301 </ b> B are photoelectric conversion units that receive light that has passed through the same microlens and generate signal charges corresponding to the amount of light received.

  The transfer switches 302A and 302B transfer the charges generated in the photodiodes 301A and 301B to the common floating diffusion region 303, respectively. The transfer switches 302A and 302B are controlled by transfer pulse signals PTXA1, PTXA2, PTXB, PTXC, PTXD1, and PTXD2, respectively. In the present embodiment, the left pixel is controlled by transfer pulse signals PTXA1, PTXA2, and PTXB, and the right pixel is controlled by transfer pulse signals PTXC, PTXD1, and PTXD2 with reference to the center of the photographing screen. The transfer pulse signals PTXA1 and PTXD1 are set as odd columns, and the transfer pulse signals PTXA2 and PTXD2 are set as even columns.

  The floating diffusion region 303 is a charge-voltage converter that temporarily holds the charges transferred from the photodiodes 301A and 301B and converts the held charges into a voltage signal. The amplifying unit 304 is a source follower MOS transistor, amplifies a voltage signal based on the charge held in the floating diffusion region 303, and outputs it as a pixel signal. The reset switch 305 is controlled by the reset pulse signal PRES, and resets the potential of the floating diffusion region 303 to the reference potential VDD. The selection switch 306 is controlled by the vertical selection pulse signal PSEL, and outputs the pixel signal amplified by the amplification unit 304 to the vertical output line 307. Reference numeral 308 denotes a common power supply VDD.

  Next, the configuration of the readout circuit of the image sensor 103 will be described with reference to FIG. FIG. 14 is an example of a readout circuit of the image sensor 103. The basic configuration of FIG. 14 is the same as that of FIG. 5 except that transfer pulse signals in the drive signal line 1222 are added, and six transfer pulse signals PTXA1, PTXA2, PTXB, PTXC, PTXD1, and PTXD2 are used. It is done.

  Next, a driving method (first driving method) in which the control circuit 109 reads pixel signals from the image sensor 103 when performing focus detection by the phase difference method will be described with reference to FIG. FIG. 15 is a timing chart showing the first driving method. Each signal in FIG. 15 is mainly output based on a command from the control circuit 109. The basic drive of FIG. 15 is the same as that of FIG. 6, but a drive signal line is added. In the operation of this additional signal, the transfer pulse signals PTXA1, PTXA2, and PTXC have the same movement, and the transfer pulse signals PTXB, PTXD1, and PTXD2 have the same movement. The other parts are the same as those in FIG. 6, and a detailed description thereof is omitted.

  Next, a driving method for reading out pixel signals when performing focus detection by the TOF method will be described with reference to FIG. The basic drive of FIG. 16 is the same as that of FIG. 11, but a drive signal line is added. In the operation using the additional signal line, the transfer pulse signals PTXA1 and PTXD1 perform the same movement, and PTXA2 and PTXD2 perform the same movement. Also, PTXB and PTXC perform the same movement. (Other parts are the same as those in FIG. 11, and thus detailed description is omitted.

  With reference to FIG. 17, a driving method (second driving method) for reading a pixel signal when performing focus detection by the TOF method using a part of the divided pixels will be described. FIG. 17 is a timing chart showing a second driving method in the present embodiment. As shown in FIG. 17, in this embodiment, the quadrants used on the left and right sides from the center of the screen are changed. Specifically, in FIG. 17, among the pair of pixels surrounded by a thick frame, only the A image is used on the left side of the screen, and only the B image is used on the right side of the screen.

  As described above, in this embodiment, the image processing circuit 106 uses the TOF method (light travel time method) based on the output signal of the first photoelectric conversion unit or the second photoelectric conversion unit according to the positions of a plurality of pixels. ) Focus detection. Preferably, the image processing circuit 106 includes the first photoelectric conversion unit or the second photoelectric conversion unit depending on whether the plurality of pixels are located on one side or the other side of the center of the image sensor 103. Based on the output signal, focus detection by the TOF method (light travel time method) is performed.

  As described above, when an image sensor having a divided pupil configuration is driven by the TOF method using adjacent unit pixels, the quadrant of the pixels used in a pair is changed depending on the position, thereby dividing the pupil configuration. The influence of the incident angle at can be reduced.

[Other Embodiments]
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed. In this case, a computer-executable program describing the procedure of the imaging apparatus control method and a storage medium storing the program constitute the present invention.

  According to each embodiment, it is possible to provide an imaging apparatus, an imaging system, an imaging apparatus control method, a program, and a storage medium capable of highly accurate focus detection.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

100: imaging device 103: imaging device 106: image processing circuit 109: control circuit

Claims (16)

An imaging device having a pixel including a first photoelectric conversion unit and a second photoelectric conversion unit for one microlens, and the pixels are two-dimensionally arranged;
Focus detection means for performing focus detection in any mode including the first mode and the second mode;
Control means for setting the mode executed by the focus detection means,
The first mode is a mode for performing focus detection by a phase difference method based on output signals of the first photoelectric conversion unit and the second photoelectric conversion unit,
The image pickup apparatus according to claim 2, wherein the second mode is a mode in which focus detection is performed by a light travel time method based on output signals of the plurality of pixels.   The control unit changes the first mode to the second mode based on a first focus detection result of the focus detection unit obtained in the first mode. The imaging apparatus according to 1.   The control means changes the first mode to the second mode when the first focus detection result obtained in the first mode does not satisfy a predetermined reliability. The imaging device according to claim 2.   The control means performs focus control based on the first focus detection result when the first focus detection result obtained in the first mode satisfies the predetermined reliability. The imaging apparatus according to claim 3.   When the first mode is changed to the second mode, the focus detection unit performs the focus control based on the second focus detection result of the focus detection unit obtained in the second mode. The imaging apparatus according to claim 4, wherein the imaging apparatus is performed.   6. The focus detection unit according to claim 1, wherein the focus detection unit performs focus detection by the light travel time method based on output signals of pixels adjacent to each other among the plurality of pixels. Imaging device.   The focus detection unit performs focus detection by the light travel time method based on the output signal of the first photoelectric conversion unit or the second photoelectric conversion unit located in the same quadrant of each of the plurality of pixels. The imaging apparatus according to claim 1, wherein the imaging apparatus is performed.   7. The focus detection unit according to claim 1, wherein the focus detection unit performs focus detection by the light travel time method based on an addition signal of the first photoelectric conversion unit and the second photoelectric conversion unit. The imaging apparatus of Claim 1.   The focus detection unit performs focus detection by the light travel time method based on the output signal of the first photoelectric conversion unit or the second photoelectric conversion unit according to the positions of the plurality of pixels. The imaging apparatus according to claim 1, wherein   The focus detection unit includes the first photoelectric conversion unit or the second photoelectric conversion unit according to whether the plurality of pixels are located on one side or the other side of the center of the image sensor. The imaging apparatus according to claim 9, wherein focus detection is performed based on the light travel time method based on the output signal. An image sensor including a plurality of pixels that photoelectrically convert light beams that have passed through different pupil regions of the imaging optical system and output a pair of signals;
Focus detection means for performing focus detection in any mode including the first mode and the second mode;
Control means for setting the mode executed by the focus detection means,
The first mode is a mode for performing focus detection by a phase difference method based on a pair of signals output from the image sensor,
The imaging apparatus according to claim 2, wherein the second mode is a mode in which focus detection is performed by an optical travel time method based on output signals of the plurality of pixels. A lens device equipped with a photographing optical system;
An imaging system comprising: the imaging device according to claim 1. A method for controlling an imaging apparatus including an imaging device having a pixel including a first photoelectric conversion unit and a second photoelectric conversion unit for one microlens, and the pixels are two-dimensionally arranged. There,
Performing focus detection in a first mode to obtain a first focus detection result;
And performing focus detection in the second mode to obtain a second focus detection result,
The first mode is a mode for performing focus detection by a phase difference method based on output signals of the first photoelectric conversion unit and the second photoelectric conversion unit,
The control method according to claim 2, wherein the second mode is a mode in which focus detection is performed by a light travel time method based on output signals of a plurality of pixels. A method for controlling an imaging apparatus including an imaging device including a plurality of pixels that photoelectrically convert light beams that have passed through different pupil regions of an imaging optical system and output a pair of signals,
Performing focus detection in a first mode to obtain a first focus detection result;
And performing focus detection in the second mode to obtain a second focus detection result,
The first mode is a mode for performing focus detection by a phase difference method based on output signals of the first photoelectric conversion unit and the second photoelectric conversion unit,
The control method according to claim 2, wherein the second mode is a mode in which focus detection is performed by an optical travel time method based on output signals of the plurality of pixels.   A program configured to cause a computer to execute the control method of the imaging apparatus according to claim 13 or 14.   A storage medium storing the program according to claim 15.