JP2014149460A - Imaging apparatus and method for controlling the same - Google Patents

Abstract

An imaging apparatus capable of performing high-speed AF is provided.
An imaging device that photoelectrically converts an optical image of a subject incident through a photographing optical system and outputs an electrical signal, and a light flux of the optical image of the subject incident on each pixel on the imaging device The focus division calculation range is set based on the pupil division unit that restricts only the light beam from the specific pupil region and the electrical signal obtained from the light beam that has passed through only the specific pupil region restricted by the pupil division unit. A range shift determination unit and a shift different for each pupil region passed with respect to an electric signal obtained by a light beam that has passed through a plurality of pupil regions different from a specific pupil region in a calculation range set by the range determination frame determination unit. An image shift unit that applies the electric signal shifted by the image shift unit to generate an image; and a focus evaluation unit that performs focus evaluation of the image obtained by the image generation unit.
[Selection] Figure 3

Description

  The present invention relates to an autofocus technique in an imaging apparatus typified by a digital camera.

  Conventionally, digital cameras and the like have been equipped with an autofocus device for focus adjustment, but digital single-lens reflex cameras generally use so-called phase difference AF (autofocus), and digital compact cameras typically use contrast AF. It is. As a feature of these AFs, contrast AF enables precise focusing, but has a drawback that AF processing is slower than phase difference AF because lens driving is required for image acquisition.

  Incidentally, in recent years, an imaging apparatus (Light Field Camera) using a technique called “Light Field Photography” has been proposed. In this imaging apparatus, by having a microlens array in addition to the conventional imaging optical system, data obtained from the imaging element includes angle information of light rays in addition to the light intensity distribution on the light receiving surface. Based on this information, an image of an arbitrary viewpoint and direction can be reconstructed.

  Among these AF techniques, a technique is proposed in which an image desired by a user can be acquired at high speed by determining a distance measurement frame after performing face detection. For example, in Patent Document 1, in an image sensor that can individually receive light beams that have passed through different pupil regions, a plurality of AF distance measurement frames are provided when an image reconstruction is generated, and an evaluation value in each frame is obtained. . Accordingly, it is possible to continuously output images in focus with respect to a plurality of subjects. Similarly, in Japanese Patent Laid-Open No. 2004-260260, there is a method capable of performing focus detection by image reconstruction after a user sets an AF range frame in an image sensor that can individually receive light beams that have passed through different pupil regions. It is disclosed.

JP 2009-258610 A JP 2011-113174 A

  However, in the conventional technique disclosed in the above-mentioned patent document, the focusing speed may be slow. That is, in Patent Document 1, it is possible to focus on a desired subject by setting a plurality of AF distance measurement frames. However, since it is necessary to acquire a large number of images by reconstruction, the calculation load is large, and focusing is performed. May slow down. In Patent Document 2, since a user sets an AF distance measurement frame in advance, real-time AF cannot be performed, and focus adjustment may take time.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an imaging apparatus capable of performing high-speed AF.

  An image pickup apparatus according to the present invention includes an image pickup device that photoelectrically converts an optical image of a subject incident through a photographing optical system and outputs an electrical signal, and an optical image of the subject incident on each pixel on the image pickup device. Based on the electric signal obtained by the pupil dividing means for restricting only the light flux from the specific pupil area of the photographing optical system and the light flux that has passed through only the specific pupil area restricted by the pupil dividing means A distance measurement frame determining means for setting a calculation range of a focal position; and the electric power obtained by a light beam that has passed through a plurality of pupil areas different from the specific pupil area in the calculation range set by the distance measurement frame determination means Image shift means for giving a different shift to each pupil region that has passed through the signal, image generation means for generating an image by adding the electrical signals shifted by the image shift means, and the image generation means A focus evaluation means for performing focus evaluation of the image obtained, characterized in that it comprises a.

  According to the present invention, it is possible to provide an imaging apparatus capable of performing high-speed AF.

1 is a block diagram showing an electrical configuration of a digital camera and a lens that are an embodiment of an imaging apparatus of the present invention. 1 is a schematic diagram of an optical system of an imaging apparatus according to an embodiment of the present invention. 6 is a flowchart illustrating an operation of the imaging apparatus according to the embodiment. The schematic diagram which shows the pupil selection method. The schematic diagram which showed the relationship between shift amount and a pupil selection position. The schematic diagram which showed the positional relationship of a pupil selection position and a ranging frame. The schematic diagram which shows the setting of a ranging frame. 1 is a schematic diagram of an optical system applicable to the present invention.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram showing an electrical configuration of a digital camera and a lens which are an embodiment of an imaging apparatus of the present invention.

  In FIG. 1, a camera 1 and a lens 2 have an imaging optical system, an image processing system, a recording / reproducing system, and a control system. The imaging optical system includes a photographing lens 3 and an imaging element 6, and the image processing system includes an image processing unit 7. The recording / reproducing system includes a memory unit 8 and a display unit 9, and the control system includes a camera system control circuit 5, an operation detection unit 10, a lens system control circuit 12, and a lens driving unit 13. The lens driving unit 13 can drive a focus lens, a shake correction lens, a diaphragm, and the like.

  The imaging system is an optical processing system that forms an image of light from an object on the imaging surface of the imaging element 6 via the photographing lens 3. The imaging element 6 converts an optical image formed by the imaging optical system into an electrical signal. Microlenses are arranged in a lattice pattern on the surface of the imaging element 6 to form a so-called microlens array (hereinafter referred to as MLA). In this embodiment, the MLA constitutes pupil dividing means. Details of the functions and arrangement of the MLA will be described later with reference to FIG. As will be described later, since the focus evaluation amount / appropriate exposure amount can be obtained from the imaging element 6, the imaging lens 3 is adjusted based on this signal, so that an appropriate amount of object light is exposed to the imaging element 6. At the same time, a subject image is formed in the vicinity of the image sensor 6.

  The image processing unit 7 includes an A / D converter, a white balance circuit, a gamma correction circuit, an interpolation calculation circuit, and the like, and can generate a recording image. In addition, an image shift unit, an image generation unit, a contrast evaluation unit, a correlation calculation unit, and the like, which are main parts of the present embodiment, can be included. In the present embodiment, these elements are described assuming that they are arranged in the camera system control circuit 5.

  The memory unit 8 includes a processing circuit necessary for recording in addition to an actual storage unit. The memory unit 8 outputs to the recording unit and generates and stores an image to be output to the display unit 9. In addition, the memory unit 8 compresses images, moving images, sounds, and the like using a predetermined method.

  The camera system control circuit 5 generates and outputs a timing signal at the time of imaging. In addition, the imaging system, the image processing system, and the recording / reproducing system are controlled in response to external operations. For example, in response to a signal detected by the operation detection unit 10 that a shutter release button (not shown) is pressed, the driving of the image sensor 6, the operation of the image processing unit 7, the compression processing of the memory unit 8, and the like are controlled. Further, the display unit 9 controls the state of each segment of the information display device that displays information on a liquid crystal monitor or the like.

  The adjustment operation of the optical system by the control system will be described. An image processing unit 7 is connected to the camera system control circuit 5 and an appropriate focal position and aperture position are obtained based on a signal from the image sensor 6. The camera system control circuit 5 issues a command to the lens system control circuit 12 via the electrical contact 11, and the lens system control circuit 12 controls the lens driving unit 13. Further, a camera shake detection sensor (not shown) is connected to the lens system control circuit 12, and in a mode for performing camera shake correction, the camera shake correction lens is controlled via the lens driving unit 13 based on a signal from the camera shake detection sensor.

  FIG. 2 is a diagram for explaining a main part of the photographing optical system in the present embodiment. In the present embodiment, it is necessary to acquire angle information in addition to the position of the light beam, which is so-called light space information. In the present embodiment, an MLA is disposed in the vicinity of the imaging plane of the photographic lens 3 in order to acquire angle information, and a plurality of pixels are associated with one lens constituting the MLA.

  FIG. 2A is a diagram schematically illustrating the relationship between the image sensor 6 and the MLA 20. FIG. 2B is a schematic diagram showing the correspondence between the pixels of the image sensor and the MLA. FIG. 2C is a diagram showing that pixels provided under the MLA by the MLA are associated with a specific pupil region.

  As shown in FIG. 2A, the MLA 20 is provided on the image sensor 6, and the front principal point of the MLA 20 is arranged in the vicinity of the imaging plane of the photographing lens 3. FIG. 2A shows a state where the MLA is viewed from the side and from the front of the imaging device, and the lens of the MLA is disposed so as to cover the pixels on the imaging device 6 when viewed from the front of the imaging device. In FIG. 2A, the microlenses constituting the MLA are illustrated in large size so that the microlenses are easy to see, but each microlens is actually only about several times as large as a pixel. The actual size will be described with reference to FIG.

  FIG. 2B is a partially enlarged view from the front of the apparatus of FIG. A grid-like frame shown in FIG. 2B indicates each pixel of the image sensor 6. On the other hand, each microlens constituting the MLA 20 is indicated by circles 20a, 20b, 20c, and 20d. As apparent from FIG. 2B, a plurality of pixels are assigned to one microlens. In the example of FIG. 2B, 5 rows × 5 columns = 25 pixels are one microlens. Is provided against. That is, the size of each microlens is 5 × 5 times the size of the pixel. At this time, when outputting as an image, 5 × 5 pixels are added (angle information is integrated) to form one pixel at the time of output. The pixel at this time is referred to as one output pixel in this embodiment.

  FIG. 2C is a diagram in which the image pickup element 6 is cut so that the longitudinal direction of the sensor includes the optical axis of the microlens and the horizontal direction of the drawing. 2, 21, 22, 23, 24, and 25 in FIG. 2C indicate pixels (one photoelectric conversion unit) of the image sensor 6. On the other hand, the upper part of FIG. 2C shows the exit pupil plane of the photographic lens 3. Actually, when the direction is aligned with the sensor diagram shown in the lower part of FIG. 2 (c), the exit pupil plane is in the direction perpendicular to the paper plane of FIG. 2 (c). It is changing. In FIG. 2C, one-dimensional projection / signal processing will be described for easy understanding. In an actual device, this can be easily extended to two dimensions.

  The pixels 21, 22, 23, 24, and 25 in FIG. 2C are in a positional relationship corresponding to 21a, 22a, 23a, 24a, and 25a in FIG. As shown in FIG. 2C, each pixel is designed to be conjugate with a specific region on the exit pupil plane of the photographing lens 3 by the microlens 20. In the example of FIG. 2C, the pixel 21 and the region 31 correspond to the pixel 22 and the region 32, the pixel 23 and the region 33, the pixel 24 and the region 34, and the pixel 25 and the region 35, respectively. That is, only the light beam that has passed through the region 31 on the exit pupil plane of the photographing lens 3 enters the pixel 21. The same applies to the other pixels. As a result, it is possible to obtain angle information from the passing area on the pupil plane and the positional relationship on the image sensor 6.

  Processing for obtaining a focus evaluation value (focus evaluation value) from a signal from the image sensor 6 using the imaging optical system shown in the present embodiment will be described with reference to FIG.

  FIG. 3 is a flowchart for obtaining a focus evaluation value in the imaging apparatus of the present embodiment. 3A shows the overall operation for obtaining the focus evaluation value, FIG. 3B shows the operation of the distance measurement frame determination unit, FIG. 3C shows the operation of the image shift unit, and FIG. FIG. 3E shows the operation of the image generation unit, FIG. 3E shows the operation of the focus evaluation unit, and FIG. 3F shows the operation of the face detection unit.

  Description will be given in order of each step from FIG. Step S1 indicates the start of the focus evaluation value acquisition operation. For example, this is the case when the operation detection unit 10 in FIG. 1 detects a specific operation from the photographer (for example, pressing the release button).

  Step S2 corresponds to acquiring image data by exposing and reading (A / D conversion) the image sensor 6 for a predetermined time. An appropriate exposure amount in photographing can be calculated from the exposure time and the exposure amount at this time. However, since it is not the main part of this embodiment, description is abbreviate | omitted.

  In step S3, the distance measurement frame determining unit is operated to obtain a result. Details of the operation of the ranging frame determination unit will be described later with reference to FIG.

  Steps S4 to S7 form a loop. In step S4, a loop for acquiring a reconstructed image at a reconstructed position necessary for performing the AF operation a plurality of times is formed. At this time, the amount of calculation for image reconstruction can be minimized by performing the fine scan operation after the coarse scan, and higher-speed AF can be performed. In the present embodiment, an operation for performing rough focus adjustment to detect a rough focus position is referred to as a coarse scan, and an operation for performing precise focus detection near the focus position detected by the coarse scan is referred to as a fine scan. Further, the calculation load can be minimized by performing the calculation of the reconstructed image only within the distance measurement frame (in the calculation range) determined in step S3.

  In step S5, the image shift unit is operated to obtain a result. Details of the operation of the image shift unit will be described later with reference to FIG. In step S6, the image generation unit is operated to obtain a result. Details of the operation of the image generation unit will be described later with reference to FIG. A series of operation | movement is complete | finished by step S7.

  In step S8, the focus evaluation unit is operated based on the reconstructed image obtained in steps S4 to S7 to obtain a result. The focus evaluation here may use a contrast AF method or a phase difference AF method. Details of the operation of the focus evaluation unit will be described later with reference to FIG.

  In step S9, a series of focus evaluation operations are terminated. As a result, the reading of the image sensor 6 is performed only once in step S2, and the focus evaluation value can be obtained.

  Next, details of the operation of the distance measurement frame determination unit will be described with reference to FIG. Step S11 indicates the start of the operation of the distance measurement frame determination unit. A distance measurement frame generally refers to an area for AF that is configured to include the center of the screen and the main subject. For example, it is configured as shown by a square in FIG. In the present embodiment, this AF frame setting is performed in a form suitable for the Light Field Camera with a minimum amount of calculation, and by performing focus detection while generating a reconstructed image only within the AF frame, high-speed AF is performed. To realize.

  In step S12, the pupil position to be selected is set. A method for setting the pupil position will be described with reference to FIG. FIG. 4 is a schematic diagram showing a method for selecting a pupil position when the microlens 20 and the image sensor 6 are viewed from the + Z direction in FIG. 4A is a schematic diagram of a pupil selection method when the pupil is divided by 5 × 5 = 25, and FIGS. 4B-1 to 4D are pupils when the pupil is divided by 4 × 4 = 16. It is a schematic diagram of a selection method.

  In the pupil position setting, since the main subject is extracted from the angle of view, it is desirable to generate an image close to pan focus. That is, when division is performed as shown in FIG. 4A, an arbitrary pixel is selected under each microlens to form an output pixel. When selecting one pixel, if one pixel (for example, (1, 1) coordinates) in the peripheral portion is selected, a deviation from the pupil center position of the reconstructed image at the time of all pixel addition occurs, so that FIG. It is preferable to select one pixel of the (3, 3) coordinate that is the center position of the pupil.

  Here, the center of the pupil position refers to the position where the optical axis and the pupil plane intersect. The above-mentioned center position refers to a pupil region that includes the center of the pupil position. If the area under the microlens is even-numbered and there is no region corresponding to the center position, one pixel close to the center position may be selected as shown in FIGS. 4B-1 and 4B-2. Further, as shown in FIG. 4C, two pixels near the center are selected and weighted as 0.5 × (3, 3) + 0.5 × (2, 2), etc. You may make it become the vicinity. Here, the neighborhood is a range where the shift amount in the image shift unit is not dominant with respect to the size of the distance measurement frame in the AF operation (corresponding to steps S4 to S7 in FIG. 3A) to be performed later. Point to.

An example of a method for selecting the vicinity of the center of the pupil will be described with reference to FIG. The number of pupil divisions is n (four divisions in FIG. 5), the F number is F, the angle between the light axis passing through the vicinity of the center of the pupil (pupil region 3 in FIG. 5) and the optical axis is φ, and the pupil peripheral region ( In FIG. 5, the angle formed by the light beam passing through the pupil region 4) and the optical axis is θ, the distance between the imaging surface and the refocus surface is yFδ (δ: size of allowable circle of confusion), and the distance measurement set at that time Let the size of the frame be FR. Then, the magnitudes of θ and φ are
tan φ = ½ nF, tan θ = (n−1) / 2 nF
It can be expressed as. However, when n is an odd number, φ = 0. At this time, the sensor shift amounts Xc and Xn on the refocus plane are:
Xc = yδ / 2n, Xn = (n−1) × yδ / 2n
It can be expressed as. As described above, the light receiving pixels in the pupil peripheral region have a (n−1) times larger shift amount than the light receiving pixels in the vicinity of the pupil center. At this time, if a range in which the shift amount is suppressed to 10% or less of the distance measurement frame size FR is referred to as a neighborhood,
Xc ≦ 0.1FR
The pupil region that satisfies the above may be set as the vicinity.

  Further, when the luminance of the subject is insufficient, a plurality of pixels are added as (2, 2) + (2, 3) + (3, 2) + (3, 3), etc. as shown in FIG. Thus, one output pixel may be formed.

  By selecting a pixel so that the center position of the pupil, its vicinity, or the center of gravity of one output pixel is the center of the pupil, even if there is a blur in the detection by the distance measurement frame determination unit, the subsequent image is In the configuration, movement (shift) of the image can be prevented. That is, it is possible to prevent the main subject from being out of the range set as the distance measurement frame. When the image as shown in FIG. 6 is obtained in step S2, the setting position of the distance measurement frame when the pupil center (3, 3) and the pupil end position (5, 1) are selected is shown in FIG. . In the vicinity of the in-focus position, there is no deviation in the setting range of the distance measurement frame due to the passing pupil region, but the actual object position and the distance measurement frame position deviation increase as the subject blur increases. Thus, in order to prevent the distance measurement frame from being removed from the subject due to out-of-focus, it is necessary to set the pupil center of gravity to be selected at the center.

  In step S13, pixel information at the pupil position set in step S12 is added for each microlens to constitute one output pixel, and a pan-focus image is generated. In step S14, it is determined whether or not the distance measurement frame setting is “face priority”. If it is not face priority, the process proceeds to step S17. If face priority is selected, the process proceeds to step S15.

  In step S15, face detection processing is performed to detect whether there is a human face (face area) as an example of the subject from the image data obtained in step S13. Details of the operation of the face detection process will be described later with reference to FIG.

  In step S16, a main face area is selected as an example of the main subject from the face information of the subject obtained in step S15. If there is no face, the distance measurement frame is set to the center. If there is only one face, the only detected face is set as the main face, and the area where the main face is detected is set as the distance measurement frame. When there are a plurality of face detections, the main face detection is performed by increasing the priority as the main face as the position is closer to the center of the screen and the size is larger, and the distance measurement frame is set as shown in FIG. Set. In the present embodiment, the method of detecting the main face based on the position and size has been described. However, the main face may be detected in consideration of other information such as reliability.

  In step S17, since it is determined in step S15 that there is no main subject, the distance measuring frame is set in the center area as shown in FIG. 7B. In step S18, the setting of all AF distance measurement frames is completed. In step S19, the process returns to the caller step S3.

  In the present embodiment, a case has been described in which a face detection priority is set and a distance measurement frame is set at the center when it does not exist. However, other distance measurement frame setting methods involving face detection are also applicable. This is because the main part of the present embodiment is in the process of performing face detection from an image closer to pan focus.

  Next, details of the face detection operation will be described with reference to FIG. Step S111 indicates the start of the face detection operation.

  In step S112, a horizontal bandpass filter is applied to the image data obtained in step S13. In step S113, a vertical band pass filter is applied to the image data processed in step S112. Edge components are extracted from the image data by the operations in steps S112 and S113.

  In step S114, pattern matching is performed on the edge component obtained in step S113, and facial features (eyes, nose, mouth, etc.) are extracted.

  In step S115, an eye detection process is performed in which a pair of candidates satisfying a preset condition (for example, the distance between two eyes or inclination) is selected as an eye from the eye candidate group extracted in step S114. . Then, candidates that do not form eyes are removed to narrow down the eye candidate group.

  In step S116, the eye candidate group narrowed down in step S115 is associated with other parts (nose, mouth, etc.) that form the corresponding face, and the face is filtered by passing through a preset non-open condition filter. To detect.

  In step S117, face information (face position coordinates, size, detected number, reliability coefficient, etc.) as an example of subject information is calculated for the face area detected in step S116, and the process ends. In step S118, the series of face detection operations is terminated, and the process returns to the caller step S15.

  Here, the method of obtaining the face information by extracting the feature amount of the image data has been described as an example, but any method that can detect the face from the image data can be replaced.

  Next, details of the operation of the image shift unit will be described with reference to FIG.

  Step S21 indicates the start of the operation of the image shift unit. Steps S22 to S27 form a loop. In step S22, loop calculation is executed by the number corresponding to the number of pupil divisions. For example, in the example shown in FIG. 2, since it is divided into 25, calculation according to each pupil position of 25 is performed. As will be described later with reference to FIG. 8, considering the image reconstruction, if the incident angle is different even on the same reconstruction plane (if the exit pupil is sufficiently far away, it is almost synonymous with the different pupil areas passing through. ) The amount of image shift is different. This is a loop for appropriately reflecting this.

  In step S23, the image shift amount in each pupil region corresponding to the evaluation position is calculated based on the data from step S24. In step S24 and step S25, the correspondence relationship between each pixel and MLA is stored, and information indicating which pupil region of each pixel is receiving light is stored.

  In step S26, pixels that have obtained light rays having the same incident angle (obtained light rays from the same pupil region) are shifted based on the information in step S23. For example, pixels 25a and 25b in FIG. 2 correspond to pixels that obtain light rays having the same incident angle. There are as many such pixels as the number of microlenses constituting the MLA.

  A series of operation | movement is complete | finished by step S27. In step S28, the process returns to the caller step S6.

  Next, details of the operation of the image generation unit will be described with reference to FIG. Step S31 indicates the start of the operation of the image generation unit.

In step S32, the area data for addition in step S35 is initialized (filled with 0). The size of the data area at this time may be as many as the number of MLA, and it is convenient if the data gradation can store the product of the original data gradation and the number of pupil divisions. For example, if the original data is 8 bits and divided into 25, if 13 bits (> 8 bits + log ₂ 25), it is not necessary to consider overflow of data.

  Steps S33 to S37 form a loop. In step S33, loop calculation is executed according to the number of microlenses constituting the MLA. For example, in the example illustrated in FIG. 2, the number of microlens is the number of pixels of the original image sensor ÷ 25 (number of pupil divisions).

  Steps S34 to S37 form a loop. In step S34, the loop calculation is executed by the number corresponding to the number of pupil divisions. For example, in the example shown in FIG. 2, since the light is divided into 25, there are light fluxes from 25 pupil positions.

  In step S35, it is determined whether or not the region is to be added. As described with reference to FIG. 3C, the pupil to be added is instructed by the processing of the image shift unit in step S5. When it corresponds to the pupil to be added, the process proceeds to step S36 and is added. In other cases, the process proceeds to step S37. If the shift amount is not an integer multiple of the pixel, the addition is performed while being internally divided in addition step S36. Specifically, it may be added according to the overlapping area. In step S39, the process returns to the caller step S6.

  Details of the operation of the focus evaluation unit will be described with reference to FIG. Step S41 indicates the start of the operation of the focus evaluation unit.

  Steps S42 to S44 form a loop. In step S42, a loop calculation is executed for the number of reconstructed images obtained in step S4. In step S43, an evaluation value for evaluating the focus is calculated. As a focus evaluation method at this time, a method of evaluating the contrast (brightness / darkness difference) of the object may be used, or a method of detecting the focus with the degree of coincidence of two images passing through different pupils may be used. Other calculation methods can be used as long as the evaluation value is associated with the focus fluctuation. In step S44, a series of operations for obtaining the focus evaluation value of each reconstructed image is terminated.

  In step S45, the in-focus position is detected. At this time, if the contrast is evaluated, the lens position having the maximum value may be detected. If the degree of coincidence between the two images is evaluated, the completely matched lens position may be set as the in-focus position. In step S46, the series of operations is terminated, and the process returns to the caller step S8.

  An example of another optical system applicable to this embodiment will be described with reference to FIG. FIG. 8 is a diagram schematically showing a state in which light rays from an object (subject) are imaged on the image sensor 6. FIG. 8A corresponds to the optical system described in FIG. 2, and is an example in which the MLA 20 is disposed in the vicinity of the imaging surface of the photographing lens 3. FIG. 8B shows an example in which the MLA 20 is arranged closer to the object than the imaging plane of the photographic lens 3. FIG. 8C shows an example in which the MLA 20 is arranged on the side farther from the object than the imaging surface of the photographing lens 3.

  In FIG. 8, 6 is an image sensor, 20 is an MLA, 31 to 35 are pupil areas used in FIG. 2, 51 is an object plane, 51a and 51b are points on the object, and 52 is an imaging optical system. The pupil plane is shown. Reference numerals 61, 62, 71, 72, 73, 81, 82, 83, and 84 denote specific microlenses on the MLA, respectively. In FIGS. 8B and 8C, 6a represents a virtual image sensor, and 20a represents a virtual MLA. These are shown for reference in order to clarify the correspondence with FIG. Also, a light beam that passes from the point 51a on the object and passes through the regions 31 and 33 on the pupil plane is indicated by a solid line, and a light beam that passes from the point 51b on the object and passes through the regions 31 and 33 on the pupil plane is indicated by a broken line. did.

  In the example of FIG. 8A, as described with reference to FIG. 2, the MLA 20 is disposed in the vicinity of the imaging surface of the photographing lens 3, so that the imaging element 6 and the pupil plane 52 of the photographing optical system have a conjugate relationship. . Furthermore, the object plane 51 and the MLA 20 are in a conjugate relationship. Therefore, the light beam emitted from the point 51a on the object reaches the microlens 61, the light beam emitted from the point 51b reaches the microlens 62, and the light beams that have passed through the regions 31 to 35 respectively correspond to those provided below the microlens. Reach the pixel.

  In the example of FIG. 8B, the light beam from the photographing lens 3 is imaged by the microlens 20, and the imaging element 6 is provided on the imaging surface. By arranging in this way, the object plane 51 and the image sensor 6 are in a conjugate relationship. The light beam that has exited from the point 51a on the object and passed through the region 31 on the pupil plane reaches the microlens 71, and the light beam that has exited from the point 51a on the object and passed through the region 33 on the pupil plane reaches the microlens 72. To do. The light beam that has exited from the point 51b on the object and passed through the region 31 on the pupil plane reaches the microlens 72, and the light beam that has exited from the point 51b on the object and passed through the region 33 on the pupil plane reaches the microlens 73. To do. The light beam that has passed through each microlens reaches a corresponding pixel provided under the microlens. In this way, images are formed at different positions depending on the point on the object and the passing area on the pupil plane. If these are rearranged at positions on the virtual imaging surface 6a, the same information as in FIG. 8A can be obtained. That is, information on the pupil region (incident angle) that has passed through and the position on the imaging device can be obtained.

  In the example of FIG. 8C, the microlens 20 re-images the light beam from the photographic lens 3 (referred to as re-image because the light beam once imaged is diffused). The image sensor 6 is provided on the image plane. By arranging in this way, the object plane 51 and the image sensor 6 are in a conjugate relationship. The light beam that has exited from the point 51a on the object and passed through the region 31 on the pupil plane reaches the micro lens 82, and the light beam that has exited from the point 51a on the object and passed through the region 33 on the pupil plane reaches the micro lens 81. To do. The light beam that has exited from the point 51b on the object and passed through the region 31 on the pupil plane reaches the microlens 84, and the light beam that has exited from the point 51b on the object and passed through the region 33 on the pupil plane reaches the microlens 83. To do. The light beam that has passed through each microlens reaches a corresponding pixel provided under the microlens. Similar to FIG. 8B, information similar to that in FIG. 8A can be obtained by rearranging the positions on the virtual imaging surface 6a. That is, information on the pupil region (incident angle) that has passed through and the position on the imaging device can be obtained.

  FIG. 8 shows an example in which position information and angle information can be acquired using MLA (phase modulation element) as pupil dividing means. However, position information and angle information (equivalent to restricting the passing region of the pupil) are shown. Other optical configurations can be used as long as they can be obtained. For example, a method of inserting a mask (gain modulation element) having a predetermined pattern into the optical path of the photographing optical system can be used.

  As described above, in the present embodiment, a range-finding frame is set from a pan-focus image based on information on a light beam that has passed through different pupil regions in the image sensor, and a reconstructed image only within that frame is generated. To evaluate the focus. Accordingly, it is possible to provide an imaging apparatus that can perform AF at high speed.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

Claims (5)

An image sensor that photoelectrically converts an optical image of a subject incident through a photographing optical system and outputs an electrical signal;
Pupil dividing means for restricting a light beam of the optical image of the subject incident on each pixel on the image sensor to only a light beam from a specific pupil region of the photographing optical system;
A range-finding frame determining unit that sets a calculation range of a focal position based on the electric signal obtained by a light beam that has passed through only a specific pupil region limited by the pupil dividing unit;
An image shift that gives a different shift for each of the pupil regions that have passed through the electrical signal obtained by the light flux that has passed through a plurality of pupil regions different from the specific pupil region in the calculation range set by the distance measuring frame determining means. Means,
Image generating means for generating an image by adding the electrical signals shifted by the image shifting means;
Focus evaluation means for performing focus evaluation of an image obtained by the image generation means;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the distance measurement frame determining unit sets the specific pupil region at a central portion of the pupil region of the photographing optical system.   The ranging frame determining means selects the specific pupil region so that the weighted and added center of gravity of the plurality of specific pupil regions is at or near the center of the pupil region of the photographing optical system. The imaging apparatus according to claim 1.   The imaging apparatus according to claim 1, wherein the range-finding frame determination unit widens a pupil region to be selected when the signal in the calculation range is smaller than a predetermined value. A method for controlling an imaging apparatus including an imaging device that photoelectrically converts an optical image of a subject incident through a photographing optical system and outputs an electrical signal,
A pupil division step of restricting a light beam of the optical image of the subject incident on each pixel on the image sensor to only a light beam from a specific pupil region of the photographing optical system;
A range-finding frame determining step for setting a calculation range of a focal position based on the electric signal obtained by a light beam that has passed through only a specific pupil region limited by the pupil dividing step;
An image that gives different shifts to each of the pupil regions that have passed through the electrical signal obtained by a light beam that has passed through a plurality of pupil regions different from the specific pupil region in the calculation range set in the ranging frame determining step. Shift process;
An image generating step of generating an image by adding the electric signals shifted by the image shifting step;
A focus evaluation step for performing focus evaluation of the image obtained in the image generation step;
An image pickup apparatus control method comprising: