JP2024096310A - Focus detection device and image sensor - Google Patents

Abstract

To provide a focusing state detector and a camera that can suitably detect a focus state.SOLUTION: A focus detector comprises: an image pick-up device that has, on an imaging surface picking up an image formed by an optical system, a plurality of areas detecting a focusing state of the image on the imaging surface, and is arranged, on each of the areas, with a plurality of rows of pixels each having a first pixel receiving light passing through a first area of the optical system and outputting a signal and a second pixel receiving light passing through a second area of the optical system and outputting a signal; and a detection unit that detects the focusing state based on the signals output from the rows of pixels, the number of which is smaller than the number of the rows of pixels arranged in one of the plurality of areas.SELECTED DRAWING: Figure 3

Description

The present invention relates to a focus detection device and an image sensor.

In a conventional focus detection device equipped with multiple light receiving element groups, a technique is known in which a defocus amount is calculated for each of the multiple light receiving element groups, one defocus amount is selected from the multiple calculated defocus amounts, and focus detection is performed based on the selected defocus amount (see, for example, Patent Document 1).

JP 2003-215437 A

The problem that this invention aims to solve is to provide a focus state detection device and an image sensor that can effectively detect the focus state.

The present invention solves the above problems by the following solutions:

[1] A focus detection device according to the present invention is characterized in that it comprises an image sensor having a plurality of focus detection areas on an imaging surface that captures an image formed by an optical system and that detects a focus state between the image and the imaging surface, and in each of the plurality of focus detection areas, a plurality of pixel rows are arranged, each including a pixel having a first photoelectric conversion unit that receives light passing through a first region of the optical system and outputs a signal, and a second photoelectric conversion unit that receives light passing through a second region of the optical system and outputs a signal; and a control unit that detects contrast information of the output of the plurality of pixel rows arranged in one of the plurality of focus detection areas, identifies a portion of pixel rows to be used for focus detection from the plurality of pixel rows arranged in the one focus detection area based on the detection result, and detects the focus state based on a signal output from the identified pixel row.
[2] An image sensor according to the present invention is characterized in that it comprises: an imaging unit having an imaging surface that captures an image formed by an optical system, a plurality of focus detection areas that detect a focus state between the image and the imaging surface, and in each of the plurality of focus detection areas, a plurality of pixel rows each including a pixel having a first photoelectric conversion unit that receives light passing through a first region of the optical system and outputs a signal and a second photoelectric conversion unit that receives light passing through a second region of the optical system and outputs a signal; and a calculation unit that detects contrast information of the output of a plurality of the pixel rows arranged in one of the plurality of focus detection areas, identifies a portion of the pixel rows to be used for focus detection from the plurality of pixel rows arranged in the one focus detection area based on the detection result, and detects the focus state based on a signal output from the identified pixel row.

FIG. 1 is a block diagram showing a camera according to the present embodiment. FIG. 2 is a front view showing the imaging surface of the imaging element shown in FIG. FIG. 3 is an enlarged front view of part III in FIG. 2, showing a schematic arrangement of the imaging pixels 221 and the focus detection pixels 222a and 222b. Figure 4(A) is a front view showing an enlarged view of one of the imaging pixels 221, Figure 4(B) is a front view showing an enlarged view of one of the first focus detection pixels 222a, Figure 4(C) is a front view showing an enlarged view of one of the second focus detection pixels 222b, Figure 4(D) is a cross-sectional view showing an enlarged view of one of the imaging pixels 221, Figure 4(E) is a cross-sectional view showing an enlarged view of one of the first focus detection pixels 222a, and Figure 4(F) is a cross-sectional view showing an enlarged view of one of the second focus detection pixels 222b. FIG. 5 is a cross-sectional view taken along line VV in FIG. FIG. 6 is a flowchart showing the operation of the camera according to the first embodiment. FIG. 7 is a diagram for explaining an example of the operation of the camera 1 according to the present embodiment. FIG. 8 is a diagram for explaining an example of the operation of a conventional camera. FIG. 9 is a diagram showing a schematic arrangement of the first focus detection pixels 222a and the second focus detection pixels 222b according to this embodiment. FIG. 10A is a graph showing the relationship between the correlation amount and the shift amount in this embodiment, and FIG. 10B is a graph showing the relationship between the correlation amount and the shift amount in the prior art. FIG. 11 is a flowchart showing the operation of the camera according to the second embodiment. FIG. 12 is a diagram showing an example of frequency components included in the output of a focus detection pixel array. FIG. 13 is a diagram for explaining a method of calculating contrast information in the third embodiment. FIG. 14 is a diagram showing the configuration of a camera according to the fourth embodiment. FIG. 15 is a block diagram showing the focus detection module shown in FIG. FIG. 16 is a front view showing the imaging surface of the imaging element shown in FIG. FIG. 17 is an enlarged front view of one of the focus detection areas 224 shown in FIG. FIG. 18A is an enlarged front view of one of the imaging pixels 221a, and FIG. 18B is a cross-sectional view. FIG. 19 is a cross-sectional view taken along line XIX-XIX in FIG. FIG. 20 is an enlarged front view of one of the focus detection areas 224 shown in FIG. FIG. 21 is a diagram for explaining a method of calculating contrast information in the fourth embodiment. FIG. 22 is a diagram showing an example of frequency components in each group. FIG. 23 is a flowchart showing the operation of the camera according to the fourth embodiment. FIG. 24 is a diagram for explaining a method of calculating contrast information in the fifth embodiment. FIG. 25 is a flowchart showing the operation of the camera according to the fifth embodiment. FIG. 26 is a front view showing a schematic arrangement of focus detection pixels 222a and 222b according to another embodiment. FIG. 27 is a front view showing a schematic arrangement of focus detection pixels 222a and 222b according to another embodiment. FIG. 28 is a front view showing a schematic arrangement of focus detection pixels 222a and 222b according to yet another embodiment.

The following describes the embodiment with reference to the drawings.

First Embodiment
1 is a diagram showing the essential components of a digital camera 1 according to an embodiment of the present invention. Digital camera 1 according to the present embodiment (hereinafter simply referred to as camera 1) is made up of a camera body 2 and a lens barrel 3, which are detachably connected to each other via a mount section 4.

The lens barrel 3 is an interchangeable lens that can be attached to and detached from the camera body 2. As shown in FIG. 1, the lens barrel 3 contains a photographic optical system that includes lenses 31, 32, and 33, and an aperture 34.

The lens 32 is a focus lens, and can adjust the focus state of the photographing optical system by moving in the direction of the optical axis L1. The focus lens 32 is movable along the optical axis L1 of the lens barrel 3, and its position is detected by the encoder 35 and adjusted by the focus lens drive motor 36.

The aperture 34 is configured to have an adjustable aperture diameter centered on the optical axis L1 in order to limit the amount of light passing through the photographing optical system and reaching the image sensor 22, and to adjust the amount of blur. The aperture diameter is adjusted by the aperture 34, for example, by sending an appropriate aperture diameter calculated in the automatic exposure mode from the camera control unit 21 via the lens control unit 37. In addition, the set aperture diameter is input from the camera control unit 21 to the lens control unit 37 by manual operation using the operation unit 28 provided on the camera body 2. The aperture diameter of the aperture 34 is detected by an aperture aperture sensor (not shown), and the current aperture diameter is recognized by the lens control unit 37.

The lens control unit 37 performs overall control of the lens barrel 3, such as driving the focus lens 32 and adjusting the aperture diameter of the diaphragm 34, based on commands from the camera control unit 21.

On the other hand, in the camera body 2, an image sensor 22 that receives light from the above-mentioned photographing optical system is provided at the planned focal plane of the photographing optical system, and a shutter 23 is provided in front of it. The image sensor 22 is composed of devices such as CCD and CMOS, and converts the received light signal into an electrical signal and sends it to the camera control unit 21. The photographed image information sent to the camera control unit 21 is sequentially sent to the liquid crystal drive circuit 25 and displayed on the electronic viewfinder (EVF) 26 of the observation optical system, and when a release button (not shown) provided on the operation unit 28 is fully pressed, the photographed image information is recorded in the memory 24, which is a recording medium. The memory 24 can be either a removable card-type memory or a built-in memory. The structure of the image sensor 22 will be described in detail later.

The camera body 2 is provided with an observation optical system for observing the image captured by the image sensor 22. The observation optical system of this embodiment includes an electronic viewfinder (EVF) 26 made of a liquid crystal display element, an LCD drive circuit 25 that drives it, and an eyepiece 27. The LCD drive circuit 25 reads the captured image information captured by the image sensor 22 and sent to the camera control unit 21, and drives the electronic viewfinder 26 based on this information. This allows the user to observe the currently captured image through the eyepiece 27. Note that instead of or in addition to the observation optical system based on the optical axis L2, a liquid crystal display can be provided on the back of the camera body 2, and the captured image can be displayed on this liquid crystal display.

The camera body 2 is provided with a camera control unit 21. The camera control unit 21 receives various lens information and transmits information such as the defocus amount and aperture diameter to the lens control unit 37. As described above, the camera control unit 21 also reads out pixel output from the image sensor 22, and generates image information by performing predetermined information processing on the read pixel output as necessary, and outputs the generated image information to the LCD drive circuit 25 of the electronic viewfinder 26 and the memory 24. The camera control unit 21 also controls the entire camera 1, such as correcting the image information from the image sensor 22 and detecting the focus adjustment state and aperture adjustment state of the lens barrel 3.

In addition to the above, the camera control unit 21 detects the focus state of the photographing optical system using a phase detection method and a contrast detection method based on the pixel data read from the image sensor 22. The specific method of detecting the focus state will be described later.

The operation unit 28 is an input switch such as a shutter release button that allows the photographer to set various operation modes of the camera 1, and allows switching between autofocus mode and manual focus mode. The various modes set by this operation unit 28 are sent to the camera control unit 21, which controls the operation of the entire camera 1. The shutter release button also includes a first switch SW1 that is turned ON when the button is pressed halfway, and a second switch SW2 that is turned ON when the button is pressed all the way.

Next, we will explain the imaging element 22 according to this embodiment.

Figure 2 is a front view showing the imaging surface of the image sensor 22, and Figure 3 is a front view enlarging part III in Figure 2 and showing a schematic arrangement of the imaging pixels 221 and the focus detection pixels 222a and 222b.

As shown in FIG. 3, the image sensor 22 of this embodiment has a plurality of image pixels 221 arranged two-dimensionally on the plane of the image sensor surface, and green pixels G having a color filter that transmits the green wavelength region, red pixels R having a color filter that transmits the red wavelength region, and blue pixels B having a color filter that transmits the blue wavelength region are arranged in a so-called Bayer arrangement. That is, in four adjacent pixel groups 223 (dense square lattice arrangement), two green pixels are arranged on one diagonal line, and one red pixel and one blue pixel are arranged on the other diagonal line. The image sensor 22 is configured by repeatedly arranging the pixel groups 223 in a two-dimensional manner on the image sensor surface of the image sensor 22, with the Bayer-arranged pixel groups 223 as units. In this embodiment, one pixel is configured by these four pixel groups 223.

The arrangement of the unit pixel groups 223 can be, for example, a dense hexagonal lattice arrangement other than the dense square lattice shown in the figure. Furthermore, the configuration and arrangement of the color filters are not limited to this, and an arrangement of complementary color filters (green: G, yellow: Ye, magenta: Mg, cyan: Cy) can also be used.

Fig. 4(A) is a front view showing an enlarged image pixel 221, and Fig. 4(D) is a cross-sectional view. One image pixel 221 is composed of a microlens 2211, a photoelectric conversion unit 2212, and a color filter (not shown). As shown in the cross-sectional view of Fig. 4(D), the photoelectric conversion unit 2212 is fabricated on the surface of the semiconductor circuit board 2213 of the image sensor 22, and the microlens 2211 is formed on the surface of the photoelectric conversion unit 2212. The photoelectric conversion unit 2212 is shaped to receive the image light beam passing through the exit pupil (e.g. F1.0) of the imaging optical system 31 by the microlens 2211, and receives the image light beam.

2, focus detection pixel column groups 22a-22i in which focus detection pixels 222a and 222b are arranged instead of the above-mentioned imaging pixel 221 are provided at the center of the imaging surface of the image sensor 22, at positions symmetrical to the left and right of the center, and at positions symmetrical to the top and bottom of the center, in total of nine positions. As shown in FIG. 3, each focus detection pixel column group is composed of four focus detection pixel columns L1-L4, and each focus detection pixel column L1-L4 is composed of a plurality of first focus detection pixels 222a and second focus detection pixels 222b arranged alternately adjacent to each other in a horizontal row. As shown in FIG. 3, in this embodiment, the focus detection pixels 222a and the focus detection pixels 222b are arranged so that they are reversed in the X-axis direction in the focus detection pixel columns L1 and L3 and the focus detection pixel columns L2 and L4. Furthermore, in this embodiment, as shown in FIG. 3, the first focus detection pixel 222a and the second focus detection pixel 222b are densely arranged without gaps at the positions of the green pixel G and the blue pixel B of the Bayer arrayed imaging pixel 221.

The positions of the focus detection pixel array groups 22a-22i shown in FIG. 2 are not limited to those shown in the figure, and they can be arranged in one location, or two to eight locations, or even in ten or more locations. Furthermore, during actual focus detection, the photographer can manually operate the operation unit 28 to select a desired focus detection pixel group from the multiple focus detection pixel array groups 22a-22i arranged as the focus detection area AFP for focus adjustment.

Figure 4(B) is an enlarged front view of one of the first focus detection pixels 222a, and Figure 4(E) is a cross-sectional view of the first focus detection pixel 222a. Also, Figure 4(C) is an enlarged front view of one of the second focus detection pixels 222b, and Figure 4(F) is a cross-sectional view of the second focus detection pixel 222b. As shown in Figure 4(B), the first focus detection pixel 222a is composed of a microlens 2221a and a rectangular photoelectric conversion unit 2222a, and as shown in the cross-sectional view of Figure 4(E), the photoelectric conversion unit 2222a is built into the surface of the semiconductor circuit board 2213 of the image sensor 22, and the microlens 2221a is formed on the surface. As shown in Fig. 4C, the second focus detection pixel 222b is composed of a microlens 2221b and a photoelectric conversion unit 2222b, and as shown in the cross-sectional view of Fig. 4F, the photoelectric conversion unit 2222b is fabricated on the surface of the semiconductor circuit board 2213 of the image sensor 22, and the microlens 2221b is formed on the surface of the photoelectric conversion unit 2222b. As shown in Fig. 3, these focus detection pixels 222a and 222b are arranged alternately adjacent to each other in a horizontal line to form each of the focus detection pixel columns L1 to L4.

The photoelectric conversion units 2222a, 2222b of the first focus detection pixel 222a and the second focus detection pixel 222b are shaped to receive light passing through a predetermined area (e.g., F2.8) of the exit pupil of the imaging optical system by the microlenses 2221a, 2221b. The first focus detection pixel 222a and the second focus detection pixel 222b are not provided with a color filter, and their spectral characteristics are a combination of the spectral characteristics of the photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). However, they can also be configured to have one of the same color filters as the imaging pixel 221, for example a green filter.

In addition, the photoelectric conversion units 2222a, 2222b of the first focus detection pixel 222a and the second focus detection pixel 222b shown in Figures 4(B) and 4(C) are rectangular, but the shape of the photoelectric conversion units 2222a, 2222b is not limited to this and can also be other shapes, such as a semicircular shape, an elliptical shape, or a polygonal shape.

Next, we will explain the so-called phase difference detection method, which detects the focus state of the imaging optical system based on the pixel output of the focus detection pixels 222a and 222b described above.

Figure 5 is a cross-sectional view taken along line V-V in Figure 3, and shows that adjacent focus detection pixels 222a-1, 222b-1, 222a-2, and 222b-2, which are arranged near the imaging optical axis L1, receive light beams AB1-1, AB2-1, AB1-2, and AB2-2 emitted from ranging pupils 351 and 352 of the exit pupil 350, respectively. Note that, of the multiple focus detection pixels 222a and 222b, only those located near the imaging optical axis L1 are shown as examples in Figure 5, but the other focus detection pixels other than the focus detection pixels shown in Figure 5 are similarly configured to receive light beams emitted from a pair of ranging pupils 351 and 352, respectively.

Here, the exit pupil 350 is an image set at a position a distance D in front of the microlenses 2221a and 2221b of the focus detection pixels 222a and 222b arranged on the planned focal plane of the imaging optical system. The distance D is a value that is uniquely determined depending on the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and the like, and this distance D is called the ranging pupil distance. In addition, the ranging pupils 351 and 352 refer to the images of the photoelectric conversion units 2222a and 2222b projected by the microlenses 2221a and 2221b of the focus detection pixels 222a and 222b, respectively.

In addition, in FIG. 5, the arrangement direction of the focus detection pixels 222a-1, 222b-1, 222a-2, and 222b-2 coincides with the arrangement direction of the pair of ranging pupils 351 and 352.

As shown in FIG. 5, the microlenses 2221a-1, 2221b-1, 2221a-2, and 2221b-2 of the focus detection pixels 222a-1, 222b-1, 222a-2, and 222b-2 are arranged near the planned focal plane of the imaging optical system. The shapes of the photoelectric conversion units 2222a-1, 2222b-1, 2222a-2, and 2222b-2 arranged behind the microlenses 2221a-1, 2221b-1, 2221a-2, and 2221b-2 are projected onto the exit pupil 350, which is separated from the microlenses 2221a-1, 2221b-1, 2221a-2, and 2221b-2 by the distance measurement distance D, and the projected shapes form the distance measurement pupils 351 and 352.

In other words, the relative positional relationship between the microlens and the photoelectric conversion unit in each focus detection pixel is determined so that the projection shape (measuring pupils 351, 352) of the photoelectric conversion unit of each focus detection pixel coincides on the exit pupil 350 at the measuring distance D, thereby determining the projection direction of the photoelectric conversion unit in each focus detection pixel.

As shown in FIG. 5, the photoelectric conversion unit 2222a-1 of the first focus detection pixel 222a-1 outputs a signal corresponding to the intensity of an image formed on the microlens 2221a-1 by the light beam AB1-1 that passes through the ranging pupil 351 and heads toward the microlens 2221a-1. Similarly, the photoelectric conversion unit 2222a-2 of the first focus detection pixel 222a-2 outputs a signal corresponding to the intensity of an image formed on the microlens 2221a-2 by the light beam AB1-2 that passes through the ranging pupil 351 and heads toward the microlens 2221a-2.

The photoelectric conversion unit 2222b-1 of the second focus detection pixel 222b-1 outputs a signal corresponding to the intensity of the image formed on the microlens 2221b-1 by the light beam AB2-1 that passes through the ranging pupil 352 and heads toward the microlens 2221b-1. Similarly, the photoelectric conversion unit 2222b-2 of the second focus detection pixel 222b-2 outputs a signal corresponding to the intensity of the image formed on the microlens 2221b-2 by the light beam AB2-2 that passes through the ranging pupil 352 and heads toward the microlens 2221b-2.

Then, by grouping the outputs of the photoelectric conversion units 2222a, 2222b of the focus detection pixels 222a, 222b described above into output groups corresponding to the ranging pupil 351 and the ranging pupil 352, respectively, data is obtained regarding the intensity distribution of a pair of images formed on the focus detection pixel row by the focus detection light beams passing through the ranging pupil 351 and the ranging pupil 352, respectively.

Then, the camera control unit 21 performs a correlation calculation shown in the following equation (1) while relatively shifting, in one dimension, the data strings relating to the intensity distribution of the pair of images, i.e., the data string based on the first focus detection pixel 222a and the data string based on the second focus detection pixel 222b among the focus detection pixel strings.
C(k) = Σ |I _{A(n + k)} -I _B(n) | ... (1)
In the above formula (1), the Σ calculation indicates an accumulation calculation (sum calculation) for n, and is limited to a range in which the data I _A(n+k) and I _B(n) exist according to the image shift amount k. The image shift amount k is an integer, and is a shift amount in units of the pixel spacing between the focus detection pixels 222a, 222b. In the calculation results of the above formula (1), the correlation amount C(k) becomes extremely small (the smaller the shift amount, the higher the degree of correlation).

Then, the correlation amount C(k) is calculated according to the above formula (1), and the defocus amount df is calculated according to the following formula (2) based on the shift amount x at which the correlation amount minimum value C(x) is obtained. Note that in the above formula (2), k is a conversion coefficient (k factor) for converting the shift amount x at which the correlation amount minimum value C(x) is obtained into a defocus amount.
df = x k ... (2)

Furthermore, in this embodiment, the camera control unit 21 detects contrast information from the output of multiple focus detection pixel rows L1 to L4, and determines the focus detection pixel row to be used in calculating the defocus amount as the specific focus detection pixel row based on the detected contrast information.

Specifically, the camera control unit 21 acquires the output of each focus detection pixel row L1 to L4 from the image sensor 22, and extracts high-frequency components from the output of the focus detection pixel rows L1 to L4 by filtering the acquired output of the focus detection pixel rows L1 to L4 with a high-frequency pass filter. The camera control unit 21 then compares the high-frequency components of each focus detection pixel row L1 to L4, and based on the comparison result, determines the focus detection pixel row of the multiple focus detection pixel rows L1 to L4 that corresponds to the subject with the highest contrast as the specific focus detection pixel row. The camera control unit 21 may also be configured to determine the focus detection pixel row of the multiple focus detection pixel rows L1 to L4 whose output contains the most high-frequency components as the specific focus detection pixel row based on the comparison result.

In this embodiment, the camera control unit 21 acquires the output of the multiple focus detection pixel rows L1 to L4 from the image sensor 22, and determines a specific focus detection pixel row from among the multiple focus detection pixel rows L1 to L4. However, the present invention is not limited to this configuration. For example, a calculation unit included in the image sensor 22 may detect contrast information for each point detection pixel row L1 to L4 based on the output of each focus detection pixel row L1 to L4, determine a specific focus detection pixel row from among the multiple focus detection pixel rows L1 to L4 based on the detected contrast information, and transmit information about the determined specific focus detection pixel row to the camera control unit 21.

The camera control unit 21 then calculates the defocus amount based on the output of the determined specific focus detection pixel row. In this way, in this embodiment, the defocus amount is calculated based only on the output of the focus detection pixel row determined as the specific focus detection pixel row out of the multiple focus detection pixel rows L1 to L4, so the time required to calculate the defocus amount can be shortened.

That is, in the past, when there were multiple focus detection pixel rows, the defocus amount was calculated for all of the focus detection pixel rows, and the defocus amount used to drive the focus lens 32 was selected from the multiple calculated defocus amounts. Therefore, in the past, not only was the calculation of defocus amounts other than the selected defocus amount wasted, but there was also the problem that the time required to calculate the defocus amount was long because the defocus amount was calculated for all of the multiple focus detection pixel rows L1 to L4. In contrast, in this embodiment, the defocus amount is calculated based only on the output of the focus detection pixel row determined as the specific focusing pixel row, so the time required to calculate the defocus amount can be shortened, and as a result, the time required for focus detection can be shortened.

In this embodiment, multiple focus detection areas AFP are set in the shooting screen of the shooting optical system corresponding to the focus detection pixel row groups 22a to 22i of the image sensor 22, and the photographer can set the focus detection area AFP to be used for focus adjustment via the operation unit 28. For example, when the photographer selects the focus detection area AFP corresponding to the focus detection pixel row group 22a as the focus detection area AFP to be used for focus adjustment, the camera control unit 21 calculates the defocus amount based on the output of the focus detection pixel rows L1 to L4 included in the focus detection pixel row group 22a. In addition, the method of setting the focus detection area AFP is not limited to the method selected by the photographer, and for example, the camera control unit 21 may be configured to perform face recognition processing based on image data output from the image sensor 22 to set the focus detection area AFP corresponding to the face of the subject as the focus detection area AFP to be used for focus adjustment. Alternatively, the outputs of the focus detection pixel rows L1 to L4 in all focus detection areas AFP set within the shooting screen may be acquired, and the focus detection area AFP to be used for focus adjustment may be set based on the acquired outputs of the focus detection pixel rows L1 to L4.

Furthermore, in this embodiment, the camera control unit 21 performs focus detection using a contrast detection method in addition to focus detection using the phase difference detection method described above. Specifically, the camera control unit 21 reads out the output of the imaging pixels 221 of the imaging element 22, and calculates a focus evaluation value based on the read pixel output. This focus evaluation value can be obtained, for example, by extracting high-frequency components of the image output from the imaging pixels 221 of the imaging element 22 using a high-frequency pass filter and integrating these. It can also be obtained by extracting high-frequency components using two high-frequency pass filters with different cutoff frequencies and integrating each of them.

The camera control unit 21 then sends a control signal to the lens control unit 37 to drive the focus lens 32 at a predetermined sampling interval (distance), calculates the focus evaluation value at each position, and determines the position of the focus lens 32 where the focus evaluation value is maximum as the in-focus position. Note that this in-focus position can be determined, for example, by calculating the focus evaluation value while driving the focus lens 32, and performing calculations such as interpolation using these focus evaluation values if the focus evaluation value rises twice and then falls twice.

Next, an example of the operation of the camera 1 in this embodiment will be described with reference to the flowchart shown in FIG.

First, in step S101, the image sensor 22 acquires output data from the image pickup pixel 221 and each of the first focus detection pixels 222a and second focus detection pixels 222b that make up the multiple focus detection pixel rows L1 to L4.

In step S102, the camera control unit 21 selects the focus detection area AFP to be used for focus adjustment. For example, if the photographer sets the focus detection area AFP to be used for focus adjustment via the operation unit 28, the camera control unit 21 can select the focus detection area AFP set by the photographer as the focus detection area AFP to be used for focus adjustment. The camera control unit 21 may also be configured to select the focus detection area AFP corresponding to the face of the subject as the focus detection area AFP to be used for focus adjustment by performing face recognition processing on the image data output from the image sensor 22. Alternatively, the camera control unit 21 may be configured to select the focus detection area AFP to be used for focus adjustment by analyzing the outputs of the focus detection pixel rows L1 to L4 of all focus detection areas AFP set within the shooting screen.

In step S103, the camera control unit 21 detects contrast information for each of the focus detection pixel rows L1 to L4 based on the output of each of the focus detection pixel rows L1 to L4. Specifically, the camera control unit 21 extracts high-frequency components from the pixel output of the focus detection pixel rows L1 to L4 by filtering the outputs of the multiple focus detection pixel rows L1 to L4 corresponding to the focus detection area AFP selected in step S102 using a high-frequency pass filter. The camera control unit 21 then detects information including the amount and intensity of the extracted high-frequency components as contrast information for each of the focus detection pixel rows L1 to L4.

In step S104, the camera control unit 21 determines the specific focus detection pixel row. Specifically, based on the contrast information for each of the focus detection pixel rows L1 to L4 detected in step S103, the camera control unit 21 determines, as the specific focus detection pixel row, the focus detection pixel row that corresponds to the subject with the greatest contrast among the multiple focus detection pixel rows L1 to L4, or the focus detection pixel row whose pixel output contains the most high-frequency components.

Then, in step S105, the camera control unit 21 calculates the defocus amount based on the output of the specific focus detection pixel row determined in step S104. Then, in step S106, the drive amount of the focus lens 32 is calculated and the focus lens 32 is driven based on the defocus amount calculated in step S105.

This is how focus detection is performed in the optical system according to the first embodiment.

Next, an example of the operation of the camera 1 according to this embodiment will be described with reference to FIG. 7. FIG. 7 is a diagram for explaining an example of the operation of the camera 1 according to this embodiment. In addition, in FIG. 7, the horizontal axis indicates time, and a scene in which the shutter release button is half-pressed at time t5 is shown. For example, in the example shown in FIG. 7, at time t1, the focus detection pixels 222a and 222b of the image sensor 22 start accumulating charges according to incident light. In addition, in this embodiment, the focus detection pixels 222a and 222b are, for example, CMOS image sensors, and in parallel with the accumulation of charges, the transfer of pixel signals according to the amount of charges accumulated after time t1 is started. Then, at time t3, the transfer of pixel signals started at time t2 ends, and the detection of contrast information and the determination of the specific focus detection pixel row are performed (steps S103 and S104). Then, at time t4, the calculation of the defocus amount is started based on the output of the determined specific focus detection pixel row (step S105). This causes the lens drive amount to be calculated, and after the lens drive amount is calculated, at time t6, a command to drive the lens is sent to the lens barrel 3, and the focus lens 32 is driven. Here, in the example shown in FIG. 7, the shutter release button is half-pressed at time t5, before the command to drive the lens is issued, so driving of the focus lens 32 begins based on the command to drive the lens at time t6.

In addition, in this embodiment, even after the shutter release button is half-pressed, the calculation of the defocus amount is repeated according to the frame rate of the image sensor 22, and a drive instruction for the focus lens 32 is repeatedly issued based on the calculated defocus amount. For example, in the example shown in FIG. 7, the accumulation of charge for the second frame begins at time t2, and as a result, the calculation of the defocus amount for the second frame is performed at time t7, and at time t8, a drive instruction for the focus lens 32 is issued based on the output result of the focus detection pixel rows L1 to L4 for the second frame. Similarly, from the third frame onwards, the calculation of the defocus amount and the drive of the focus lens 32 based on the defocus amount are repeated for each frame based on the output of the focus detection pixel rows L1 to L4.

In this manner, in this embodiment, the calculation of the defocus amount and the driving of the focus lens 32 based on the defocus amount are repeated for each frame according to the frame rate of the image sensor 22. In this manner, in order to calculate the defocus amount for each frame and instruct the driving of the focus lens 32, it is necessary to perform the process from the calculation of the defocus amount to the instruction to drive the focus lens 32 within the time of one frame. In this regard, in this embodiment, one specific focus detection pixel row is determined based on the contrast information, and the defocus amount is calculated only for the output of this specific focus detection pixel row. As shown in FIG. 7, this shortens the time required to calculate the defocus amount, so that the defocus amount can be calculated within the time of one frame and the focus lens 32 can be driven based on the calculated defocus amount.

On the other hand, FIG. 8 is a diagram for explaining an example of the operation of a conventional camera, and like FIG. 7, the horizontal axis shows time. In the example shown in FIG. 8, the calculation of the defocus amount, which takes a relatively long time, is performed for all focus detection pixel rows L1 to L4, so the calculation time of the defocus amount is longer than in the present embodiment shown in FIG. 7. As a result, in the example shown in FIG. 8, the time required from the calculation of the defocus amount to the lens drive instruction is longer than the time for one frame of the frame rate compared to the present embodiment shown in FIG. 7, and there were cases where the lens drive instruction could not be issued for each frame. Specifically, in the example shown in FIG. 8, since the calculation of the defocus amount and the lens drive instruction cannot be executed within the time for one frame, the time lag T2 from the accumulation of electric charge to the drive instruction of the focus lens 32 is twice as large as the time lag T1 of the present embodiment shown in FIG. 7.

As a result, in the example shown in FIG. 8, at time 12, an instruction to drive the focus lens 32 is issued based on the focus state of the optical system at time t11, so the focus lens 32 may be driven to a position far beyond the in-focus position or the ability to track the subject may be reduced. In contrast, in this embodiment, as shown in FIG. 7, an instruction to drive the focus lens 32 can be issued at time t10, which has less time lag, based on the focus state of the optical system at time t9, so the focus lens 32 can be appropriately driven to the in-focus position compared to the conventional example shown in FIG. 8.

As described above, in the first embodiment, contrast information is detected for each of the focus detection pixel rows L1 to L4 based on the output of each of the focus detection pixel rows L1 to L4 in the focus detection area AFP. Then, the focus detection pixel row to be used for focus detection is determined based on the detected contrast information. That is, among the multiple focus detection pixel rows L1 to L4, the focus detection pixel row corresponding to the subject with the highest contrast, or the focus detection pixel row whose pixel output contains the most high-frequency components, is determined as the specific focus detection pixel row, and the defocus amount is determined only for this specific focus detection pixel row. This makes it possible to detect the focus state of the optical system for the subject with the highest contrast or the subject with the most high-frequency components, and also makes it possible to reduce the time required to calculate the defocus amount compared to the conventional case in which the defocus amount is calculated for all of the multiple focus detection pixel rows L1 to L4, and to repeatedly detect the focus state at an appropriate timing.

In addition, in the first embodiment, the defocus amount is calculated based on the output of one of the multiple focus detection pixel rows L1 to L4, so the following effects can be achieved compared to adding or averaging the outputs of multiple focus detection pixel rows L1 to L4. That is, if the subject has contrast in the diagonal direction of the imaging pixel 221, adding or averaging the outputs of multiple focus detection pixel rows L1 to L4 may actually reduce the contrast, and the subject may not be detected properly. In contrast, in this embodiment, the defocus amount is calculated based on the output of one specific focus detection pixel row, so even in such cases, it is possible to prevent a decrease in contrast and to properly detect the subject.

Second Embodiment
Next, a second embodiment will be described. In the second embodiment, the camera 1 shown in FIG. 1 has the same configuration as the first embodiment described above, except that it operates as described below.

In the second embodiment, the camera control unit 21 is provided with multiple filters that transmit frequency components in different frequency bands, extracts frequency components in different frequency bands from the output of each focus detection pixel row L1 to L4, and detects multiple pieces of contrast information for each focus detection pixel row L1 to L4 based on the extracted frequency components. For example, if the camera control unit 21 is provided with three filters that extract frequency components in three different frequency bands, the camera control unit 21 can detect three pieces of contrast information from the output of one focus detection pixel row. Note that the frequency band of the frequency components extracted in the second embodiment is not particularly limited and can be any frequency band from the low frequency band to the high frequency band.

Then, based on the multiple contrast information, the camera control unit 21 detects one or more focus detection pixel rows L1 to L4 to be used for focus detection from among the multiple focus detection pixel rows L1 to L4 as specific focus detection pixel rows. For example, the camera control unit 21 can detect one or more focus detection pixel rows, from among the multiple focus detection pixel rows L1 to L4, for which the magnitude of contrast of the corresponding subject is equal to or greater than a predetermined value, as specific focus detection pixel rows. Alternatively, the camera control unit 21 can detect one or more focus detection pixel rows, for which the amount of high-frequency components included in the output is equal to or greater than a predetermined value, as specific focus detection pixel rows.

When determining the specific focus detection pixel rows, the camera control unit 21 determines the specific focus detection pixel rows so that the number of specific focus detection pixel rows is less than the number of focus detection pixel rows L1 to L4 corresponding to the focus detection area AFP. For example, in this embodiment, since four focus detection pixel rows L1 to L4 are arranged in the focus detection area AFP, the camera control unit 21 determines the specific focus detection pixel rows so that the number of specific focus detection pixel rows is at least three or less.

Then, when the camera control unit 21 determines a plurality of focus detection pixel rows as specific focus detection pixel rows, it adds or averages the outputs of the plurality of specific focus detection pixel rows, and calculates the defocus amount based on the added or averaged outputs of the plurality of specific focus detection pixel rows. Here, Fig. 9 is a schematic diagram showing only the first focus detection pixel 222a and the second focus detection pixel 222b constituting the four focus detection pixel rows L1 to L4, excluding the imaging pixel 221 from the imaging surface of the image sensor 22 shown in Fig. 3. For example, when adding the outputs of a plurality of specific focus detection pixel rows, the camera control unit 21 adds the output of pixel A1 L1 of the first focus detection pixel row _L1 shown in Fig. 9 to the output of pixel A1 _L2 of the second focus detection pixel row L2, the output of pixel A1 L3 of the third focus detection pixel row L3, and the output of pixel A1 _L4 of the fourth focus detection pixel row L4, which receive the focus detection light beam passing through _{the same ranging pupil} , to obtain a pixel addition output I _A1 . Similarly, the output of pixel B1L1 in the first focus detection pixel row _L1 is added to the output of pixel B1L2 in the second focus detection pixel row L2, the output of pixel _B1L3 in the third focus detection pixel row _L3 , and the output of pixel _B1L4 in the fourth focus detection pixel row L4, which receive the focus detection light beam passing through the same focus detection pupil, to obtain a pixel addition output _IB1 . Similarly, _IA2 is obtained from the outputs of _A2L1 , _A2L2 , _A2L3 , and _A2L4 , _IB2 is obtained from the outputs of _B2L1 , _B2L2 , _B2L3 , and _B2L4 , _IA3 is obtained from the outputs of _A3L1 , _A3L2 , _A3L3 , and _A3L4 , and _IB3 is obtained from the outputs of _B3L1 , _B3L2 , _B3L3 , and _B3L4 .

Then, the camera control unit 21 uses the obtained pixel addition output to perform the correlation calculation shown in the above equation ₍ 1) while relatively shifting in one dimension the data string based on the first focus detection pixel 222a, i.e., the first image data string I _A1 , I _A2 , I _A3 , ... , I _An , and the data string based on the second focus detection pixel 222b, i.e., the second image data string I _B1 , I B2 , I _B3 , ... , I _Bn .

3, in the focus detection pixel rows L1 to L4, the first focus detection pixel 222a and the second focus detection pixel 222b are disposed at positions shifted by 0.5 pixels from each other. Conventionally, only one of the first focus detection pixel row L1 and the second focus detection pixel row L2 of this embodiment is used as the focus detection pixel row, so the first focus detection pixel 222a and the second focus detection pixel 222b are disposed at positions shifted by 0.5 pixels from each other, and the first image data row I _A1 , I _A2 , I _A3 ,..., I _An and the second image data row I _B1 , I _B2 , I _B3 ,..., I _Bn obtained using these focus detection pixels are data shifted by 0.5 pixels from each other. Therefore, when the correlation calculation is performed, the minimum value of the correlation amount C(k) is also shifted by 0.5 pixels, as shown in Fig. 10B. In such a case, in order to calculate the shift amount and defocus amount at which the correlation amount C(k) shows the minimum value, it is necessary to use an interpolation calculation or the like, and as a result, the focus detection accuracy may be reduced. In particular, when the contrast level of the output detected by the focus detection pixel is low, such a problem tends to be prominent.

In contrast, in this embodiment, the first focus detection pixel 222a and the second focus detection pixel 222b are positioned in the focus detection pixel rows L1 and L3 and the focus detection pixel rows L2 and L4 at positions shifted by 0.5 pixels in the X-axis direction. Therefore, when the pixel sum output obtained by adding the pixel outputs is used, as shown in FIG. 10(A), it is possible to accurately determine the shift amount at which the correlation amount C(k) is at a minimum value in the in-focus state (a state in which the defocus amount is zero), thereby appropriately improving the focus detection accuracy.

Next, the operation of the camera 1 according to the second embodiment will be described with reference to FIG. 11. FIG. 11 is a flowchart showing an example of the operation of the camera 1 according to the second embodiment.

In step S201, similar to step S101 in the first embodiment, the image sensor 22 acquires output data from the image pickup pixel 221 and each of the first focus detection pixels 222a and second focus detection pixels 222b that make up the multiple focus detection pixel rows L1 to L4. In step S202, similar to step S102 in the first embodiment, the camera control unit 21 selects a focus detection area AFP to be used for focus adjustment.

Then, in step S203, the camera control unit 21 detects contrast information. In the second embodiment, the camera control unit 21 extracts multiple frequency components from the output of each of the focus detection pixel rows L1 to L4 by filtering the output of each of the focus detection pixel rows L1 to L4 using multiple filters that extract frequency components in different frequency bands. The camera control unit 21 then detects information including the amount and intensity of the extracted frequency components as contrast information.

In step S204, the camera control unit 21 determines a specific focus detection pixel row based on the multiple contrast information detected in step S203. Specifically, the camera control unit 21 determines, as the specific focus detection pixel row, one or more focus detection pixel rows among the multiple focus detection pixel rows L1 to L4 in which the contrast of the corresponding subject is equal to or greater than a predetermined value, based on the contrast information detected in step S203. Alternatively, the camera control unit 21 determines, as the specific focus detection pixel row, one or more focus detection pixel rows among the multiple focus detection pixel rows L1 to L4 in which the amount of high-frequency components included in the output is equal to or greater than a predetermined value, based on the multiple contrast information detected in step S203.

Then, in step S205, the camera control unit 21 calculates the defocus amount to be used in driving the focus lens 32 based on the one or more specific focus detection pixel rows determined in step S204. For example, if the camera control unit 21 determines multiple focus detection pixel rows as specific focus detection pixel rows, it can calculate a single output by adding or averaging the outputs of the multiple specific focus detection pixel rows, and calculate the defocus amount to be used in driving the focus lens 32 based on this output.

In step S206, similar to step S106 in the first embodiment, the drive amount of the focus lens 32 is calculated and the focus lens 32 is driven based on the defocus amount calculated in step S205.

This is how focus detection is performed in the optical system according to the second embodiment.

In this way, in the second embodiment, for each focus detection pixel row L1 to L4, frequency components in different frequency bands are extracted based on the output of the focus detection pixel rows L1 to L4, and multiple contrast information is detected based on the multiple extracted frequency components. Then, based on the multiple contrast information, one or multiple specific focus detection pixel rows are determined from the focus detection pixel rows L1 to L4, and the defocus amount is determined based on the pixel output of the determined specific focus detection pixel row. In this way, in the second embodiment, instead of calculating the defocus amount for all focus detection pixel rows L1 to L4, the defocus amount is calculated for a number of specific focus detection pixel rows that is fewer than the number of focus detection pixel rows L1 to L4, thereby shortening the defocus amount calculation time and making it possible to repeatedly detect the focus state at an appropriate timing.

Third Embodiment
Next, a third embodiment will be described. In the third embodiment, the camera 1 shown in FIG. 1 has the same configuration as the first embodiment described above, except that it operates as described below.

In the third embodiment, when detecting the focus state of the optical system, the camera control unit 21 performs filtering processing using a predetermined band-pass filter on the output of the focus detection pixel rows L1 to L4 acquired from the image sensor 22, thereby extracting frequency components corresponding to the subject from the output of the focus detection pixel rows L1 to L4. For example, the camera control unit 21 converts the output of the focus detection pixel rows L1 to L4 using a Fourier transform to remove low-frequency components due to the background and high-frequency components corresponding to noise from the frequency components contained in the output of the focus detection pixel rows L1 to L4, and extracts the frequency components corresponding to the subject. Then, as shown in FIG. 12, the camera control unit 21 detects the data obtained by filtering the output of the focus detection pixel rows L1 to L4 as contrast information representing the frequency components contained in the output of the focus detection pixel rows L1 to L4. Note that FIG. 12 is a graph showing an example of frequency components extracted from the output of the focus detection pixel rows L1 to L4.

Furthermore, the camera control unit 21 calculates the total sum of the differences between successive output values in the frequency components based on the contrast information of the detected frequency components, thereby calculating information indicating the amount and intensity of the frequency components included in the outputs of the focus detection pixel rows L1 to L4 as the contrast amount of the frequency components. Specifically, the camera control unit 21 calculates the contrast amount of the frequency components based on the following formula (3).
Contrast amount=Σ|a _m −a _m−1 | … (3)
In addition, in the above equation (3), a _m is the output value of the frequency component corresponding to the output of the focus detection pixel 222a, 222b located at the mth position (mth in the arrangement direction of the focus detection pixel row), and a _m-1 is the output value of the frequency component corresponding to the output of the focus detection pixel 222a, 222b located at the m-1th position.

That is, the outputs of the focus detection pixel rows L1 to L4 are row data composed of the outputs of a plurality of focus detection pixels 222a, 222b arranged in the horizontal direction (X direction), and the frequency components included in the outputs of the focus detection pixel rows L1 to L4 also have output values corresponding to the outputs of the focus detection pixels 222a, 222b constituting the focus detection pixel rows L1 to L4, respectively, as shown in Fig. 12. Therefore, as shown in the above formula (3), the camera control unit 21 can calculate the difference between the output value of the frequency component corresponding to the output of the focus detection pixel 222a, 222b located second in the focus detection pixel rows L1 to L4 and the output value of the frequency component corresponding to the output of the focus detection pixel 222a, 222b located first, and similarly, can calculate the difference between the output values a _m and a _m-1 of the frequency components corresponding to the outputs of successive focus detection pixels 222a, 222b (hereinafter also referred to as successive output values in the frequency components). Then, by finding the sum of the differences between successive output values a _m and a _m-1 in the frequency components, the contrast amount of the frequency components contained in the outputs of the focus detection pixel arrays L1 to L4 can be calculated.

The contrast amount of the frequency component increases as the intensity (amplitude) of the frequency component increases, and also increases as the amount (frequency) of the frequency component increases. For example, when the contrast of the subject corresponding to the focus detection pixel row is high and the intensity of the output value near the center of the frequency component increases as shown in FIG. 13(A), the difference between successive output values near the center of the frequency component also increases as shown in FIG. 13(B), and as a result, the contrast amount of the frequency component also increases. In addition, the greater the amount of the frequency component (for example, in the example shown in FIG. 12, the amount of frequency components in the focus detection pixel row L3 is greater than that in the focus detection pixel row L1), the more frequently the output value of the frequency component fluctuates, and therefore the contrast amount of the frequency component, which is the sum of the differences between successive output values in the frequency component, also increases. In this way, by calculating the sum of the differences between successive output values in the frequency component as the contrast amount of the frequency component, the contrast amount of the frequency component can be calculated as a value representing the amount and intensity of the frequency component. Note that FIG. 13(A) is a diagram showing an example of the output values of frequency components contained in the output of a focus detection pixel row, and FIG. 13(B) is a graph showing the difference between successive output values in the frequency components shown in FIG. 13(A).

In this way, the camera control unit 21 extracts a predetermined frequency component corresponding to the subject from the output of each of the focus detection pixel rows L1 to L4, and calculates the contrast amount of the extracted frequency component for each of the focus detection pixel rows L1 to L4. The camera control unit 21 then determines the focus detection pixel row with the highest contrast amount among the focus detection pixel rows L1 to L4 as the specific focus detection pixel row. For example, in the example shown in FIG. 12, among the focus detection pixel rows L1 to L4, the amount and intensity of frequency components contained in the output of the focus detection pixel row L3 are the largest, and as a result, the contrast amount of the frequency components contained in the output of the focus detection pixel row L3 is the highest, so the focus detection pixel row L3 is determined as the specific focus detection pixel row. The camera control unit 21 then calculates the defocus amount based on the output of the determined specific focus detection pixel row.

In this way, in the third embodiment, as shown in the above formula (3), the sum of the differences in the output values of the frequency components corresponding to the outputs of the focus detection pixels 222a, 222b that are consecutive in the arrangement direction in the focus detection pixel rows L1 to L4 is calculated as the contrast amount of the frequency components included in the outputs of the focus detection pixel rows L1 to L4. This makes it possible to appropriately calculate the contrast amount of the frequency components according to the amount and strength of the frequency components caused by the subject. Then, by calculating the defocus amount based on the output of the focus detection pixel row with the largest contrast amount, focus detection can be performed based on the output of the focus detection pixel row that is likely to correspond to the subject, and this makes it possible to detect the focus adjustment state of the optical system for the subject with higher accuracy.

Fourth Embodiment
Next, a fourth embodiment will be described. Fig. 14 is a diagram showing the configuration of a camera 1a according to the fourth embodiment. In the fourth embodiment, the camera 1a shown in Fig. 14 has the same configuration as the first embodiment described above, except that it operates as described below.

First, the camera 1a according to the fourth embodiment will be described. The camera 1a has a camera body 2a and a lens barrel 3. The camera body 2a is a single-lens reflex digital camera and is equipped with a mirror system 250 for directing light from a subject to an image sensor 220, an optical viewfinder 235, a photometry sensor 237, and a focus detection module 261. The mirror system 250 is equipped with a quick-return mirror 251 that rotates a predetermined angle around a rotation axis 253 between a subject observation position and an image capture position, and a sub-mirror 252 that is supported by the quick-return mirror 251 and rotates in accordance with the rotation of the quick-return mirror 251. In FIG. 14, the state in which the mirror system 250 is in the subject observation position is shown by a solid line, and the state in which it is in the subject image capture position is shown by a two-dot chain line.

When the mirror system 250 is in the observation position of the subject, it is inserted in the optical path of the optical axis L1, but when it is in the imaging position of the subject, it rotates so as to move out of the optical path of the optical axis L1.

The quick-return mirror 251 is a half mirror, and when it is in the subject observation position, a portion of the light beam (optical axis L1) from the subject is reflected by the quick-return mirror 251 (optical axes L2 and L3) and directed to the optical viewfinder 235 and photometry sensor 237, while a portion of the light beam (optical axis L4) is transmitted and directed to the sub-mirror 252. In contrast, the sub-mirror 252 is a total reflection mirror, and directs the light beam (optical axis L4) that has transmitted through the quick-return mirror 251 to the focus detection module 261.

Therefore, when the mirror system 250 is in the observation position, the light beam (optical axis L1) from the subject is guided to the optical viewfinder 235, photometry sensor 237, and focus detection module 261, and the photographer observes the subject while exposure calculations and detection of the focus adjustment state of the focus lens 32 are performed by the focus detection module 261. Then, when the photographer fully presses the release button, the mirror system 250 rotates to the shooting position, and all of the light beam (optical axis L1) from the subject is guided to the image sensor 220, and the captured image data is stored in the camera memory 24.

The light beam (optical axis L2) from the subject reflected by the quick return mirror 251 is focused on the focusing screen 231, which is arranged on a plane optically equivalent to the image sensor 220, and can be observed through the pentaprism 233 and eyepiece 27. At this time, the transmissive LCD display 232 displays a focus detection area mark and the like superimposed on the subject image on the focusing screen 231, and also displays information related to shooting, such as the shutter speed, aperture value, and number of shots, in the area outside the subject image. This allows the photographer to observe the subject and its background, as well as shooting-related information, through the optical viewfinder 235 when in preparation for shooting.

The photometry sensor 237 is composed of a two-dimensional color CCD image sensor or the like, and divides the shooting screen into multiple areas and outputs a photometry signal according to the brightness of each area in order to calculate the exposure value when shooting. The signal detected by the photometry sensor 237 is output to the camera control unit 21 and used for automatic exposure control, recognition of specific subjects (e.g. face recognition), etc.

The focus detection module 261 is a dedicated focus detection element that performs automatic focusing control using a phase difference detection method, and is fixed at a position of the light beam (optical axis L4) reflected by the sub-mirror 252 that is optically equivalent to the imaging surface of the image sensor 220.

Figure 15 is a diagram showing an example of the configuration of the focus detection module 261 shown in Figure 14. The focus detection module 261 of this embodiment has a condenser lens 261a, an aperture mask 261b in which a pair of apertures are formed, a pair of reimaging lenses 261c, and a pair of line sensors 261d. Although not shown, the line sensor 261d of this embodiment has a pixel row in which a plurality of pixels having a microlens arranged near the intended focal plane of the imaging optical system and a photoelectric conversion unit arranged relative to this microlens are arranged. A pair of image signals can be obtained by receiving a pair of light beams passing through a pair of different areas of the exit pupil of the focus lens 32 at each pixel arranged in the pair of line sensors 261d. Then, the phase shift of the pair of image signals acquired by the pair of line sensors 261d can be obtained by a correlation calculation described later, thereby detecting the focus adjustment state.

For example, as shown in FIG. 15, when the subject P is imaged on the equivalent surface (intended imaging surface) 261e of the image sensor 220, the subject is in focus. However, if the focus lens 32 moves in the direction of the optical axis L1, and the imaging point shifts from the equivalent surface 261e toward the subject (called front focus) or toward the camera body (called back focus), the subject becomes out of focus.

Next, the image sensor 220 according to the fourth embodiment will be described. FIG. 16 is a front view showing the image sensor 220 according to the fourth embodiment, and FIG. 17 is a front view showing an enlarged view of one of the focus detection areas 224 shown in FIG. 16. As shown in FIG. 16, the image sensor 220 according to the fourth embodiment has focus detection areas 224a to 224i for focus detection set at positions corresponding to the focus detection area AFP set on the shooting screen, that is, at the center of the imaging surface of the image sensor 220, positions symmetrical to the left and right of the center, and positions symmetrical above and below these, for a total of nine positions.

In the focus detection areas 224a to 224i, as shown in FIG. 17, a plurality of imaging pixels 221a are arranged two-dimensionally on the plane of the imaging surface. The imaging pixels 221a include green pixels G having color filters that transmit light in the green wavelength region, red pixels R having color filters that transmit light in the red wavelength region, and blue pixels B having color filters that transmit light in the blue wavelength region, and the individual pixels are arranged in a Bayer array.

Next, the configuration of the imaging pixel 221a will be described. FIG. 18(A) is a front view showing an enlarged image of one of the imaging pixels 221a, and FIG. 18(B) is a cross-sectional view. As shown in FIG. 18(A), the imaging pixel 221a is composed of a microlens 2211, a pair of photoelectric conversion units 2214 and 2215, and a color filter (not shown). As shown in the cross-sectional view of FIG. 18(B), the photoelectric conversion units 2214 and 2215 are built into the surface of the semiconductor circuit board 2213 of the imaging element 220, and the microlens 2211 is formed on the surface. The pair of photoelectric conversion units 2214 and 2215 are the same size and are arranged symmetrically with respect to the optical axis of the microlens 2211. In addition, the pair of photoelectric conversion units 2214 and 2215 are each shaped to receive the imaging light beam passing through the exit pupil (for example, F1.0) of the imaging optical system 31 by the microlens 2211. As shown in FIG. 18B, one photoelectric conversion unit 2215 of the imaging pixel 221a receives one light beam AB1, while the other photoelectric conversion unit 2214 of the imaging pixel 221a receives a light beam AB2 that is symmetrical to the light beam AB1 with respect to the optical axis of the microlens 2211.

The photoelectric conversion units 2214, 2215 of each imaging pixel 221a receive the pair of light beams AB1, AB2 and output a pair of image signals corresponding to the intensities of the received light beams AB1, AB2. In other words, the photoelectric conversion units 2214, 2215 output a pair of image signals corresponding to the intensity of the image formed on the microlens 2211 by the light beams AB1, AB2. In this embodiment, the pair of image signals output from the photoelectric conversion units 2214, 2215 of the imaging pixel 221a are added together to generate a pixel signal of the imaging pixel 221a, and image data is generated from the pixel signals of the multiple imaging pixels 221a.

Note that, although the photoelectric conversion units 2214 and 2215 shown in FIG. 18(A) are rectangular, the shape of the photoelectric conversion units 2214 and 2215 is not limited to this and can also be other shapes, such as an ellipse, a semicircle, or a polygon.

Next, a phase difference focus detection method (hereinafter also referred to as image plane phase difference detection) using a pair of image signals output from the photoelectric conversion units 2214, 2215 of the imaging pixels 221a will be described. In the following, as shown in FIG. 17, a plurality of imaging pixels 221a arranged in the horizontal direction (X direction) are regarded as an imaging pixel row 225, and as described later, a pair of image data is generated from one imaging pixel row 225 by combining the outputs of the photoelectric conversion units 2214, 2215 of each imaging pixel 221a constituting the imaging pixel row 225, and a defocus amount is calculated based on the generated pair of image data.

Figure 19 is a cross-sectional view taken along line XIX-XIX in Figure 17, and shows that the imaging pixel 221a-1 arranged on the imaging optical axis L and the adjacent imaging pixel 221a-2 receive light beams AB1-1, AB1-2, AB2-1, and AB2-2 emitted from the ranging pupils 341 and 342 of the exit pupil 340. However, although not shown, pairs of photoelectric conversion units of the other imaging pixels also receive pairs of light beams emitted from the pair of ranging pupils 341 and 342. Note that in Figure 19, the arrangement direction of the photoelectric conversion units 2214-1, 2225-1, 2214-2, and 2215-2 of the imaging pixel 221a coincides with the arrangement direction of the pair of ranging pupils 351 and 352.

The photoelectric conversion unit 2214-1 of the imaging pixel 221a-1 outputs a signal corresponding to the intensity of the image formed on the microlens 2211-1 by one light beam AB1-1 that passes through one ranging pupil 341 and heads toward the microlens 2211-1. In response to this, the photoelectric conversion unit 2215-1 outputs a signal corresponding to the intensity of the image formed on the microlens 2211-1 by the other light beam AB2-1 that passes through the other ranging pupil 342 and heads toward the microlens 2211-1.

Similarly, the photoelectric conversion unit 2214-2 of the imaging pixel 221a-2 outputs a signal corresponding to the intensity of the image formed on the microlens 2211-2 by one light beam AB1-2 that passes through one ranging pupil 341 and heads toward the microlens 2211-2. In response to this, the photoelectric conversion unit 2215-2 outputs a signal corresponding to the intensity of the image formed on the microlens 2211-2 by the other light beam AB2-2 that passes through the other ranging pupil 342 and heads toward the microlens 2211-2.

Then, by grouping the outputs of the photoelectric conversion units 2214, 2215 of each imaging pixel 221a constituting the imaging pixel row 225 into output groups corresponding to the ranging pupil 341 and ranging pupil 342, respectively, it is possible to obtain data on the intensity distribution of a pair of images formed on the imaging pixel row 225 by the light beams AB1, AB2 passing through the ranging pupil 341 and ranging pupil 342, respectively, that is, a pair of image data. For example, in the example shown in FIG. 17, eight or more imaging pixel rows 225 are set, and the camera control unit 21 can obtain a pair of image data from each imaging pixel row 225.

The camera control unit 21 can detect the amount of image shift caused by the pupil division phase difference detection method by performing image shift detection calculation processing such as correlation calculation processing or phase difference detection processing on a pair of image data obtained from each imaging pixel row 225. The camera control unit 21 can then obtain the amount of defocus by performing a conversion calculation on the obtained amount of image shift according to the distance between the centers of gravity of a pair of ranging pupils.

In this way, the camera control unit 21 can calculate the defocus amount for each imaging pixel row 225 based on a pair of image data obtained from each imaging pixel row 225. However, in the fourth embodiment, in order to reduce the time required for focus detection and perform focus detection at an appropriate timing, the camera control unit 21 identifies the imaging pixel row 225 to be used for focus detection, and calculates the defocus amount based only on the output of the identified imaging pixel row 225.

Specifically, the camera control unit 21 first extracts frequency components corresponding to the subject from the output of each imaging pixel row 225, and detects information related to the extracted frequency components as contrast information. For example, as in the third embodiment, the camera control unit 21 applies filtering to the output of each imaging pixel row 225 using a predetermined band-pass filter to remove low-frequency components caused by the background, etc., and high-frequency components corresponding to noise from the frequency components included in the output of the imaging pixel row 225, and extracts the frequency components corresponding to the subject. Then, the camera control unit 21 can detect, for example, data obtained by filtering the output of the imaging pixel row 225 as contrast information of the frequency components included in the output of the imaging pixel row 225.

The camera control unit 21 also calculates a contrast amount representing the amount and intensity of the frequency components based on the contrast information of the extracted frequency components. Here, FIG. 20 is an enlarged front view of one of the focus detection areas 224a-224i shown in FIG. 16, similar to FIG. 17, and FIG. 21 is a diagram showing the imaging pixels 221a of the first group LA shown in FIG. 20. In FIGS. 20 and 21, the imaging pixel rows 225 are represented as imaging pixel rows A1-A5, B1-B5, C1-C5, D1-D5, and E1-E5, respectively. In the examples shown in FIGS. 21(A) and 21(B), each imaging pixel row will be described as having 12 imaging pixels 221a.

First, the camera control unit 21 groups the multiple imaging pixel rows 225 set in the focus detection area 224 into groups of a predetermined number of imaging pixel rows 225 (for example, five imaging pixel rows 225). For example, in the example shown in FIG. 20, the camera control unit 21 groups each of five imaging pixel rows 225 consecutive in the vertical direction (Y direction), such as a first group LA consisting of imaging pixel rows A1 to A5, a second group LB consisting of imaging pixel rows B1 to B5, a third group LC consisting of imaging pixel rows C1 to C5, a fourth group LD consisting of imaging pixel rows D1 to D5, and a fifth group LE consisting of imaging pixel rows E1 to E5. Note that the number of imaging pixel rows 225 to be grouped is not limited to five rows and can be set as appropriate.

The camera control unit 21 then calculates the output value of the frequency components in each group by adding up the output values of the frequency components included in the output of the imaging pixel rows 225 belonging to the same group for each group, and detects it as contrast information of the frequency components in each group. For example, in the example shown in FIG. 21(A), the camera control unit 21 calculates the output value a1 by adding up the output values of the frequency components corresponding to the output of the first imaging pixel 221a in each of the imaging pixel rows A1 to A5. Similarly, the camera control unit 21 calculates the output value a2 by adding up the output values of the frequency components corresponding to the output of the second imaging pixel 221a in each of the imaging pixel rows A1 to A5. Similarly, the camera control unit 21 calculates the output values a3, a4, ..., a12 for the third and subsequent imaging pixels 221a in each of the imaging pixel rows A1 to A5, by adding up the output values of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the horizontal direction (X direction). This allows the camera control unit 21 to detect the output values a1, a2, a3, ..., a12 as contrast information for the frequency components in the first group LA.

The camera control unit 21 also detects contrast information of the frequency components in the second group LB, the third group LC, the fourth group LD, etc., in the same way as the first group LA. The camera control unit 21 then calculates the contrast amount of the frequency components in each group based on the contrast information of the frequency components of each group. For example, in the example shown in FIG. 21(A), the camera control unit 21 can calculate the contrast amount of the frequency components in the first group LA by calculating the sum of the difference between a1 and a2, the difference between a2 and a3, ..., the difference between a11 and a12 using the output values a1, a2, a3, ..., a12 of the frequency components in the first group LA, as shown in the above formula (3).

In the above example, the output values of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the horizontal direction (X direction) are added to calculate the output value of the frequency components in each group. However, the present invention is not limited to this configuration. For example, the output values of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the horizontal direction (X direction) may be averaged to calculate the output value of the frequency components in each group. For example, when the brightness or contrast of the subject is relatively low, the output values of the frequency components corresponding to the output of the imaging pixels 221a are added to calculate the output value of the frequency components in each group, and when the brightness or contrast of the subject is relatively high, the output values of the frequency components corresponding to the output of the imaging pixels 221a are averaged to calculate the output value of the frequency components in each group.

Also, as shown in Fig. 21B, the output value of the frequency component in each group may be calculated by adding or averaging the output values of the frequency components corresponding to the output of the photoelectric conversion units 2214 and 2215 arranged at the same position in the horizontal direction (X direction). That is, as shown in Fig. 21B, the camera control unit 21 calculates the output value b1 by adding or averaging the output values of the frequency components corresponding to the output of the photoelectric conversion unit 2214 of the first imaging pixel 221a in each of the imaging pixel rows A1 to A5. Similarly, the camera control unit 21 calculates the output value c1 by adding or averaging the output values of the frequency components corresponding to the output of the photoelectric conversion unit 2215 of the first imaging pixel 221a in each of the pixel rows A1 to A5. Similarly, the camera control unit 21 calculates output values b2, c2, b3, c3, ..., b12, and c12 for the photoelectric conversion units 2214 and 2215 of the second and subsequent imaging pixels 221a by adding or averaging the output values of the frequency components corresponding to the outputs of the photoelectric conversion units 2214 and 2215 arranged at the same position in the horizontal direction (X direction) in each of the pixel columns A1 to A5. This allows the camera control unit 21 to obtain output values consisting of b1, c1, b2, c2, b3, c3, ..., b12, and c12 as contrast information of the frequency components in the first group LA. Similarly, the camera control unit 21 obtains contrast information of the frequency components in the groups LB, LC, LD, LE, ... as shown in FIG. 22. Note that FIG. 22 is a diagram showing an example of the frequency components in each of the groups LA, LB, ..., LE shown in FIG. 20.

Then, the camera control unit 21 calculates the contrast amount of the frequency components in each group based on the contrast information of the frequency components in each group. For example, in the example shown in FIG. 21B, the camera control unit 21 can calculate the sum of the differences between consecutive output values of the frequency components in the first group LA as the contrast amount of the frequency components, as shown in the above formula (3). The camera control unit 21 may also calculate column data by adding the output values b1 and c1, b2 and c2, b3 and c3, ..., b12 and c12, respectively, and calculate the sum of the differences between consecutive values in the calculated column data as the contrast amount of the frequency components. Alternatively, the camera control unit 21 may be configured to separate the output values b1, c1, b2, c2, b3, c3, ..., b12, and c12 into column data consisting of the output values b1, b2, b3, ..., b12, and column data consisting of the output values c1, c2, c3, ..., c12, calculate the contrast amount of the frequency components in each column data, and calculate the larger or smaller contrast amount as the contrast amount of the frequency components in each group.

In addition, in the above example, a configuration was exemplified in which contrast information of frequency components in each group was detected by adding or averaging all frequency components included in the output of all imaging pixel rows 225 belonging to the same group, but this configuration is not limited to this, and a configuration in which the number of frequency components to be added or averaged is changed based on, for example, the brightness of the subject scene, the luminance of the image, or the contrast of the image may be changed. For example, a configuration in which the number of frequency components to be added is reduced as the brightness of the subject scene, the luminance of the image, or the contrast of the image is higher, and the number of frequency components to be added is increased as the brightness of the subject scene, the luminance of the image, or the contrast of the image is lower.

Then, the camera control unit 21 determines the group with the largest contrast amount among the contrast amounts of the frequency components calculated for each group as a specific group, and calculates the defocus amount based on the output of the imaging pixels 221a included in the specific group. For example, in the example shown in FIG. 22, the amount and intensity of the frequency components in the third group LC are large, so that the contrast amount of the frequency components in the third group LC is the largest among the groups LA to LE. In this case, the camera control unit 21 identifies the third group LC as a specific group, and calculates the defocus amount based on the output of the imaging pixels 221a included in the third group.

For example, as shown in FIG. 21B, the camera control unit 21 calculates a pair of image data (image data consisting of output values b1, b2, b3, ... and image data consisting of output values c1, c2, c3, ...) by adding or averaging the outputs of the photoelectric conversion units 2214, 2215 included in a specific group that are arranged at the same position in the horizontal direction (X direction). Then, as shown in the above formula (1), the camera control unit 21 calculates the correlation amount C(k) of the pair of image data by performing a correlation calculation while relatively shifting the calculated pair of image data, and can calculate the defocus amount as shown in the above formula (2) based on the calculated correlation amount C(k). Then, the camera control unit 21 can adjust the focus of the optical system by driving the focus lens 32 based on the calculated defocus amount.

In the fourth embodiment, the camera control unit 21 can control the driving of the focus lens 32 based on the defocus amount calculated by the focus detection module 261, and can also control the driving of the focus lens 32 based on the defocus amount calculated based on the output of the image sensor 220. For example, when the brightness of the light received by the image sensor 220 is low, the camera control unit 21 can control the driving of the focus lens 32 based on the defocus amount calculated by the focus detection module 261. This is because the size of each pixel constituting the line sensor 261d of the focus detection module 261 is designed to be larger than the image pixel 221a, so that even when the light intensity of the light beam is relatively weak, focus detection can be performed with relatively high accuracy. On the other hand, for example, when shooting a video, the driving of the focus lens 32 can be controlled based on the defocus amount calculated based on the output of the image sensor 220. This is because, in image plane phase difference detection based on the output of the image sensor 220, focus detection of the optical system can be performed while capturing an image with the image sensor 220 even during video shooting.

Next, the operation of the camera 1a according to the fourth embodiment will be described. FIG. 23 is a flowchart showing an example of the operation of the camera 1a according to the fourth embodiment. Note that in the example of the operation of the camera 1a shown in FIG. 23, a scene in which the camera control unit 21 performs focus detection based on the output of the image sensor 220 will be described as an example.

First, in step S401, the image sensor 220 acquires output data from the image pixel 221a (the photoelectric conversion units 2214 and 2215 of the image pixel 221a). In step S402, the camera control unit 21 selects a focus detection area AFP to be used for focus adjustment. For example, the photographer can manually operate the operation unit 28 to select the focus detection area AFP for focus adjustment. As a result, the camera control unit 21 determines the focus detection area 224 corresponding to the selected focus detection area AFP as the focus detection area 224 for focus detection.

Then, in step S403, the camera control unit 21 sets a plurality of imaging pixels 221a arranged one-dimensionally in the horizontal direction (X direction) as imaging pixel rows 225 in the focus detection area 224 determined in step S402, and detects contrast information of the frequency components contained in the output of each imaging pixel row 225.

Then, in step S404, the camera control unit 21 groups the multiple imaging pixel rows 225 in the focus detection area 224 into groups of a predetermined number of imaging pixel rows 225. For example, as shown in FIG. 20, the camera control unit 21 can group the multiple imaging pixel rows 225 into groups LA, LB, LC, LD, LE, ... for each group of five imaging pixel rows.

In step S405, the camera control unit 21 adds up the output values of the frequency components included in the output of the imaging pixel rows 225 belonging to the same group, thereby detecting contrast information of the frequency components in each group. Specifically, as shown in FIG. 21(A), the camera control unit 21 detects contrast information of the frequency components in each group by adding up the output values of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the horizontal direction (X direction) in multiple imaging pixel rows 225 belonging to the same group.

Then, in step S406, the camera control unit 21 calculates the contrast amount of the frequency components in each group based on the contrast information of the frequency components in each group detected in step S405. For example, the camera control unit 21 can calculate the contrast amount of the frequency components in each group from the output values of the frequency components in each group based on the above formula (3).

In step S407, the camera control unit 21 compares the contrast amounts of the frequency components calculated for each group in step S406, and the group with the highest contrast amount is determined as the specific group for focus detection. Then, in step S408, the camera control unit 21 calculates the defocus amount based on the output of the imaging pixels 221a included in the specific group determined in step S407. For example, as shown in FIG. 21B, the camera control unit 21 calculates a pair of image data by adding the outputs of the photoelectric conversion units 2214 and 2215 arranged at the same position in the horizontal direction (X direction) in each imaging pixel row 225 in the specific group. Then, the camera control unit 21 performs a correlation calculation of the calculated pair of image data, and calculates the defocus amount based on the calculation result. Then, in step S409, the camera control unit 21 drives the focus lens 32 based on the defocus amount calculated in step S408.

This is how focus detection is performed in the optical system according to the fourth embodiment.

In this way, in the fourth embodiment, as shown in FIG. 17, a plurality of imaging pixels 221a each having a pair of photoelectric conversion units 2214, 2215 are two-dimensionally arranged on the image sensor 220. Among the plurality of imaging pixels 221a, a plurality of imaging pixels 221a arranged in the horizontal direction (X direction) are set as an imaging pixel row 225, and the outputs of the photoelectric conversion units 2214, 2215 of the imaging pixels 221a constituting each imaging pixel row 225 are collected into output groups corresponding to the ranging pupil 351 and the ranging pupil 352 to obtain a pair of image data, and an image plane phase difference detection is performed to calculate a defocus amount based on the obtained pair of image data. As a result, in the fourth embodiment, the focus state of the optical system can be appropriately detected even during video shooting, for example.

In the fourth embodiment, the imaging pixel rows 225 are grouped into groups of a predetermined number of imaging pixel rows 225, and the output values of the frequency components included in the output of the imaging pixel rows 225 belonging to the same group are added for each group to detect contrast information of the frequency components in each group. Then, based on the contrast information of the frequency components in each group, the contrast amount of the frequency components in each group is calculated, the group with the highest contrast amount is determined as a specific group, and focus detection is performed based on the output of the imaging pixels 221a in the specific group. As a result, in the fourth embodiment, focus detection can be performed in a group that contains a large number of high-frequency components and is likely to contain a subject, so that the focus state of the optical system with respect to the subject can be detected more appropriately. In the fourth embodiment, the group with the highest contrast amount is determined as a specific group, and image plane phase difference detection is performed based on the output of the imaging pixels 221a in the specific group, so that the time required to calculate the defocus amount can be shortened compared to the case where the defocus amount is calculated based on the output of all the imaging pixels 221a, and focus detection can be performed at an appropriate timing.

Fifth Embodiment
Next, a fifth embodiment will be described. In the fifth embodiment, a camera 1a shown in Fig. 14 has a similar configuration to the fourth embodiment described above, except that it operates as described below.

In the image sensor 220 according to the fifth embodiment, similar to the fourth embodiment, the image sensor 220 has image pixels 221a, each having a pair of photoelectric conversion units 2214, 2215, arranged two-dimensionally. In the fifth embodiment, the camera control unit 21 detects a specific subject, such as a person's face, and extracts the output of the image pixels 221a corresponding to the specific subject at a ratio according to the color indicating the specific subject, thereby extracting the frequency component corresponding to the specific subject.

Specifically, the camera control unit 21 first detects a specific subject, such as a human face, by performing template matching using a template image, such as a human face. Then, the camera control unit 21 identifies the area of the image sensor 220 that corresponds to the specific subject as the target area. Note that the camera control unit 21 may be configured to identify the target area within the focus detection areas 224a to 224i, or may be configured to identify the target area in the entire image sensor 220.

Then, the camera control unit 21 detects whether or not a target area exists in the focus detection area 224 selected by the user or the camera control unit 21. That is, when a focus detection area AFP for performing focus adjustment is selected by the user or the camera control unit 21, the camera control unit 21 determines the focus detection area 224 corresponding to the focus detection area and determines whether or not a target area exists in the focus detection area 224. Then, when a target area exists in the focus detection area 224, the camera control unit 21 specifies a pixel group 223 corresponding to the target area in the focus detection area 224. Furthermore, the camera control unit 21 sets a pixel group row 226 including a pixel group 223 corresponding to the target area. For example, as shown in FIG. 24, the camera control unit 21 can set a row in which a plurality of pixel groups 223 including a pixel group 223 corresponding to the target area are arranged one-dimensionally in the horizontal direction (X direction) as the pixel group row 226. Note that FIG. 24 is a diagram that illustrates the imaging pixels 221a of the first group LA shown in FIG. 20. In addition, in the example shown in FIG. 24, the pixel column 226 is described as having six pixel groups 223.

Then, the camera control unit 21 extracts a frequency component corresponding to the pixel group array 226 from the output of the pixel group array 226, and detects contrast information related to the extracted frequency component. Specifically, the camera control unit 21 first extracts the output values of the four imaging pixels 221a (including R pixels, G pixels, and B pixels) constituting the pixel group 223 at a ratio according to a color indicating a specific subject such as a person's face (for example, the skin color of the specific subject). For example, the camera control unit 21 can extract the output values of the R pixels and G pixels higher than the output of the B pixels so as to have a ratio according to the skin color of the specific subject. Then, the camera control unit 21 detects the output value of the pixel group 223 by adding or averaging the output values extracted from the four imaging pixels 221a (R pixels, G pixels, and B pixels) constituting the pixel group 223. Furthermore, the camera control unit 21 detects the output of the pixel group array 226 consisting of a plurality of pixel groups 223 by detecting the output values of the pixel groups 223 constituting each pixel group array 226 along the arrangement direction of the pixel groups 223. The camera control unit 21 then performs filtering on the output of the pixel group array 226 using a predetermined band-pass filter to extract frequency components corresponding to the subject from the output of the pixel group array 226, and detects contrast information of the extracted frequency components.

Furthermore, the camera control unit 21 groups the multiple pixel group arrays 226 corresponding to the target area into groups of a predetermined number of pixel group arrays 226, and detects contrast information of the frequency components in each group by adding up the frequency components corresponding to the output of the pixel group arrays 226 that belong to the same group. For example, in the example shown in FIG. 24, the pixel group arrays 226 that belong to the same group are represented as A1' to A3'. In this case, the camera control unit 21 obtains an output value d1 by adding up the output values of the frequency components corresponding to the output of the first pixel group 223 in each pixel group array A1' to A3'. Similarly, the camera control unit 21 obtains an output value d2 by adding up the output values of the frequency components corresponding to the output of the second pixel group 223 in each pixel group array A1' to A3'. Similarly, for the third and subsequent pixel groups 223 from the left, the camera control unit 21 can obtain output values d3, d4, d5, and d6 by adding the output values of the frequency components corresponding to the output of each pixel group 223, and detect the obtained output values d1, d2, d3, d4, d5, and d6 as contrast information of the frequency components in the group. Note that the camera control unit 21 may be configured to detect contrast information of the frequency components in each group by averaging the output values of the frequency components corresponding to the output of pixel groups 223 arranged at the same position in the horizontal direction (X direction).

Then, the camera control unit 21 calculates the contrast amount of the frequency components in each group based on the contrast information of the frequency components in each group. Furthermore, when there are multiple groups corresponding to the target area in the focus detection area 224, the camera control unit 21 similarly calculates the contrast amount of the frequency components in the other groups. Then, the camera control unit 21 determines the group with the highest contrast amount among the groups for which the contrast amount has been calculated as a specific group, and calculates the defocus amount based on the output of the imaging pixels 221a in the specific group. Note that the method of calculating the defocus amount in the specific group can be performed in the same manner as in the fourth embodiment.

Next, the operation of the camera 1 according to the fifth embodiment will be described with reference to FIG. 25. Note that FIG. 25 is a flowchart showing an example of the operation of the camera 1 according to the fifth embodiment.

First, in step S501, the camera control unit 21 detects a target area corresponding to a specific subject. For example, the camera control unit 21 can detect a specific subject by performing template matching using a template image of a specific subject such as a person's face, and detect the area of the image sensor 220 that corresponds to the detected specific subject as the target area.

Then, in step S502, similar to step S402 in the fourth embodiment, a focus detection area AFP to be used for focus adjustment is selected, and the focus detection area 224 corresponding to the selected focus detection area AFP is determined as the focus detection area 224 for performing focus detection.

In step S503, the camera control unit 21 determines whether the target area detected in step S501 exists within the focus detection area 224 selected in step S502. If the target area exists within the focus detection area 224, the process proceeds to step S504. On the other hand, if the target area does not exist within the focus detection area 224, the process proceeds to step S513. In steps S513 to S515, similar to steps S403 to S405 in the fourth embodiment, contrast information of the imaging pixel rows 225 set in the focus detection area 224 is detected (step S513), the imaging pixel rows 225 are grouped into groups of a predetermined number of imaging pixel rows (step S514), and the output values of the frequency components included in the output of the imaging pixel rows 225 belonging to the same group are added for each group, thereby detecting contrast information of the frequency components in each group. (step S515).

On the other hand, if a target area is detected in the focus detection area 224 in step S503 (step S503 = Yes), the process proceeds to step S504. In step S504, the camera control unit 21 extracts the output values of the four imaging pixels 221a constituting each pixel group 223 in the target area at a ratio corresponding to the color indicating the specific subject (for example, the skin color of the specific subject). Then, in step S505, the camera control unit 21 detects the output of each pixel group 223 included in the target area based on the output of the imaging pixels 221a extracted in step S504, and detects the output of each pixel group row 226 consisting of multiple pixel groups 223. That is, the camera control unit 21 detects the output value of the pixel group 223 corresponding to the skin color of the specific subject by adding or averaging the output values of the imaging pixels 221a extracted at a ratio corresponding to the color indicating the specific subject in pixel group 223 units. Then, the camera control unit 21 detects the output of the pixel group array 226 by detecting the output values of the multiple pixel groups 223 that make up the pixel group array 226 along the arrangement direction of the pixel groups 223 in the pixel group array 226.

Furthermore, in step S506, the camera control unit 21 detects contrast information of the frequency components contained in the output of the pixel group array 226 based on the output of the pixel group array 226 detected in step S505. For example, the camera control unit 21 applies a predetermined filter process to the output of the pixel group array 226 to extract frequency components corresponding to the subject from the output of the pixel group array 226, and detects contrast information of the extracted frequency components.

In step S507, the camera control unit 21 groups the pixel group arrays 226 corresponding to the target area into groups of a predetermined number of pixel group arrays 226. Then, in step S508, the camera control unit 21 detects contrast information of the frequency components in each group by adding up the output values of the frequency components included in the output of the pixel group arrays 226 that belong to the same group.

In step S509, the camera control unit 21 calculates the contrast amount of the frequency components in each group based on the contrast information of the frequency components in each group detected in step S509 or step S515, for example as shown in the above formula (3).

Then, in steps S510 to S512, the group with the highest contrast is determined as the specific group (step S510) in the same manner as in steps S407 to S409 in the fourth embodiment, and the defocus amount is calculated based on the output of the imaging pixels 221a included in the specific group (step S511). Then, the focus lens 32 is driven based on the calculated defocus amount (step S512).

In this way, in the fifth embodiment, by extracting the output of each imaging pixel 221a at a ratio according to the color indicating the specific subject, it is possible to extract the frequency components resulting from the specific subject, such as a person's face, with higher accuracy. Then, based on the frequency components extracted in this way, a group for which focus detection is to be performed is determined, and focus detection is performed based on the output of the imaging pixels 221a in that group, making it possible to more appropriately detect the focus state of the optical system for the specific subject.

Although the embodiments have been described above, the above-mentioned embodiments have been described to facilitate understanding of the present invention, and are not described to limit the present invention. Therefore, each element disclosed in the above-mentioned embodiments is intended to include all design modifications and equivalents that fall within the technical scope of the present invention.

For example, in the first to third embodiments described above, a configuration having four focus detection pixel rows L1 to L4 for one focus detection area AFP is exemplified, but the number of focus detection pixel rows is not limited to this, and the number of focus detection pixel rows in one focus detection area may be two or three, or may be five or more. Also, in the fourth and fifth embodiments described above, a configuration in which imaging pixels 221 having a pair of photoelectric conversion units 2214, 2215 are arranged in the focus detection areas 224a to 224i is exemplified, but it is also possible to have a configuration in which imaging pixels 221 are arranged over the entire image sensor 220, for example.

In the first to third embodiments described above, the image sensor 22 includes focus detection pixel rows L1 to L4, and the defocus amount is calculated based on the output of the focus detection pixel rows L1 to L4. However, the present invention is not limited to this configuration. For example, a phase difference AF detection module may be provided separately from the image sensor 22, and the defocus amount may be calculated based on the light reception signal received by the phase difference AF detection module. Specifically, the phase difference AF detection module may have a plurality of pairs of line sensors that receive light passing through the optical system, detect contrast information for each line sensor, and determine a pair of line sensors to be used for focus detection from among the plurality of line sensors based on the detected contrast information. When determining a pair of line sensors to be used for focus detection, a configuration may be used in which a calculation unit provided in the phase difference detection module determines the pair of line sensors to be used for focus detection, or the camera control unit 21 may determine the pair of line sensors to be used for focus detection by acquiring the output of each pair of line sensors.

In this way, the "photoreceptor group" may be, for example, focus detection pixel rows L1 to L4, or the pair of line sensors described above. Also, the "focus detection device" may be, for example, the image sensor 22, 220 or a camera 1 equipped with the image sensor, or a phase-difference AF detection module or a camera 1 equipped with the image sensor.

Furthermore, in the first to third embodiments described above, the first focus detection pixel 222a and the second focus detection pixel 222b are arranged in reverse in the X-axis direction in the focus detection pixel rows L1, L3 and the focus detection pixel rows L2, L4, but this configuration is not limited thereto. For example, as shown in FIG. 26, the focus detection pixels 222a, 222b constituting the focus detection pixel rows L1a to L4a may be arranged in the same position in the X-axis direction.

Also, as shown in the focus detection pixel rows L1b and L2b in Fig. 27, each focus detection pixel 222a and 222b may be configured to include a pair of photoelectric conversion units 2222a and 2222b. Specifically, as shown in Fig. 27, in each focus detection pixel 222a and 222b, a pair of photoelectric conversion units 2222a and 2222b is arranged in the X-axis direction, and the light beam emitted from the ranging pupil 351 is received by one of the pair of photoelectric conversion units 2222a and 2222b, and the light beam emitted from the ranging pupil 352 is received by the other photoelectric conversion unit. This allows a pair of image data rows to be output in each focus detection pixel row L1b and L2b. In addition, in FIG. 27, the photoelectric conversion unit that receives the light beam irradiated from the ranging pupil 351 is shown in gray, and the photoelectric conversion unit that receives the light beam irradiated from the ranging pupil 352 is shown in white. In addition, in the example shown in FIG. 27, a configuration having two focus detection pixel rows L1b, L2b is illustrated, but this configuration is not limited to this, and for example, a configuration having only one focus detection pixel row L1b, L2b, or a configuration having three or more rows, may be used.

Furthermore, the arrangement of the first focus detection pixels 222a and the second focus detection pixels 222b that make up each focus detection pixel row may be such that they are at least arranged alternately on the same row, and for example, the focus detection pixel row may be configured to include normal imaging pixels 221, as in the focus detection pixel rows L1c to L4c shown in FIG. 28.

In addition to the above-described embodiments, when the contrast of the subject (or the brightness of the subject) is equal to or greater than a predetermined value, as shown in the first embodiment, the focus detection pixel row corresponding to the subject with the greatest contrast among the focus detection pixel rows L1 to L4, or the focus detection pixel row whose output contains the most high-frequency components, may be determined as the specific focus detection pixel row. On the other hand, when the contrast of the subject (or the brightness of the subject) is less than a predetermined value, as shown in the second embodiment, one or more focus detection pixel rows corresponding to the subject with the contrast equal to or greater than a predetermined value, or one or more focus detection pixel rows whose output contains the amount of high-frequency components equal to or greater than a predetermined value may be determined as the specific focus detection pixel row.

Furthermore, in the above-mentioned fourth embodiment, the contrast amount of the frequency component is calculated for each group by adding up the frequency components corresponding to the output of each imaging pixel row 225 for each group, and the defocus amount is calculated based on the output of the imaging pixel 221a included in the group with the highest contrast amount. However, the present invention is not limited to this configuration, and can be configured as follows. That is, the contrast amount of the frequency component is calculated for each imaging pixel row 225 from the frequency component corresponding to the imaging pixel row 225, and the imaging pixel row 225 with the highest contrast amount is identified for each group from the multiple imaging pixel rows 225 included in the same group. Then, the defocus amount is calculated for each group based on the output of the imaging pixel 221a included in the imaging pixel row 225 with the highest contrast amount, and the focus adjustment of the optical system is performed based on these defocus amounts. Also, the contrast amount of the frequency component may be calculated for each imaging pixel row 225, and the defocus amount may be calculated for each imaging pixel row 225, and the focus adjustment of the optical system may be performed based on the defocus amount of the imaging pixel row 225 with the highest contrast amount.

In the fourth and fifth embodiments described above, the defocus amount is calculated based on the output of the imaging pixel 221a included in the group with the highest contrast amount of the frequency component. However, the present invention is not limited to this configuration. For example, the output of the imaging pixel 221a in each group can be weighted based on the magnitude of the contrast amount of the frequency component in each group, and the defocus amount can be calculated based on the weighted output of the imaging pixel 221a. In this case, the camera control unit 21 can increase the weight of the output of the imaging pixel 221a in the group as the contrast amount of the frequency component in the group increases. The camera control unit 21 can also calculate the defocus amount for each group, and calculate the defocus amount used for focus adjustment by weighting the calculated defocus amount according to the contrast amount of the frequency component in each group.

Furthermore, in the above-mentioned fourth and fifth embodiments, a configuration in which a plurality of imaging pixels 221a are arranged two-dimensionally in each focus detection area 224a to 224i is exemplified, but this configuration is not limited to this, and for example, the focus detection area 224a to 224i may have only a single imaging pixel row 225, or may have two or more imaging pixel rows 225 that are discrete or continuous. Furthermore, in the above-mentioned first to third embodiments, a configuration in which the focus detection pixels 222a and 222b are arranged in the horizontal direction (X direction) in the focus detection pixel rows L1 to L4 is exemplified, but this configuration is not limited to this, and for example, the focus detection pixels 222a and 222b can be arranged in the vertical direction (Y direction). Also, in the above-mentioned fourth and fifth embodiments, a configuration in which a plurality of imaging pixels 221a arranged in the vertical direction (Y direction) is arranged as the imaging pixel row 225 may be used. In this case, the output value of the frequency component in each group can be calculated by adding up the output values of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the vertical direction (Y direction) in each imaging pixel column 225 belonging to the same group.

In addition, in the fourth and fifth embodiments described above, a single-lens reflex digital camera 1a equipped with an image sensor 220 has been described as an example, but the present invention is not limited to this configuration. For example, the digital camera 1 shown in FIG. 1 may be configured to be equipped with an image sensor 220. In this case, too, focus detection can be performed in the same way as in the fourth and fifth embodiments.

1, 1a... Digital camera 2, 2a... Camera body 21, 220... Camera control unit 22... Image sensor 221, 221a... Imaging pixels L1 to L4... Focus detection pixel row 225... Imaging pixel row 226... Pixel group row 222a, 222b... Focus detection pixel 3... Lens barrel 32... Focus lens 36... Focus lens drive motor 37... Lens control unit

Claims (8)

an image sensor having an imaging surface that captures an image formed by an optical system, the image sensor having a plurality of focus detection areas that detect a focus state between the image and the imaging surface, and a plurality of pixel rows arranged in each of the plurality of focus detection areas, the pixel rows including pixels each having a first photoelectric conversion unit that receives light passing through a first area of the optical system and outputs a signal, and a second photoelectric conversion unit that receives light passing through a second area of the optical system and outputs a signal;
a control unit that detects contrast information of outputs from a plurality of pixel rows arranged in one of the plurality of focus detection areas, identifies a portion of pixel rows to be used for focus detection from the plurality of pixel rows arranged in the one focus detection area based on a detection result, and detects a focus state based on a signal output from the identified pixel row;
A focus detection device comprising: 2. The focus detection device according to claim 1,
A focus detection device, wherein the plurality of pixels arranged within one focus detection area are arranged adjacent to each other in a first direction and a second direction perpendicular to the first direction. 3. The focus detection device according to claim 1,
the one focus detection area includes a plurality of groups each having a plurality of the pixel columns,
The control unit detects contrast information of the output of multiple pixel rows for each of the multiple groups, identifies a portion of the multiple groups arranged in the single focus detection area to be used for focus detection based on the detection results, and detects a focus state based on a signal output from the pixel row in the identified group. 4. The focus detection device according to claim 3,
the plurality of groups includes a first group and a second group;
The control unit detects a contrast amount of a predetermined frequency component in the first group based on outputs of the plurality of pixel columns included in the first group, detects a contrast amount of the predetermined frequency component in the second group based on outputs of the plurality of pixel columns included in the second group, and if the contrast amount in the first group is greater than the contrast amount in the second group, identifies the plurality of pixel columns included in the first group as pixel columns to be used for focus detection. 3. The focus detection device according to claim 1,
A focus detection device that identifies a portion of pixel rows whose contrast is equal to or greater than a predetermined value as focus detection pixel rows based on outputs from the plurality of pixel rows arranged within the one focus detection area. 3. The focus detection device according to claim 1,
A focus detection device that identifies, based on the outputs of the plurality of pixel rows arranged within one of the focus detection areas, some pixel rows in which the amount of high-frequency components contained in the output is equal to or greater than a predetermined value as pixel rows to be used for focus detection. 3. The focus detection device according to claim 1,
the plurality of pixel rows arranged within the one focus detection area include a first pixel row and a second pixel row,
The control unit detects the contrast amount of a predetermined frequency component of the first pixel row and the second pixel row, and when the contrast amount of the output of the first pixel row is greater than the contrast amount of the output of the second pixel row, identifies the first pixel row as a pixel row to be used for focus detection, and detects a focus state based on the signal output from the first pixel row. an imaging section having a plurality of focus detection areas on an imaging surface that captures an image formed by an optical system and that detects a focus state between the image and the imaging surface, and arranging a plurality of pixel rows in each of the plurality of focus detection areas, the pixel rows including pixels each having a first photoelectric conversion unit that receives light passing through a first area of the optical system and outputs a signal, and a second photoelectric conversion unit that receives light passing through a second area of the optical system and outputs a signal;
a calculation unit that detects contrast information of outputs from a plurality of pixel rows arranged in one of the plurality of focus detection areas, identifies a portion of pixel rows to be used for focus detection from the plurality of pixel rows arranged in the one focus detection area based on a detection result, and detects a focus state based on a signal output from the identified pixel row;
An imaging element comprising:

Images