JP2013197796A - Image sensor and imaging device - Google Patents

Abstract

In order to acquire an image that generates a plurality of parallaxes, it has been necessary to prepare a complicated photographing optical system corresponding to the number of images.
An imaging element that corresponds to at least one of a plurality of two-dimensionally arranged photoelectric conversion elements that photoelectrically convert an incident subject light beam into an electric signal, and a plurality of photoelectric conversion elements. And an open / close section that can switch between a closed state in which at least part of the cross-sectional area of the subject light beam is shielded and an open state in which part is transmitted by switching optical characteristics.
[Selection] Figure 1

Description

  The present invention relates to an imaging element and an imaging apparatus.

A stereo imaging device that captures a stereo image composed of a right-eye image and a left-eye image using two imaging optical systems is known. Such a stereo imaging device causes parallax to occur in two images obtained by imaging the same subject by arranging two imaging optical systems at regular intervals.
[Prior art documents]
[Patent Literature]
[Patent Document 1] JP-A-8-47001

  However, in order to acquire an image that generates a plurality of parallaxes, it has been necessary to prepare a complicated photographing optical system corresponding to the number of images.

  In the first aspect of the present invention, the imaging device is at least one of a plurality of two-dimensionally arranged photoelectric conversion devices that photoelectrically convert an incident subject light beam into an electric signal, and a plurality of photoelectric conversion devices. By switching the optical characteristics, it is possible to switch between a closed state that shields at least part of the cross-sectional area of the subject luminous flux and an open state that transmits part of it. And an opening / closing part.

  In the second aspect of the present invention, the image pickup apparatus is an image pickup device that reads out the image pickup device and an image pickup signal picked up by the image pickup device and generates image data based on an open / close state of an open / close unit at the time of image pickup. A processing unit.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is a figure explaining the structure of the digital camera 10 which concerns on embodiment of this invention. 1 is a schematic diagram illustrating a cross section of an image sensor 100. FIG. It is the schematic which shows the outline which looked at the image pick-up element 100 of FIG. 2 from the to-be-photographed object side. 3 is a schematic cross-sectional view of a liquid crystal shutter 300. FIG. It is explanatory drawing which shows the relationship between the opening / closing state of the opening / closing area | regions 361 and 362 in one pixel, and the opening 104 formed by it. 2 is a schematic diagram illustrating a state in which a part of the image sensor 100 is enlarged. FIG. FIG. 7 is a schematic diagram illustrating a relationship between parallax pixels and a subject corresponding to FIG. 6. The example of the repeating patterns 110 and 111 is shown. Examples of other repeated patterns 112 and 113 are shown. Examples of other repetitive patterns 114, 115, 116, 117 are shown. 5 is a schematic cross-sectional view showing a method for manufacturing the image sensor 100. FIG. 5 is a schematic cross-sectional view showing a method for manufacturing the image sensor 100. FIG. 5 is a schematic cross-sectional view showing a method for manufacturing the image sensor 100. FIG. 5 is a schematic cross-sectional view showing a method for manufacturing the image sensor 100. FIG. It is a schematic diagram of other image sensors 119. It is a schematic sectional drawing which shows the other image pick-up element 118. FIG. The relationship between the aperture mask 123 of the image sensor 118 and the individual electrode 330 is shown. 3 is a diagram illustrating a Bayer array of the color filter 102. FIG. It is a figure explaining other arrangement | sequences of the color filter.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  The digital camera according to the present embodiment, which is an embodiment of the imaging device, generates image data by imaging subject light flux from a plurality of viewpoints with a single imaging device while maintaining high resolution. Each image having a different viewpoint is called a parallax image.

  FIG. 1 is a diagram illustrating the configuration of a digital camera 10 according to an embodiment of the present invention. The digital camera 10 includes a photographic lens 20 as a photographic optical system, and guides a subject light beam incident along the optical axis 21 to the image sensor 100. The photographing lens 20 may be an interchangeable lens that can be attached to and detached from the digital camera 10. The digital camera 10 includes an image sensor 100, a drive unit 230, a control unit 201, an A / D conversion circuit 202, a memory 203, an image processing unit 205, a memory card IF 207, an operation unit 208, a display unit 209, an LCD drive circuit 210, and an AF. A sensor 211 is provided.

  As shown in the figure, the direction parallel to the optical axis 21 toward the image sensor 100 is defined as the z-axis plus direction, the direction toward the front of the drawing in the plane orthogonal to the z-axis is the x-axis plus direction, and the upward direction on the drawing is y. The axis is defined as the plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

  The taking lens 20 is composed of a plurality of optical lens groups, and forms an image of a subject light flux from the scene in the vicinity of its focal plane. In FIG. 1, for convenience of explanation, the photographic lens 20 is represented by a single virtual lens arranged in the vicinity of the pupil.

  The image sensor 100 is disposed near the focal plane of the photographic lens 20. The image sensor 100 is an image sensor such as a CCD or CMOS sensor in which a plurality of photoelectric conversion elements are two-dimensionally arranged. Hereinafter, a CMOS example will be described, but the present invention can also be applied to a CCD.

  The driving unit 230 includes a reading driving unit 232 that drives to read an image signal from the photoelectric conversion element of the image sensor 100 and an opening / closing driving unit 234 described later. The image sensor 100 is timing-controlled by the reading drive unit 232, converts the subject image formed on the light receiving surface into an image signal, and outputs the image signal to the A / D conversion circuit 202.

  The A / D conversion circuit 202 converts the image signal output from the image sensor 100 into a digital image signal and outputs the digital image signal to the memory 203. The image processing unit 205 performs various image processing using the memory 203 as a work space, and generates image data.

  In addition, the image processing unit 205 generates parallax image data, 2D image data as non-parallax image data, or the like from an input image signal in accordance with the pixel arrangement of the image sensor 100 or is selected. It also functions to adjust image data according to the image format.

  The generated image data is converted into a display signal by the LCD drive circuit 210 and displayed on the display unit 209. The data is recorded on the memory card 220 attached to the memory card IF 207.

  The AF sensor 211 is a phase difference sensor in which a plurality of distance measuring points are set for the subject space, and detects the defocus amount of the subject image at each distance measuring point. A series of shooting sequences is started when the operation unit 208 receives a user operation and outputs an operation signal to the control unit 201. Various operations such as AF and AE accompanying the imaging sequence are executed under the control of the control unit 201. For example, the control unit 201 analyzes the detection signal of the AF sensor 211 and executes focus control for moving a focus lens that constitutes a part of the photographing lens 20. The following image sensor 100 may be used instead of the AF sensor 211, and in that case, the separate AF sensor 211 may not be provided.

  FIG. 2 is a schematic diagram illustrating a cross section of the image sensor 100. FIG. 3 is a schematic diagram showing an outline of the image sensor 100 of FIG. 2 as viewed from the subject side.

  As shown in FIG. 2, the image sensor 100 is configured by arranging a microlens 101, a color filter 102, a liquid crystal shutter 300, a wiring layer 105, and a photoelectric conversion element 108 in order from the subject side. The photoelectric conversion element 108 is configured by a photodiode that converts incident light into an electrical signal. A plurality of photoelectric conversion elements 108 are two-dimensionally arranged on the surface of the substrate 109. As shown in FIG. 3, the substrate 109 is provided with a floating diffusion 142 for accumulating electric charges from the photoelectric conversion element 108, and a circuit such as a selection transistor and an amplification transistor, adjacent to the photoelectric conversion element 108. The circuit area 140 is arranged.

  A wiring layer 105 is provided over the layer provided with the photoelectric conversion element 108 with an insulating layer 144 interposed therebetween. An image signal converted by the photoelectric conversion element 108, a control signal for controlling the photoelectric conversion element 108, and the like are transmitted and received through the wiring 106 provided in the wiring layer 105.

  On the insulating layer 144, a liquid crystal shutter 300 that defines a subject light beam incident on the photoelectric conversion element 108 is disposed. The liquid crystal shutter 300 includes a pair of substrates 304 and 306, a liquid crystal 320 sealed between the pair of substrates 304 and 306, and a seal portion 302 that holds the liquid crystal 320 between the substrates 304 and 306. . The liquid crystal shutter 300 further includes a set of an individual electrode 310 and a driving element 350 provided corresponding to the photoelectric conversion element 108, and a common electrode 316 used in common for the plurality of photoelectric conversion elements 108.

  In the example shown in FIGS. 2 and 3, a set of the individual electrode 310 and the drive element 350 and a set of the individual electrode 312 and the drive element 352 are provided for one photoelectric conversion element 108. A set of the individual electrode 310 and the drive element 350 and a set of the individual electrode 312 and the drive element 352 are arranged in the x direction with respect to the corresponding photoelectric conversion element 108. That is, the liquid crystal shutter 300 is divided into two segments corresponding to one pixel.

  The color filter 102 is provided on the liquid crystal shutter 300. The color filter 102 is colored so as to transmit a specific wavelength band with respect to each photoelectric conversion element 108, and is provided in one-to-one correspondence with each photoelectric conversion element 108. In order to output a color image, it is only necessary to arrange at least two types of color filters that are different from each other. However, in order to obtain a higher quality color image, it is preferable to arrange three or more types of color filters. For example, a red filter that transmits a red wavelength band, a green filter that transmits a green wavelength band, and a blue filter that transmits a blue wavelength band may be arranged in a lattice pattern.

  The microlens 101 is provided on the color filter 102. The microlens 101 is a condensing lens for guiding more incident subject light flux to the photoelectric conversion element 108. The microlenses 101 are provided in a one-to-one correspondence with the photoelectric conversion elements 108. In consideration of the relative positional relationship between the pupil center of the taking lens 20 and the photoelectric conversion element 108, the optical axis of the microlens 101 is shifted so that more subject light flux is guided to the photoelectric conversion element 108. It is preferable.

  As described above, one unit of the liquid crystal shutter 300, the color filter 102, and the microlens 101 provided in a one-to-one correspondence with each photoelectric conversion element 108 is referred to as a pixel. For example, when the effective pixel area of the image sensor 100 is about 24 mm × 16 mm, the number of pixels reaches about 12 million.

  Note that in the case of an image sensor with good light collection efficiency and photoelectric conversion efficiency, the microlens 101 may not be provided. In the case of a back-illuminated image sensor, the wiring layer 105 is provided on the side opposite to the photoelectric conversion element 108.

  For the sake of explanation, the open / close region 361 on the individual electrode 310 side is open and the open / close region 362 on the individual electrode 312 side is closed on the −x side of the two pixels in FIGS. 2 and 3. On the + x side, the open / close region 363 on the individual electrode 310 side is closed, and the open / close region 364 on the individual electrode 312 side is open.

  As shown in FIG. 3, the individual electrode 310 covers a part of the region where the photoelectric conversion element 108 is arranged in a plan view, for example, approximately half of the −x side in the example shown in FIG. 3. Similarly, the individual electrode 312 covers a region different from the region covered with the individual electrode 310 in the region where the photoelectric conversion element 108 is disposed in plan view. For example, in the example shown in FIG. 3, the individual electrode 312 covers approximately half of the + x side corresponding to the fact that the individual electrode 310 covers approximately half of the −x side.

  Therefore, in the pixel on the −x side in FIG. 3, approximately half of the cross-sectional area of the subject light beam toward the photoelectric conversion element 108 is blocked by the open / close region 362 in the closed state, and optically on the −x side. An effective opening 104l is formed. Similarly, in the pixel on the + x side, approximately half of the cross-sectional area of the subject luminous flux toward the photoelectric conversion element 108 is blocked by the open / close region 363 in the closed state, and an optically effective opening on the + x side. 104l is formed.

  Note that the subject luminous flux is also limited by the opening 107 formed by the wiring 106 of the wiring layer 105. However, in the present embodiment, the opening 107 is sufficiently larger than the photoelectric conversion element 108 and the individual electrodes 310 and 312 on the pupil of the optical system, and the opening 107 does not contribute to the formation of an effective opening.

  FIG. 4 is a schematic sectional view of the liquid crystal shutter 300. In FIG. 4, one set of the individual electrode 310 and the drive element 350 is shown. The liquid crystal shutter 300 includes polarizing films 365 and 366, a connection portion 360, and alignment films 321 and 322 in addition to the individual electrode 310, the driving element 350, the pair of substrates 304 and 306, and the seal portion 302.

  The substrate 306 is formed of a material that is transparent to visible light and is optically isotropic, for example, glass. A polarizing film 366 is disposed on the surface of the substrate 306 far from the photoelectric conversion element 108. The polarizing film 366 has an optical axis and transmits light having a polarization direction parallel to the optical axis among incident light. In the example shown in FIG. 4, the optical axis of the polarizing film 366 is arranged in the x direction, and transmits polarized light in the x direction and blocks polarized light in the y direction.

  A common electrode 316 is disposed on the surface of the substrate 306 opposite to the polarizing film 366. The common electrode 316 is formed of a material that is transparent to visible light and has conductivity, such as an ITO film. One common electrode 316 is provided for each of the plurality of individual electrodes 310. In the example illustrated in FIG. 4, one common electrode 316 is disposed on the entire surface of the substrate 306, that is, the entire image sensor 100.

  An alignment film 321 is disposed on the surface of the common electrode 316 opposite to the substrate 306. The alignment film 321 is a film in which polymers are aligned, and is formed, for example, by rubbing polyimide. The alignment direction of the alignment film 321 is arranged in parallel with the optical axis of the polarizing film 366.

  The other substrate 304 is also formed of the same material as the substrate 306. A polarizing film 365 is disposed on the surface of the substrate 304 that is closer to the photoelectric conversion element 108. Similar to the polarizing film 366, the polarizing film 365 also transmits light having a polarization direction parallel to the optical axis of incident light. In the example shown in FIG. 4, the polarizing film 366 transmits polarized light in the y direction and blocks polarized light in the x direction. Thus, the pair of polarizing films 365 and 366 are arranged in so-called crossed Nicols.

  The individual electrode 310, the drive element 350, and the connection portion 360 are disposed on the surface of the substrate 304 opposite to the polarizing film 365. As illustrated in FIG. 3, the individual electrode 310 covers a part of a region where the photoelectric conversion element 108 is disposed in a plan view, for example, approximately half of the −x side. The individual electrode 310 is made of the same material as the common electrode 316. The drive element 350 is electrically connected to the open / close drive unit 234, and based on a drive signal from the open / close drive unit 234, a voltage is applied between the individual electrode 310 and the common electrode 316, or the voltage is set to zero. To do. An example of the driving element 350 is a switch using a transistor. The connection unit 360 electrically connects the driving element 350 and the individual electrode 310.

  An alignment film 322 is disposed on the surface of the substrate 304 on which the individual electrode 310 is provided. The alignment film 322 may be formed using the same material and method as the alignment film 321. The alignment direction of the alignment film 321 is arranged in parallel with the optical axis of the polarizing film 365.

  A region sandwiched between the pair of alignment films 321 and 322 is filled with liquid crystal 320. The liquid crystal 320 is aligned along the alignment of the alignment films 321 and 322. Here, since the pair of alignment films 321 and 322 are orthogonal to each other, the polymers in the liquid crystal 320 are twisted. Therefore, the liquid crystal 320 operates in a normally white TN mode. That is, the twisted arrangement is maintained in the state where no voltage is applied between the individual electrode 310 and the common electrode 316, and the polymer is arranged perpendicular to the substrate 304 or the like in the state where the voltage is applied. As a result, the optical characteristics of the liquid crystal 320 are switched.

  According to the liquid crystal shutter 300 having the above configuration, the open / close region 361 is in an open state in which light is transmitted when the drive element 350 does not apply a voltage between the individual electrode 310 and the common electrode 316. Therefore, as shown in FIG. 4, light A from the region W in the drawing corresponding to the open / close region 361 enters the photoelectric conversion element 108. On the other hand, in a state where a voltage is applied between the individual electrode 310 and the common electrode 316 by the drive element 350, the open / close region 361 is in a closed state that blocks light. Therefore, the light A from the region W in the drawing corresponding to the open / close region 361 does not enter the photoelectric conversion element 108.

  FIG. 5 is an explanatory diagram showing the relationship between the open / closed states of the open / close regions 361 and 362 in one pixel and the opening 104 formed thereby. When both the open / close regions 361 and 362 are open, a large opening 104c is formed at the center. The opening 104c is defined by the opening 107 of the wiring layer 105 other than the open / close regions 361 and 362, and the center of gravity thereof is located at the center of the subject luminous flux on the pupil of the optical system. Therefore, no parallax occurs when the subject is imaged using the opening 104c. Pixels that do not cause parallax may be referred to as non-parallax pixels.

  When the open / close region 361 is closed and the open / close region 362 is open, an opening 104r whose area is substantially half of the opening 104c and shifted to the right in the drawing is formed. That is, the center of gravity of the opening 104r is displaced to the right with respect to the center of the subject light beam on the pupil of the optical system, and transmits the right half region of the subject light beam. Therefore, parallax occurs when the subject is imaged using the opening 104r. Pixels that cause parallax may be referred to as parallax pixels.

  When the opening / closing region 361 is in the open state and the opening / closing region 362 is in the closed state, an opening 104l whose area is substantially half of the opening 104c and shifted to the left in the drawing is formed. That is, the center of gravity of the opening 104l is displaced to the left with respect to the center of the subject luminous flux on the pupil of the optical system, and transmits the left half region of the subject luminous flux.

  When both the open / close regions 361 and 362 are in the closed state, no opening is formed, and the subject luminous flux does not enter the photoelectric conversion element 108.

  FIG. 6 is a schematic diagram illustrating a state in which a part of the image sensor 100 is enlarged. The relationship between the effective opening 104 formed by opening and closing the liquid crystal shutter 300 and the generated parallax will be described. Here, in order to simplify the explanation, the color arrangement of the color filter 102 is not considered until it is mentioned later. In the following description that does not refer to the color arrangement of the color filter 102, it can be considered that the image sensor is a collection of only parallax pixels having the color filter 102 of the same color. Therefore, the repetitive pattern described below may be considered as an adjacent pixel in the color filter 102 of the same color.

  The example shown in FIG. 6 shows a state in which two types of openings 104l and 104r are formed by the liquid crystal shutter 300 in correspondence with FIGS. The openings 104l and 104r are both rectangles that are long in the y direction. The opening 104l is shifted to the −x side, and the opening 104r is shifted to the + x side. The opening 104l and the opening 104r have the same shape and size, and the shift directions are opposite, but the absolute values of the shift amounts are the same.

  In the entire imaging device 100, a photoelectric conversion element group in which two parallax pixels having openings 104l and 104r are arranged in the x direction is two-dimensionally and periodically arranged. That is, in the image sensor 100, the repeating pattern 110 including the photoelectric conversion element group is periodically spread. In the example shown in FIG. 6, the pixels adjacent to the opening 104l in the y direction are also arranged so as to be the opening 104l, but may be arranged so that other openings are adjacent. Here, the x direction is, for example, the horizontal direction when the digital camera 10 is imaged in the horizontal position. In this case, a table in which the two-dimensional position on the image sensor 100 and the open / close state of the liquid crystal shutter 300 are associated with each other is recorded in advance in the memory 203 or the like, and the open / close drive unit 234 refers to the table. , Openings 104l and 104r are formed in the corresponding pixels, respectively.

  FIG. 7 is a schematic diagram for explaining the relationship between the parallax pixels and the subject, corresponding to FIG. In particular, FIG. 7A shows a photoelectric conversion element group of a repetitive pattern 110t arranged at the center orthogonal to the optical axis 21 in the image sensor 100, and FIG. 7B shows a repetitive pattern arranged in the peripheral portion. A 110 u photoelectric conversion element group is schematically shown.

  In addition, the subject 30 in FIGS. 7A and 7B is in a focus position with respect to the photographing lens 20. FIG. 7C schematically shows the relationship when the subject 30 existing at the out-of-focus position with respect to the photographing lens 20 is captured corresponding to FIG.

  First, the relationship between the parallax pixels and the subject when the photographing lens 20 captures the subject 30 that is in focus will be described. The subject light flux passes through the pupil of the photographic lens 20 and is guided to the image sensor 100. Two partial areas Pl and Pr are defined for the entire cross-sectional area through which the subject light flux passes. For example, in the + x-side parallax pixels of the photoelectric conversion element group constituting the repetitive patterns 110t and 110u, as can be seen from the enlarged view, only the subject light beam emitted from the partial region Pr reaches the photoelectric conversion element 108. In addition, an opening 104 r is formed by the liquid crystal shutter 300. Similarly, an opening 104l is formed corresponding to the partial region Pl toward the -x side parallax pixel.

  In other words, for example, the position of the opening 104r is determined by the inclination of the principal ray Rr of the subject light beam emitted from the partial region Pr, which is defined by the relative positional relationship between the partial region Pr and the + x side parallax pixel. You can say that. When the photoelectric conversion element 108 receives the subject light flux from the subject 30 present at the in-focus position via the opening 104r, the subject light flux forms an image on the photoelectric conversion element 108 as shown by the dotted line. To do. Similarly, it can be said that the position of the opening 104l is determined by the inclination of the principal ray Rl toward the parallax pixel on the -x side.

  As shown in FIG. 7A, the light beam emitted from the minute region Ot on the subject 30 that intersects the optical axis 21 among the subject 30 existing at the in-focus position passes through the pupil of the photographing lens 20. Then, each pixel of the photoelectric conversion element group constituting the repetitive pattern 110t is reached. That is, each pixel of the photoelectric conversion element group constituting the repetitive pattern 110t receives a light beam emitted from one minute region Ot through two partial regions Pr and Pl. Although the minute region Ot has an extent corresponding to the positional deviation of each pixel of the photoelectric conversion element group constituting the repetitive pattern 110t, it can be approximated to substantially the same object point.

  Similarly, as shown in FIG. 7B, the luminous flux emitted from the minute region Ou on the subject 30 that is separated from the optical axis 21 among the subject 30 that exists at the in-focus position passes through the pupil of the photographing lens 20. It passes through and reaches each pixel of the photoelectric conversion element group constituting the repetitive pattern 110u. That is, each pixel of the photoelectric conversion element group constituting the repetitive pattern 110u receives a light beam emitted from one minute region Ou through two partial regions Pr and Pl. Similarly to the micro area Ot, the micro area Ou has an extent corresponding to the positional deviation of each pixel of the photoelectric conversion element group constituting the repetitive pattern 110u, but substantially the same object point. Can be approximated.

  In other words, as long as the subject 30 exists at the in-focus position, the minute area captured by the photoelectric conversion element group differs according to the position of the repetitive pattern 110 on the image sensor 100, and each pixel constituting the photoelectric conversion element group Captures the same minute region through different partial regions. In each repetitive pattern 110, corresponding pixels receive the subject luminous flux from the same partial area. That is, in the figure, for example, each of the + x side parallax pixels of the repetitive patterns 110t and 110u receives the subject luminous flux from the same partial region Pr.

  In the repetitive pattern 110t arranged at the center orthogonal to the optical axis 21, the position of the opening 104r where the + x side parallax pixels receive the subject light beam from the partial region Pr and + x in the repetitive pattern 110u arranged in the peripheral portion The position of the opening 104r where the parallax pixel on the side receives the subject light beam from the partial region Pr is strictly different. However, from a functional point of view, these can be treated as the same type of aperture in terms of an aperture for receiving the subject light flux from the partial region Pr.

  Next, the relationship between the parallax pixels and the subject when the photographing lens 20 captures the subject 30 existing in the out-of-focus state will be described. Also in this case, the subject luminous flux from the subject 30 existing at the out-of-focus position passes through the two partial regions Pr and Pl of the pupil of the photographing lens 20 and reaches the image sensor 100. However, the subject light flux from the subject 30 existing at the out-of-focus position forms an image at another position, not on the photoelectric conversion element 108. For example, as illustrated in FIG. 7C, when the subject 30 exists at a position farther from the imaging element 100 than the subject 30, the subject luminous flux forms an image on the subject 30 side with respect to the photoelectric conversion element 108. Conversely, if the subject is present at a position closer to the image sensor 100 than the subject 30 in FIGS. 7A and 7B, the subject luminous flux forms an image on the opposite side of the subject 30 from the photoelectric conversion element 108. To do.

  Therefore, the subject luminous flux radiated from the minute region Ot ′ among the subjects 30 existing at the out-of-focus position depends on which of the two partial regions Pr and Pl corresponds to the corresponding pixels in the different sets of repetitive patterns 110. To reach. For example, as shown in the enlarged view of FIG. 7C, the subject luminous flux that has passed through the partial region P1 is incident on the photoelectric conversion element 108 having the opening 104l included in the repetitive pattern 110t ′ as the principal ray Rl ′. . Even if the subject light beam is emitted from the minute region Ot ′, the subject light beam that has passed through another partial region does not enter the photoelectric conversion element 108 included in the repetitive pattern 110t ′, and the repetitive pattern in the other repetitive pattern. The light enters the photoelectric conversion element 108 having a corresponding opening. In other words, the subject luminous flux that reaches each photoelectric conversion element 108 constituting the repetitive pattern 110 t ′ is a subject luminous flux emitted from a minute area different from the subject 30. That is, a subject luminous flux having a principal ray R1 ′ enters the photoelectric conversion element 108 corresponding to the −x side opening 104l, and the principal ray is Rr + to the photoelectric conversion element 108 corresponding to the + x side opening 104r. A subject luminous flux is incident, and this subject luminous flux is a subject luminous flux emitted from a minute region different from Ot ′ of the subject 30. Such a relationship is the same in the repeated pattern 110u arranged in the peripheral portion in FIG.

  Then, when viewed as a whole of the imaging device 100, for example, the -x side parallax image obtained by capturing the subject 30 with the photoelectric conversion element 108 corresponding to the opening 104l and the subject 30 with the photoelectric conversion element 108 corresponding to the opening 104r. The captured + x side parallax images are not shifted from each other if they are images of the subject existing at the in-focus position, and are shifted if they are images of the subject existing at the out-of-focus position. The direction and amount of the shift are determined by how much the subject existing at the out-of-focus position is deviated from the in-focus position and by the distance between the partial area Pr and the partial area Pl. That is, the -x side parallax image and the + x side parallax image form a parallax image. Therefore, when the outputs of the pixels corresponding to each other are collected in each of the repetitive patterns 110 configured as described above, a plurality of parallax images are obtained.

  Here, the depth information of the subject on the image can be obtained by the parallax on a pair of different parallax images. In this case, it is preferable that the conditions other than the parallax are the same as much as possible, and a pair of parallax images from which a large parallax can be obtained. Therefore, it is preferable that in the repetitive pattern, pixels having the same opening size, the same absolute value of the shift amount, and different shift directions are paired. A parallax pair corresponding to such a pixel pair may be referred to as a parallax pair.

  FIG. 8 shows an example of the repeating patterns 110 and 111. In the repetitive pattern 110, the liquid crystal shutter 300 forms openings 104l corresponding to all the pixels. In other words, the pixels having the openings 104l are repeatedly arranged two-dimensionally. On the other hand, in the repeated pattern 111, the liquid crystal shutter 300 forms openings 104r corresponding to all the pixels.

  The opening / closing drive unit 234 drives the liquid crystal shutter 300 to image one of the repetitive patterns 110 and 111, and then the open / close drive unit 234 drives the liquid crystal shutter 300 to repeat the patterns 110 and 111. A pair of parallax images can be obtained using one image sensor 100 by imaging the subject in a state where the other of the two is formed. The image processing unit 205 generates a parallax image based on the open / closed state of the liquid crystal shutter 300 during imaging. In this example, the image processing unit 205 generates all the parallax images by collecting the image signals from all the pixels of the image sensor 100, so that a parallax image with high spatial resolution can be obtained.

  FIG. 9 shows examples of other repeating patterns 112 and 113. The repetitive pattern 112 is formed of four pixels, and the upper left and lower right have an opening 104r, and the upper right and lower left have an opening 104l. The repeated pattern 113 is also formed of 4 pixels, and the upper left and lower right have an opening 104l, and the upper right and lower left have an opening 104r. As in the case of FIG. 8, by forming these repeated patterns 112 and 113 by the opening / closing drive unit 234 and capturing an image, a pair of captured images can be obtained using one image sensor 100.

  In this case, the image processing unit 205 generates one parallax image by collecting the pixels of the opening 104r of the repetitive pattern 112 and the pixels of the opening 104r of the repetitive pattern 113, and generates the one parallax image of the repetitive pattern 112 and the repetitive pattern. The other parallax image is generated by gathering the pixels of the opening 104l of 113. Thereby, a parallax image with high spatial resolution can be obtained. Also, since each of the two types of apertures is used at any time of imaging, the blurring of the subject due to the time difference of imaging can be distributed to each of the parallax images so as to be inconspicuous. In addition, even if there is a problem with imaging in one of the two imaging operations, a pair of parallax images can be generated by complementing the pixels of the respective openings obtained by the other imaging.

  FIG. 10 shows examples of other repeating patterns 114, 115, 116 and 117. Each of the repeated patterns 114, 115, 116, and 117 is formed by four pixels. In the repetitive pattern 114, the upper left and lower right pixels have an opening 104r, and the upper right and lower left pixels have an opening 104c. The repetitive pattern 115 corresponds to the repetitive pattern 115 which is reversed left and right. The repetitive pattern 116 corresponds to the repetitive pattern 115 that is vertically inverted. The repetitive pattern 117 corresponds to the repetitive pattern 114 that is vertically inverted.

  By forming these repetitive patterns 114, 115, 116, and 117 by the opening / closing drive unit 234 and capturing an image, a pair of captured images and a parallax-free image can be obtained using one image sensor 100. A parallax image with high spatial resolution can be obtained by, for example, generating one parallax image by gathering the pixels of the opening 104r of the repeating pattern 114 and the pixels of the opening 104r of the repeating pattern 117. Similarly, a parallax-free image with high spatial resolution can be obtained. Here, two values can be obtained for one pixel with respect to the opening 104c by four times of imaging, but they may be used on average, or one of them may be used.

  11 to 14 are schematic cross-sectional views illustrating a method for manufacturing the image sensor 100. First, as shown in FIG. 11, various configurations are formed on the pair of substrates 304 and 306 of the liquid crystal shutter 300.

  One surface of the substrate 304 is provided with the individual electrode 310, the drive element 350, and the connection portion 360 by repeating a known film formation process, exposure process, and etching process. A material for the alignment film 322 is applied thereon, and the polymer is aligned by rubbing to form the alignment film 322. Instead of rubbing, the alignment film 322 may be formed by irradiating a UV-polarized polymer with UV-polarized polymer. Further, a seal portion 302 is further formed on the alignment film 322. The seal portion 302 is formed in a frame shape on the alignment film 322 of the substrate 304 by screen printing or the like. FIG. 11 shows only a part of it.

  Further, a common electrode 316 is provided on one surface of the substrate 306 by a known film formation process or the like. An alignment film 321 is formed on the common electrode 316 in the same manner as the alignment film 322.

  As shown in FIG. 12, a pair of substrates 304 and 306 are bonded together. Further, the liquid crystal 320 is sealed between the facing alignment films 321 and 322. Polarizing films 365 and 366 are attached to the outer surfaces of the substrates 304 and 306, and the liquid crystal shutter 300 is formed.

  As shown in FIG. 13, the color filter 102 and the microlens 101 are arranged on the polarizing film 366. In this case, the micro lens 101 may be provided on the color filter 102, and the color filter 102 including the micro lens 101 and the liquid crystal shutter 300 may be aligned and pasted.

  As shown in FIG. 14, a photoelectric conversion element 108 is provided on a substrate 109, and a wiring layer 105 is provided thereon via an insulating layer 144. Since a known method may be used to form these, description thereof is omitted. The upper surface of the insulating layer 144 is planarized. The liquid crystal shutter 300 having the microlens 100 and the color filter 102 is aligned and attached to the top surface of the planarized insulating layer 144, whereby the image sensor 100 is formed.

  In the example shown in FIGS. 1 to 14, the individual electrodes 310 and 312 are divided into two segments for each pixel, but the number of segments is not limited. For example, one individual electrode 310 may be provided for each pixel. In this case, the pixel in which the opening 104c is formed in the open state, the pixel in which the opening 104l is formed in the closed state, and the pixel in which the opening 104c is formed in the open state and the opening 104r is formed in the closed state are repeated as a pattern 112. You may arrange. In addition, individual electrodes divided into three or more segments may be provided for each pixel. You may mix the pixel which provided the individual electrode of two segments, and the pixel which provided the individual electrode of one or three or more segments.

  FIG. 15 is a schematic diagram of another image sensor 119. FIG. 15 is a schematic view of one pixel viewed from the subject side corresponding to FIG.

In the example illustrated in FIG. 15, the liquid crystal shutter 300 includes individual electrodes 370, 374, 378, and 382 that are divided into four segments in the ± x direction and the ± y direction with respect to one photoelectric conversion element 108, and separately. Drive elements 372, 376, 380, and 384 that can be driven are included. As a result, 2 ⁴ = 16 openings can be formed. As an example, FIG. 15 shows that the open / close region corresponding to the + y side individual electrodes 370 and 374 is in the closed state, and the open / close region corresponding to the −y side individual electrodes 378 and 382 is in the open state. Shows a state in which the opening 104u is formed. Instead of segmenting the individual electrodes in the xy direction, the individual electrodes may be segmented in a strip shape in one direction.

  FIG. 16 is a schematic cross-sectional view showing another image sensor 118. FIG. 17 shows the relationship between the aperture mask 123 of the image sensor 118 and the individual electrode 330.

  The opening mask 123 is disposed on the upper surface of the insulating layer 144 and is formed of, for example, aluminum of the same material as the wiring 106. The aperture mask 123 is provided with a displaced aperture 124. The opening 124 corresponds to a region where the subject luminous flux is displaced on the pupil. In the example shown in FIG. 17, the opening 124 is displaced to the −x side. In the individual electrode 330, an opening 324 is provided in the electrode portion 323. The size of the opening 324 is smaller than that of the opening 124. In the example shown in FIG. 17, the opening 324 has a narrower slit shape than the opening 124.

  Therefore, in the example shown in FIG. 17, a partial region of the subject light beam defined by the opening 124 enters the photoelectric conversion element 108 in the open state. In the closed state, the subject light flux defined by the synthetic aperture 125 where the aperture 124 and the aperture 324 overlap is incident on the photoelectric conversion element 108. As a result, two types of parallax images that are transmitted through different portions in the cross-sectional area of the subject luminous flux can be obtained.

  In the example shown in FIG. 16, the opening 124 of the opening mask 123, the individual electrode 330, and the driving element 350 of pixels adjacent to each other are arranged symmetrically. The opening 324 may be a through hole or may be covered with a film that is transparent to visible light.

  FIG. 18 is a diagram for explaining the Bayer array of the color filter 102. As shown in the figure, the Bayer array is an array in which the green filter is assigned to the upper left and lower right pixels, the red filter is assigned to the lower left pixel, and the blue filter is assigned to the upper right pixel. Here, the upper left pixel to which the green filter is assigned is the Gb pixel, and the lower right pixel to which the green filter is assigned is the Gr pixel. In addition, a pixel to which a red filter is assigned is an R pixel, and a pixel to which blue is assigned is a B pixel. A horizontal direction in which Gb pixels and B pixels are arranged is defined as Gb row, and a horizontal direction in which R pixels and Gr pixels are arranged is defined as Gr row. A vertical direction in which Gb pixels and R pixels are arranged is a Gb column, and a vertical direction in which B pixels and Gr pixels are arranged is a Gr column. For example, the Bayer array shown in FIG. 18 may be assigned to four pixels such as the repeating pattern 110 shown in FIGS.

  FIG. 19 is a diagram for explaining another arrangement of the color filter 102. In the example of FIG. 19, the green filter is assigned to the upper left pixel, the red filter is assigned to the lower left pixel, the blue filter is assigned to the upper right pixel, and the no-filter or no-color filter is assigned to the lower right pixel. For example, the array shown in FIG. 19 may be assigned to four pixels such as the repeating pattern 110 shown in FIGS.

  In the above embodiment, the liquid crystal shutter 300 is disposed between the insulating layer 144 and the color filter 102, but the arrangement location is not limited thereto. For example, the liquid crystal shutter 300 may be disposed immediately above the photoelectric conversion element 108, directly above the wiring layer 105, between the microlens 101 and the color filter, and the like. Further, instead of providing the liquid crystal shutter 300 for all the pixels, the liquid crystal shutter 300 or the individual electrode 310 may not be provided for some pixels.

  In the above embodiment, the liquid crystal shutter 300 is normally white, but may be normally black. Further, a mode other than the TN mode may be used, and the driving method may be active or passive. The liquid crystal shutter 300 may be provided upside down, and the common electrode 316 may be disposed closer to the photoelectric conversion element 108 than the individual electrode 310. An antireflection film may be provided on one or both of the polarizing films 365 and 366 of the liquid crystal shutter 300. Thereby, it is possible to prevent the light reflected from the surface of the liquid crystal shutter 300 from entering the adjacent photoelectric conversion element 108 as stray light.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  DESCRIPTION OF SYMBOLS 10 Digital camera, 20 Shooting lens, 21 Optical axis, 30 Subject, 100 Image sensor, 101 Micro lens, 102 Color filter, 104, 104l, 104r Aperture, 105 Wiring layer, 106 Wiring, 107 aperture, 108 Photoelectric conversion element, 109 Substrate, 110 repeat pattern, 111 repeat pattern, 112 repeat pattern, 113 repeat pattern, 114 repeat pattern, 115 repeat pattern, 116 repeat pattern, 117 repeat pattern, 118, 119 image sensor, 123 aperture mask, 124 aperture, 125 synthetic aperture , 140 circuit region, 142 floating diffusion, 144 insulating layer, 201 control unit, 202 A / D conversion circuit, 203 memory, 220 memory card, 20 7 memory card IF, 208 operation unit, 209 display unit, 210 LCD drive circuit, 211 AF sensor, 205 image processing unit, 230 drive unit, 232 reading drive unit, 234 opening / closing drive unit, 300 liquid crystal shutter, 302 seal unit, 304 , 306 substrate, 316 common electrode, 310 individual electrode, 312 individual electrode, 320 liquid crystal, 321 alignment film, 322 alignment film, 323 electrode part, 324 opening, 330 individual electrode, 350 driving element, 352 driving element, 360 connection part, 361, 362, 363, 364 Open / close region, 365, 366 Polarizing film, 370 Individual electrode, 372 Driving element, 374 Individual electrode, 376 Driving element, 378 Individual electrode, 380 Driving element, 382 Individual electrode, 384 Driving element

Claims (10)

A plurality of two-dimensionally arranged photoelectric conversion elements that photoelectrically convert incident subject luminous flux into electrical signals;
A closed state that is provided corresponding to at least one of the plurality of photoelectric conversion elements and switches at least a part of the cross-sectional area of the subject light flux by switching optical characteristics, and transmits the part. An image sensor comprising: an opening / closing part capable of switching between open states.   The imaging device according to claim 1, wherein the open / close section causes a displaced partial region in a cross-sectional region of the subject light beam to be incident on a corresponding one of the plurality of photoelectric conversion elements in a closed state.   The image sensor according to claim 1, wherein the opening / closing part is formed by a liquid crystal shutter.   The imaging device according to claim 3, wherein the opening / closing unit includes a plurality of segments that can be switched between an open state and a closed state with respect to one of the photoelectric conversion elements.   The imaging device according to claim 1, wherein the opening / closing portion is provided with an antireflection film.   The imaging element according to claim 1, wherein a plurality of the opening / closing sections are provided corresponding to the plurality of photoelectric conversion elements on a one-to-one basis.   An opening mask that is fixed in a one-to-one correspondence with each of the plurality of photoelectric conversion elements and transmits a part of the cross-sectional area of the subject luminous flux that is different from the closed state in the corresponding one of the opening / closing sections. The imaging device according to claim 6 further provided. A plurality of microlenses provided in a one-to-one correspondence with each of the plurality of photoelectric conversion elements;
8. The image sensor according to claim 6, wherein each of the plurality of opening / closing sections is provided between a corresponding one of the plurality of microlenses and a corresponding one of the plurality of photoelectric conversion elements.   The imaging device according to claim 1, further comprising a plurality of color filters provided in a one-to-one correspondence with each of the plurality of photoelectric conversion devices. The imaging device according to any one of claims 1 to 9,
An image pickup apparatus comprising: an image processing unit that reads out an image pickup signal picked up by the image pickup device and generates image data based on an open / close state of the open / close unit at the time of image pickup.

Images