JP2002191060A - Three-dimensional imaging unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional imaging unit that separates an image of a focal plane imaged by a single-lens camera into two depending on the focused state so as to obtain distance information of an object or an entire scene through the use of a correlation method. SOLUTION: Micro split prism pairs 21 in the three-dimensional imaging unit are arranged as a two-dimensional array to configure a split prism plate 22. A micro lens 25 has optical transmission sections 39, 40 that can sense a light for each pixel and a light shade sections 41, 42 that do not sense a light. The micro slit prism pair 21 has micro spit prisms 31, 32 that are components of the pair 21 and they are placed in a state of crossing in the opposite direction. Furthermore, pixels 33, 34 belonging to respective rows of a two-dimensional solid-state imaging device are placed behind the micro split prisms 31, 32.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to distance information (distance image) in addition to a grayscale image (luminance image) of a subject as an object.
And a three-dimensional imaging apparatus for obtaining three-dimensional information of a target object by using the acquired distance image in real time.

[0002]

2. Description of the Related Art Methods for passively obtaining distance information are roughly classified into a correlation method and a contrast detection method. Here, we focus on the former.

[0003] As an example of the former, a TTL-SIR (Through The Lens) used in a binocular or multi-view stereo system, or autofocus (AF) is used.
Secondary imaged Registra
tion; secondary imaging phase difference detection) or USP4,1
85, 191 (ThroughCa
(Mera Lens) system and the like are known.

[0004] In the stereo system, parallax is obtained by taking correspondence between a plurality of images having parallax captured by two eyes or multiple eyes, and distance information of a subject is obtained from the parallax.

Here, the principle of a TTL-SIR or TCL system used for AF will be described with reference to FIGS.

In FIGS. 7 (a) to 7 (c), for the light beam condensed on the focusing surface 2 via the photographing lens 1, the light beam passing through the upper half of the pupil of the photographing lens 1 and the lower half thereof are shifted. It is decomposed into a luminous flux.

FIG. 7A shows a state where the object P ₀ is in focus, FIG. 7B shows a state where the object P _{1 is} in front focus,
And Figure. 7 (c) the state of the rear focus on the subject P _2, represents respectively.

When a light beam passing through the upper half of the pupil of the photographing lens 1 is represented by a solid line, and a light beam passing through the lower half is represented by a broken line,
The position where both pass through the focus surface 2 has the property of being inverted between the front pin and the rear pin. Using this, the in-focus state is determined.

Specifically, a plurality of elements having the configuration shown in FIG. 8A are arranged in an array on the focus surface 2. FIG.
In (a), two light receiving elements 4 and 5 are arranged corresponding to a micro lens (or a fly-eye lens) 3. The light-receiving element 4 is located
As shown by the broken line, the incident light beam 7 is received. The light receiving element 5 receives a light beam incident on the microlens 3 as shown by a solid line. In other words, each of the light receiving elements 4 and 5 has directionality with respect to the received light beam.

A plurality of (for example, 24) sets of such microlenses and light receiving elements are arranged in the vicinity of the focusing surface 2. In order to distinguish the directionality, the light receiving element group arranged at the position of the light receiving element 4 is referred to as an A series, and the light receiving element group arranged at the position of the light receiving element 5 is referred to as a B series.
Then, the light intensity distribution received in the A series and the light skin distribution received in the B series are shifted from the in-focus state as shown by the light intensity distributions 8 and 9 in FIG. 8B, respectively. In the front focus state or the rear focus state, the image is shifted right and left according to the shift amount from the in-focus state. By measuring this, the shift amount (defocus amount) from the in-focus state can be known.

In FIG. 8B, the vertical axis represents (C
The light intensity distribution (taken with a CD) and the horizontal axis are spatial coordinates or numbers (in the scanning direction of the image sensor, for example, in a row).

As shown in FIG. 9, a split image system used for focusing a camera is a combination of two thin wedge-shaped prisms 11 and 12 in opposite gradients (referred to as a split prism pair). Thus, the intersection of the oblique sides of the prisms 11 and 12 determines the focus plane 13. The light beam 15 converging at a point P ′ on the surface 14 shifted by dz from the focus surface 13 is converted into a light beam diverging from two points A and B separated by 2δdz. Here, δ is the declination angle of each prism, and is represented by (n−1) α, where α is the vertex angle of each prism and n is the refractive index. This is observed with the naked eye 17 via the eyepiece 16. Actually, the focus is adjusted so that the lateral displacements of the images displayed in the upper and lower windows instead of the points coincide. This can be classified as a correlation method among the two methods described above.

[0013]

In the binocular or multi-view stereo method, calibration of camera parameters such as internal parameters such as a focal length, external parameters such as the position and orientation between cameras, and the like, It is necessary to solve the so-called response problem that requires response.
However, both are challenging issues, and we are still in the process of seeking satisfactory solutions every day.

In the TTL-SIR or TCL system, distance information is obtained for the center of the field of view of a camera or a partial area of a subject, and distance information of the entire scene cannot be obtained.

In addition, the twin-lens or multi-lens stereo method has a problem that coordination with a zoom operation and a focusing operation is not performed well. For example, when performing a zoom operation, in the two-lens or multi-view stereo method, it is necessary to cause all the lenses to be used to perform the same zoom operation. In addition, when the focusing operation is performed, camera parameters change, and there is a difficulty that correction of calibration is required in some cases.

Accordingly, in the present invention, distance information is obtained by using a correlation method by separating an image of a focus plane obtained by capturing distance information of a subject or an entire scene with a single-lens camera in accordance with an in-focus state. It is an object of the present invention to provide a three-dimensional imaging device capable of performing such operations.

It is another object of the present invention to provide a three-dimensional image pickup apparatus capable of reducing a corresponding problem without calibrating camera parameters.

[0018]

That is, in the three-dimensional imaging apparatus according to the first aspect of the present invention, at least a part of a surface on which light is incident is divided into a plurality of regions, and each light beam incident on each of the regions is An optical element arranged near the focus plane of the imaging optical system, the optical element being configured to be deflected in one direction determined for each region from among a plurality of predetermined directions; A lens array in which each lens is disposed coaxially, and an imaging element having a light receiving surface at a position substantially conjugate with the optical element with respect to the lens array, and capable of imaging an image for each deflection direction of the light beam. , Is provided.

A three-dimensional imaging apparatus according to a second aspect of the present invention includes a split prism plate provided near an in-focus position of an imaging lens and including an array of inversely crossed micro split prism pairs, and a coaxial with the micro split prism pair. An array of microlenses arranged in a matrix and a focusing direction of the microsplit prism pair with respect to the microlens array are arranged conjugate with each other, and have a scanning direction parallel to the longitudinal direction of the microsplit prism pair. A solid-state image sensor array for picking up signal light from the split prism with separate scanning lines.

According to a third aspect of the present invention, there is provided a three-dimensional imaging apparatus, comprising: a light-receiving element having a plurality of pixels, each of which has a light-sensitive area and a non-light-sensitive area where light is not sensitized; A lens array in which each lens is disposed in correspondence with the above. The relative positional relationship between the photosensitive region of each pixel and the optical axis of the corresponding lens is determined by a plurality of predetermined relative positional relationships. , And the light receiving element is capable of capturing an image separately for each relative positional relationship.

Further, a three-dimensional imaging device according to a fourth aspect of the present invention is a solid-state imaging device array with microlenses provided near the focus position of the imaging lens, wherein the optical axis of the microlens array and the solid-state imaging device array are arranged. A relative positional relationship with a photodiode opening provided in each pixel is shifted by half a pixel in a scanning direction between an even-numbered scan line and an odd-numbered scan line.

In the three-dimensional image pickup apparatus according to the first aspect of the present invention, the optical element disposed near the focus plane of the image pickup optical system has at least a part of the light incident surface divided into a plurality of regions. I have. Each light beam incident on each of the regions is configured to be deflected to one direction determined for each region from among a plurality of predetermined directions.
Further, in the lens array, each lens is coaxially arranged in each of the above regions. The imaging element has a light receiving surface at a position substantially conjugate with the optical element with respect to the lens array, and is capable of capturing an image while distinguishing an image for each deflection direction of the light beam.

In the three-dimensional image pickup apparatus according to the second aspect of the present invention, the split prism plate provided near the focus position of the image pickup lens includes an array of reversely crossed micro split prism pairs. An array of microlenses is arranged coaxially with the split prism pair. The solid-state imaging device array is arranged conjugate with the focus plane of the micro split prism pair with respect to the micro lens array, has a scanning direction parallel to the longitudinal direction of the micro split prism pair, and includes a micro split prism. Are picked up by separate scanning lines.

In the three-dimensional image pickup apparatus according to the third aspect of the present invention, the light receiving element has a plurality of pixels each having a light-sensitive area and a light-insensitive area for each pixel. Further, each lens is arranged in a lens array corresponding to each pixel of the light receiving element. The relative positional relationship between the photosensitive region of each pixel and the optical axis of the corresponding lens is one of a plurality of predetermined relative positional relationships.
It is configured to correspond to one. The light receiving element,
An image can be captured separately for each relative positional relationship.

Further, in the three-dimensional image pickup apparatus according to the fourth invention, a solid-state image pickup device array with microlenses is provided near the focus position of the image pickup lens. Then, the relative positional relationship between the optical axis of the microlens array and the openings of the photodiodes provided in each pixel of the solid-state imaging device array is half in the scanning direction between the even-numbered scan lines and the odd-numbered scan lines. They are displaced by pixels.

[0026]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a first embodiment of the present invention. FIG. 1A shows a micro-split prism pair applied to a three-dimensional image pickup apparatus (hereinafter referred to as a micro-split prism pair). )
(B) is a diagram showing a part of FIG. (A) in three dimensions, (c).
FIG. 3 is a view showing a micro lens.

In FIG. 1A, the micro split prism pairs 21 are arranged in a two-dimensional array to form a split prism plate 22. In the split prism plate 22, at least a part of a surface on which light enters is divided into a plurality of regions, and each light beam incident on each of the regions is divided into a plurality of predetermined directions for each region. An optical element configured to be deflected in one defined direction.

In the drawing, reference numeral 23 denotes a focus surface. This focus surface 23 is a micro split prism pair 2
Pass through the intersection of the hypotenuse of 1.

The above micro split prism pair 21
Behind the incident light ray 28 of the micro lens 25,
A two-dimensional solid-state image sensor 26 is provided. On the other hand, a defocus surface 24 is formed by defocusing a predetermined distance dz from the focus surface 23. This defocus surface 24
, An optical axis 27 is orthogonal, and a converging light beam 28 is incident. Then, in the figure, O ₀ its convergence point, two points O _1,
O ₂ represents two points separated by 2δdz corresponding to the defocus amount dz.

FIG. 1B shows a part of FIG. 1A three-dimensionally, but shows the micro split prism pair 21 with the micro lens 25 removed to avoid complication.

In FIG. 1B, the micro-split prism pair 21 has micro-split prisms 31 and 32, which are the respective components, and these micro-split prisms 31 and 32 are arranged in a state of reverse intersection. Have been. The micro split prisms 31 and 32
Behind the two-dimensional solid-state imaging device, for example, even-numbered,
Pixels 33 and 34 belonging to each odd-numbered row are arranged. The direction of arrow C in the figure indicates the direction of scanning.

Here, the micro-split prism arranged in the state of reverse crossing is a combination of two thin wedge-shaped prisms having opposite gradients. Normally, a point (actually, a line) where the steepest slope line is seen from the lateral direction determines the focus position.

In the split prism pair, as one definition of the crossing, assuming a plane perpendicular to both the focus plane and the boundary plane between the prism bears in the split prism pair, one of the prisms has The normal direction of the light entrance or exit surface and the normal vector of the corresponding surface of the other prism are on different sides with respect to the assumed plane. (A state in which both vectors are on different sides with respect to the plane means that if the starting points of both vectors are placed on the plane, each end point is located in a different space of the space divided into two by the plane. Meaning.

A light ray 35 incident parallel to the optical axis 27 defined by the micro-split prisms 31 and 32
And 36 are the micro split prism 3 respectively.
Deflected at 1 and 32, as in pixels 37 and 38,
The light is incident on pixels that are shifted in the front-rear direction with respect to the scanning direction C.

At least a part of the entrance surface of the split prism plate has a portion where a split prism group is formed. At this time, a split prism group may be formed on the entire incident surface. Then, the incident surfaces of the individual prisms of the split prism group are regarded as one region. The light incident on each area (incident surface of each prism) is deflected to one direction determined for each area from a plurality of predetermined directions (two directions in the present embodiment). Is done. Each prism deflects light in one of two directions.

As described above, the split prism plate is suitable, but not limited to this. At least a part of the surface on which light is incident is divided into a plurality of regions, and each of the light incident on each of the above regions is divided into a plurality of regions. It is sufficient that the light beam is an optical element configured to be deflected to one direction determined for each region from among a plurality of predetermined directions.

As shown in FIG. 1C, the microlens 25 is configured by alternately arranging light transmitting portions 39 and 40 and light shielding portions 41 and 42 indicated by oblique lines in the drawing. ing.

FIG. 2A is a diagram showing an outline of the overall configuration of the present three-dimensional imaging apparatus.

In FIG. 2A, a light beam incident along an optical axis 44 is reflected by a quick return mirror (or semi-transparent mirror) 46 through an imaging lens 45, and is reflected by a lens 47. The detection amount 48 of the focus amount dz is reached. When the quick return mirror 46 is retracted from the optical path, the light beam that has passed through the imaging lens 45 is
The main two-dimensional solid-state imaging device 49 is reached.

FIG. 2B shows an optical path of a principal ray incident on the image pickup apparatus, wherein H ₁ and H ₂ represent principal points of the image pickup lens 45.

Since the structure of the detection device 48 has a certain thickness, it is desirable that the incident light beam be incident on the detection device 48 substantially perpendicularly. For this reason, the lens 47 is mounted in front of the detecting device 48, and the imaging optical system (45 and 47) is made telecentric (the principal ray is made to enter the detecting device 48 almost perpendicularly).

However, in a camera using a CCD or a CMOS as an image pickup device, an image pickup optical system is often designed so that a chief ray is vertically incident on the image pickup device (image pickup lens 4).
5 is telecentric), but in such an imaging optical system, the lens 47 is not required.

Next, the operation of the three-dimensional imaging apparatus thus configured will be described.

In FIGS. 1A and 1B, the light beam converging on the point O ₀ of the defocus surface 24 shifted by dz from the focus surface 23 is the micro split prism pair 21.
, The light diverges from two points 0 ₁ and 0 ₂ separated by 2δdz. The images of these two points 0 ₁ and 0 ₂ are formed on the two-dimensional solid-state imaging device 26 via the micro lens 25. The magnification M is set to, for example, “1”.

In this case, an image of the same size at the two points 0 ₁ and 0 ₂ separated by 2δdz by the microlens 25 is formed on the two-dimensional solid-state image sensor 26. The longitudinal length of the micro split prism pair 21 is 2
The three-dimensional solid-state imaging device 26 has a length of, for example, several pixels, and the width of each pixel is one pixel.

Further, the micro split prism pair 2
1 is arranged in parallel with the scanning direction of the two-dimensional solid-state imaging device 26. In other words, the odd-numbered scanned image is, for example, the micro split prism pair 21
Of the micro-split prism pair 21 is picked up, and the image scanned even-numbered is an image that has passed through the other of the micro split prism pair 21. Both provide two images that are shifted in the horizontal direction by a pair of 2Mδdz by the function of the micro-split brain pair 21.

In the case of normal camera focusing, the deviation is adjusted by the naked eye so that the shift amount becomes zero. Here, a micro lens 25 shown in FIG. 1C is used instead of the naked eye. This is because the micro lens 25 has the light shielding portion 41 and the micro lens 2
The right half of 5 selectively transmits a light beam from one of the two divided pupils (for example, the lower half) of a photographic lens (not shown), and the left half of the microlens 25 is a pupil of a photographic lens (not shown) divided into two. This has a role of transmitting the light flux from the other (for example, the upper half) preferentially.

With such considerations, the odd-numbered scanned image and the even-numbered scanned image paired by the micro-split brace pair 21 are stored in a memory (not shown), respectively. The correlation between each pair of rows is taken, and a map of the lateral shift amount 2Mδdz (a function of the position on the image sensor) is obtained. Two-dimensional data of the defocus amount dz, that is, M and δ are known, in other words , Distance information of the entire subject or the scene can be obtained.

At this time, since the two images to be correlated are images captured by the same imaging lens and imaging device,
Compared to a case where a correlation between two images taken with different arrangements and postures of different imaging lenses is taken, the simplification is greatly simplified, and a so-called correspondence problem is reduced.

Defocus amount dz and distance a to the subject a
The following equation is established between the above and via the focal length f of the imaging lens. (1 / a) + (1 / b) = 1 / f (1) Δb = − (b ² / a ² ) △ a≡dz (2) where b is an image plane (focus) Plane). The distance a to the subject is obtained from the AF function of the imaging apparatus, and the defocus amount dz and the above (1)
The three-dimensional information Δa of the subject can be obtained from the equation (2) and the equation (2).

The AF function of the imaging apparatus uses a signal from a dedicated AF sensor or uses the function of the micro split prism pair 21 described above, for example, an average defocus amount in an area near the center of the visual field. Find dz,
This can also be achieved by feeding back the imaging lens so that this becomes zero.

As described above, the image pickup device capable of picking up an image by distinguishing the image for each deflection direction of the light beam is distinguished by associating an odd number and an even number of scanning lines for each light deflection direction (two directions). Receiving light. The above-mentioned image pickup device picks up an image while distinguishing the image for each elevation direction of light, and it is a preferable method to distinguish the image by a scanning line.

Next, referring to FIGS. 2A and 2B, an image of a subject (not shown) captured by the imaging lens 45 is a quick return mirror (or a semi-transparent mirror).
46, the grayscale image (luminance image) of the main 2
The image is captured by the three-dimensional solid-state image sensor 49. The distance information of the subject is obtained by the detection device 48 for the defocus amount dz as described above. The operation of the lens 47 is performed by converting a principal ray (a ray bundle converging to the principal point H ₁ and diverging from the principal point H ₂ ) constituting an angle of view of the imaging lens 45 into a principal ray (telecentric optical system) parallel to the optical axis. To convert.

Next, the function of the micro split prism pair 21 shown in FIG. 1A will be described.

In FIG. 3A, if a macro split prism 51 (for example, the prism 11 in FIG. 9) is schematically represented, it corresponds to a micro split prism 52. This is similar to a macro prism being equivalent to a linear Fresnel lens. The height of the micro split prism 52 is selected to be an integral multiple of the center wavelength of the spectral region used for imaging.

FIG. 4 explains the improvement of the sensitivity of the micro split prism pair 21.

Normal micro split prism 53
Has a shape as shown in FIG.
The micro split prism 54 shown in FIG.
FIG. 9A is a diagram in which the pitch of the micro split prism 53 shown in FIG. In this case, the apex angle of the micro-split prism 54 (the bottom angle on the left side in the figure) is about four times the apex angle of the micro-split prism 53, and the sensitivity is four times. This role is effective, for example, when it is desired to select a higher distance resolution at the center of the visual field than at the periphery.

As described above, according to the first embodiment,
The split prism plate 22 shown in FIG. 1A basically has linear Fresnel lenses formed every other row, and has a possibility that the sensitivity at the center of the field of view and the sensitivity at the periphery can be arbitrarily changed. I have. Further, by using a two-dimensional array having a microlens 25 as an element as a substrate, a micro split prism pair 21
There is also an advantage that the dimensional array can be integrated and closely formed.

Many changes can be made to the first embodiment.

Next, one of them will be described with reference to FIG.

A curve 55 shown in FIG. 5A represents an aspherical surface. Further, a curve 56 shown in FIG.
Represents an approximation of the aspheric surface 55 with a Fresnel surface equivalent to the aspheric surface 55. By replacing the micro-split prism with a Fresnel surface equivalent to the aspheric surface 55,
Since the sensitivity of the micro split prism is proportional to the apex angle, it can be seen in FIG. 5B that the sensitivity increases from the right side to the left side.

Therefore, for example, if the left end is set as the center of the field of view, a function is obtained in which the sensitivity increases nonlinearly at the center of the field of view, and decreases as it moves to the periphery. This simulates the function that the human eye has high sensitivity at the center of the visual field.

In FIG. 1B, the width of each of the micro-split prisms 31 and 32 is set to the width of one pixel. However, writing into the memory is slightly complicated.
Of course, it is possible even if it extends over a plurality of lines.

According to the present invention, the image handled by the detecting device 48 shown in FIG. 2A is obtained by the single-lens image pickup lens 45, and the luminance image picked up by the main two-dimensional solid-state image sensor 48 Since they are the same, the imaging lens 4
It has a feature that it can operate in cooperation with the zoom operation and focusing operation of No. 5. This is a difficult task for a multi-view stereo, as described above.

Another feature is that distance information can be obtained by correlating distance information of the entire visual field, instead of obtaining distance information only at the center of the visual field as seen in a conventional focusing device.

Further, by correlating the image obtained from the even-numbered scan of the image obtained by the single-lens imaging lens 45 with the image obtained from the odd-numbered scan paired with the image, So-called response problems can be reduced.

Next, a second embodiment of the present invention will be described.

FIG. 6 is a diagram showing a configuration example of a three-dimensional solid-state imaging device according to the second embodiment of the present invention.

In FIG. 6, the two-dimensional solid-state imaging device is configured to have rows 60, 61, and 62. Each row 6 above
0, 61, and 62 have pixels 65, 66,... For example, the pixel 65 has a photodiode (PD) opening 64 that is a photosensitive area. Further, in the pixel 65, a portion other than the PD opening 64 is a non-photosensitive region. Incidentally, 63 is a micro lens.

The arrangement of the PD openings 64 in the pixels 65 is the same in each row, but the position of the microlenses is shifted by half a pixel in the row 61 compared to the adjacent rows 60 and 62. In other words, the positions of the PD openings 64 in the pixels in the even-numbered rows and the odd-numbered rows are the same, but the microlenses are shifted by half a pixel.

Next, the operation of the second embodiment will be described.

It is desirable that the micro lens 63 has a dense structure. For example, in the example shown in FIG.
It has a hexagonal shape. Since this center is displaced from the center of the PD opening 64, the received light beam has directivity or directivity. This directivity increases as the width of the PD opening 64 (width parallel to the scanning direction of the row 60) decreases.

The directivity of light reception in an adjacent row takes a direction opposite to the optical axis of the microlens in each row.
This takes a form in which the function described in FIG. 8A is shared by two adjacent rows.

In other words, in the images captured by the even-numbered rows and the odd-numbered rows of the two-dimensional solid-state imaging device (referred to as even-numbered images and odd-numbered images, respectively), when the subject is viewed through the divided pupils, A lateral displacement corresponding to the generated parallax occurs, and distance information corresponding to the parallax based on the pupil division principle shown in FIG. 7 can be obtained by correlating each row between the even image and the odd image.

As described above, the second embodiment does not require an optical element having anisotropy such as the micro-split prism found in the first embodiment described above. Depending on the position of the PD opening in the corresponding pixel, the light receiving direction has anisotropy. Therefore, there is an advantage that the configuration is simpler than that of the first embodiment and the manufacture is easy.

Various modifications can be made to the second embodiment.

In FIG. 6, the microlenses are shifted by half a pixel for each row. However, the microlenses may be arranged in a square shape in a square shape, and the PD aperture may be shifted by a half pixel for each row. .

[0079]

As described above, according to the present invention, the distance information of the subject or the whole scene is separated by a single-lens camera into two images in focus according to the in-focus state. , A three-dimensional imaging device capable of obtaining distance information by using the method can be provided.

Further, according to the present invention, it is possible to provide a three-dimensional image pickup apparatus capable of reducing a corresponding problem without calibrating camera parameters.

[Brief description of the drawings]

FIG. 1 shows a first embodiment of the present invention.
(A) is a cross-sectional view illustrating a microscopic split prism pair applied to a three-dimensional imaging device, (b) is a diagram showing a part of (a) in a three-dimensional manner, and (c) is a microlens. FIG.

2A is a diagram illustrating an outline of an overall configuration of a three-dimensional imaging device, and FIG. 2B is a diagram illustrating an optical path of a principal ray incident on the present imaging device.

FIG. 3 is a view schematically showing a macro split prism 51.

4A and 4B are diagrams illustrating an improvement in sensitivity of the micro split prism pair 21. FIG. 4A schematically illustrates a normal micro split prism 53, and FIG. 4B illustrates a micro split prism illustrated in FIG. FIG. 53 is a diagram in which the pitch of 53 is shortened to １ ／.

5A is a modification of the first embodiment, and FIG.
FIG. 5B is a diagram in which the aspheric surface 55 is approximated by an equivalent Fresnel surface.

FIG. 6 is a diagram illustrating a configuration example of a three-dimensional solid-state imaging device according to a second embodiment of the present invention.

FIG. 7 shows TTL-SIR or TC used for AF.
Serve to explain the principle of L system, (a) represents showing a state focus on the subject P ₀ Figure, (b) is a diagram showing a state before the pin with respect to the subject P _1, (c ) is a diagram showing a state of the rear focus on the subject P _2.

FIG. 8 shows TTL-SIR or TC used for AF.
4A and 4B illustrate the principle of the L system. FIG. 4A is a diagram illustrating a relationship between a photographing lens and a light receiving element, and FIG. 5B is a light intensity distribution diagram of a light receiving element group arranged at the light receiving element position in FIG. It is.

FIG. 9 is a diagram illustrating a split image method used for focusing a camera.

[Explanation of symbols]

 21 micro split prism pair, 22 split prism plate, 23 focus surface, 24 defocus surface, 25 micro lens, 26 solid state image pickup device, 27, 44 optical axis, 28, 35, 36 light beam, 31, 32 micro split prism, 33 , 34, 37, 38 pixels, 39, 40 light transmitting part, 41, 42 light shielding part, 45 imaging lens, 46 quick return mirror (semi-transparent mirror), 47 lens, 48 detecting device, 49 two-dimensional solid-state image sensor.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G02B 5/04 G02B 5/04 F G03B 15/00 G03B 15/00 BU 35/08 35/08 H04N 5 / 225 H04N 5/225 Z F term (reference) 2F065 AA06 AA53 FF10 JJ03 JJ09 JJ26 LL00 LL04 LL10 LL11 LL47 LL59 QQ23 QQ38 QQ41 2H042 CA12 CA17 2H059 AA09 5C022 AA13 AB26 AC42 AC54 5C06 AB

Claims (7)

[Claims] At least a part of a surface on which light is incident is divided into a plurality of regions, and each light beam incident on each of the regions is determined for each region from a plurality of predetermined directions. An optical element arranged near the focus plane of the imaging optical system, configured to be deflected in one direction, a lens array in which each lens is arranged coaxially in each of the regions, and the lens array A three-dimensional imaging device, comprising: a light receiving surface at a position substantially conjugate with the optical element, and an image pickup element capable of distinguishing an image for each deflection direction of the light flux. 2. A split prism plate including an array of reversely crossed micro split prism pairs provided near a focus position of an imaging lens; an array of micro lenses arranged coaxially with the micro split prism pairs; The microlens array is arranged conjugate with the focus plane of the microsplit prism pair, has a scanning direction parallel to the longitudinal direction of the microsplit prism pair, and separates the signal light from each microsplit prism into a separate scanning line. A three-dimensional imaging device, comprising: a solid-state imaging device array that performs imaging by: 3. The three-dimensional imaging apparatus according to claim 2, wherein the split prism plate is formed in close contact with the microlens array. 4. The three-dimensional imaging device according to claim 2, wherein the split prism plate has a region in which sensitivity is improved at a central portion of a visual field. 5. The apparatus according to claim 4, further comprising sensitivity improving means for improving the sensitivity at the central portion of the visual field, wherein said sensitivity improving means includes an aspherical surface or a linear Fresnel lens equivalent thereto. The three-dimensional imaging device according to claim 1. 6. A light-receiving element having a plurality of pixels, having a photosensitive area capable of sensitizing light and a non-photosensitive area not sensitizing light for each pixel, and each lens being provided corresponding to each pixel of the light-receiving element. And the relative positional relationship between the photosensitive region of each pixel and the optical axis of the corresponding lens corresponds to one of a plurality of predetermined relative positional relationships. The three-dimensional imaging device is characterized in that the light-receiving element is capable of imaging separately for each of the relative positional relationships. 7. A solid-state imaging device array with microlenses provided near a focus position of an imaging lens, comprising: an optical axis of the microlens array; and a photodiode opening provided in each pixel of the solid-state imaging device array. A three-dimensional imaging apparatus, wherein the relative positional relationship between the even-numbered scanning lines and the odd-numbered scanning lines is shifted by half a pixel in the scanning direction.