JPH04236935A - line of sight detection device - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention relates to an optical device having a line of sight detection device, and more particularly, the present invention relates to an optical device having a line of sight detection device, and particularly to an optical device that allows an observer (photographer) to The object has a line-of-sight detection device that detects the axis of the direction of the gaze point observed by the observer, the so-called line of sight (visual axis), by using the reflected image of the eyeball obtained when the surface of the observer's eyeball is illuminated. It concerns an optical device, for example a camera.

0002

2. Description of the Related Art Various devices (for example, eye cameras) have been proposed to detect what position on an observation surface an observer is observing, or to detect the so-called line of sight (visual axis).

For example, in Japanese Patent Laid-Open No. 61-172552, a parallel light beam from a light source is projected onto the anterior segment of the observer's eyeball, and the corneal reflection image due to the light reflected from the cornea and the image formation position of the pupil are used. and looking for the visual axis. Figure 12(A),
(B) is a diagram explaining the principle of the line of sight detection method, (A) is a schematic diagram of the main parts of the line of sight detection optical system, and (B) is an illustration of the output signal from the photoelectric element array 6 in (A). It is an explanatory diagram of strength.

In the figure, reference numeral 5 denotes a light source such as a light emitting diode that emits infrared light that is insensitive to the observer, and is placed on the focal plane of the projection lens 3.

The infrared light emitted from the light source 5 is transmitted through the projection lens 3.
The light becomes parallel and is reflected by the half mirror 2, and the eyeball 20
1's cornea 21 is illuminated. At this time, a corneal reflected image (virtual image) d, which is a part of the infrared light reflected on the surface of the cornea 21, passes through the half mirror 2, is focused by the light receiving lens 4, and is re-imaged at a position Zd' on the photoelectric element array 6. do.

Furthermore, the light beams from the ends a and b of the iris 23 form images of the ends a and b at positions Za' and Zb' on the photoelectric element array 6 through the half mirror 2 and the light receiving lens 4. do. When the rotation angle θ, which is the angle between the optical axis (optical axis A) of the light-receiving lens 4 and the optical axis I of the eyeball, is small, the end portion a of the iris 23
, b as Za, Zb, the coordinate Zc of the center position c of the pupil 24 is expressed as Zc≈(Za+Zb)/2.

Furthermore, since the Z coordinate of the corneal reflected image d and the Z coordinate of the center of curvature O of the cornea 21 match, the Z coordinate of the generated position d of the corneal reflected image is Zd, and the distance from the center of curvature O of the cornea 21 to the pupil 24 is If the distance to the center C is LOC, then the rotation angle θ, which is the angle between the eyeball optical axis A and the optical axis A, is LOC*SINθ≒Zc−Zd (1)
The relational expression is approximately satisfied.

For this reason, in the calculation means 9, as shown in FIG.
The rotation angle θ of the optical axis i of the eyeball 201 can be determined by detecting the position of each singular point (the corneal reflection image d and the ends a and b of the iris) projected onto the six surfaces of the photoelectric element array as shown in FIG. can. At this time, equation (1) can be replaced as follows. However, β is a magnification determined by the position of the eyeball with respect to the light receiving lens 4.

By the way, the optical axis A of the observer's eyeball does not coincide with the visual axis. Japanese Unexamined Patent Publication No. 1-274736 discloses detecting the line of sight by correcting the angle between the optical axis and the visual axis of the observer's eyeball. Here, the horizontal rotation angle θ of the optical axis of the observer's eyeball is calculated, and when the angle correction value between the optical axis of the eyeball and the visual axis is δ, the horizontal line of sight θH of the observer is calculated.
θH=θ±δ ‥‥‥‥(
3). Here, if the rotation angle to the right with respect to the observer is positive, the sign ± is selected as + if the observer's left eye looks into the observation device, and - if the observer has the right eye.

Furthermore, although FIG. 12A shows an example in which the observer's eyeball rotates within the Z-X plane (for example, a horizontal plane), the observer's eyeball rotates within an X-Y plane (for example, a vertical plane). It can be detected in the same way even when the object rotates within the object.

However, since the vertical component of the observer's line of sight coincides with the vertical component θ' of the optical axis of the eyeball, the vertical line of sight θV is θV=θ' (4
).

FIG. 13 is a schematic diagram of a main part of an optical system when the line of sight detection device of FIG. 12 is applied to a part of a finder system of a single-lens reflex camera.

In the figure, object light transmitted through a photographing lens 101 is reflected by a flip-up mirror 102 and forms an image near the focal plane of a focusing plate 104. In addition, focus plate 10
The subject light diffused in 4 is passed through a condenser lens 105,
A photographer's eye point 201a is transmitted through a pentagonal roof prism 106 and an eyepiece 1 having a light splitting surface 1a.
is incident on .

The line of sight detection optical system includes an illumination means (optical axis c) consisting of a light source 5 such as an infrared light emitting diode that is insensitive to the photographer (observer) and a projection lens 3, a photoelectric element array 6, It consists of a light receiving means (optical axis A) consisting of a half mirror 2 and a light receiving lens 4, and is arranged above the eyepiece 1 having a light splitting surface 1a made of a dichroic mirror. The infrared light emitted from the infrared light emitting diode 5 is transmitted to the light splitting surface 1
It is reflected at point a and illuminates the photographer's eyeball 201. Furthermore, a part of the infrared light reflected by the eyeball 201 is transmitted to the light splitting surface 1.
The light is reflected again at a point a, and is focused onto a photoelectric element array 6 via a light receiving lens 4 and a half mirror 2. The direction of the photographer's line of sight is calculated in the calculation means 9 from the image information of the eyeball obtained on the photoelectric element array 6 (for example, the output signal shown in FIG. 12(B)). In other words, the point on the focus plane 104 that the observer is observing (
(attention point).

At this time, the focal plane 10 that the photographer sees from the horizontal line of sight θH and the vertical line of sight θV
The position (Zn, Yn) on 4 is obtained as follows. However, m is a constant determined by the camera's finder system.

In this way, if it is possible to know which position on the focus plane 104 the photographer is observing in a single-lens reflex camera, for example, if the point at which focus can be detected by the camera's automatic focus detection device is only the center of the screen, then If the camera is set at multiple locations on the screen, and the photographer wants to select one of the points for automatic focus detection, the camera can save the user the trouble of selecting and inputting that one point. This method is effective for automatically selecting a point, that is, a gaze point, as a point for focus detection and automatically selecting the point to perform automatic focus detection.

Cameras are generally used by many people, regardless of age or gender, and the size of the eyeballs of the photographers differs from person to person. Therefore, in commercially available eye cameras for measuring line of sight, errors in line of sight detection are corrected by correcting individual differences among users.

[0018]

[Problems to be Solved by the Invention] In the above-mentioned line of sight detection method, the calculation formula (2) for the rotation angle θ of the eyeball is based on the parameter LOC (from the center of curvature O of the angle 21 to the center of the pupil 24), which is related to the size of the eyeball. distance). For this reason, the size of the eyeball of the person using the camera, that is, the parameter LO
If C deviates significantly from the value corresponding to the preset distance LOC, an error will occur between the calculated rotation angle θ of the eyeball and the actual rotation angle of the eyeball, resulting in a decrease in line of sight detection accuracy. There was a point.

Furthermore, the correction angle δ between the optical axis and visual axis of the eyeball in equation (3) also varies depending on the characteristics such as the size of the photographer's eyeball. For this reason, if the correction angle δ is also set to a constant value, there is a problem that depending on the photographer, an error may occur between the calculated line-of-sight direction θH and the actual line-of-sight direction, reducing the accuracy of line-of-sight detection. There was a point.

[0020] Generally, in commercially available eye cameras for measuring line of sight, corrections are made for individual differences among users. However, since the optical axis of the user's eyeballs and the optical axis of the camera that captures the scenery that the user is thought to be viewing do not match, the index that the user is looking at must be kept away from the eye camera. The disadvantage is that the index cannot be integrated with the eye camera.

Furthermore, the eye camera is adjusted so that the position of the indicator photographed by the camera and displayed on the television monitor matches the position of the line of sight detected when the user is gazing at the indicator. The disadvantage was that it required an experimental assistant and was troublesome to make adjustments.

The present invention provides an optical constant detection means for detecting the optical constants of the observer's eyeballs (for example, the radius of curvature of the cornea and the depth of the anterior chamber) and a line-of-sight correction calculation for correcting line-of-sight detection errors due to individual differences in the optical constants of the eyeballs. The purpose of the present invention is to provide an optical device equipped with a line-of-sight detection device that can automatically correct line-of-sight detection errors due to individual differences such as eyeball size and perform highly accurate line-of-sight detection. do.

[0023]

[Means for Solving the Problems] An optical device having a line of sight detection device of the present invention detects the line of sight of an observer looking into a finder system with a line of sight detection means, and uses a value obtained by the line of sight detection means. The optical constant of an observer's eyeball is detected by an optical constant detection means, and the line of sight detected by the line of sight detection means is corrected by a line of sight correction calculation means using the value obtained by the optical constant detection means. It is a feature.

In particular, the present invention is characterized in that the eyeball optical constant detection means includes an optical constant detection execution switch, an in-finder display means, an optical constant calculation means, and an eyeball optical constant storage means. .

[0025]

[Example] Figure 1 (A) is a schematic diagram of the main parts of the optical system of Example 1 when the present invention is applied to a single-lens reflex camera, and Figure 1 (B)
is an explanatory diagram of a part of the same figure (A). Figure 2 is Figure 1 (A
3 (A) is a schematic diagram of a part of the automatic focus detection device of
) is an explanatory diagram of the principle of the line of sight detection method in the present invention, and FIG. 3 (
B) is an explanatory diagram of the output intensity from the image sensor of FIG. 3(A).

In the figure, reference numeral 1 denotes an eyepiece lens, and inside thereof a dichroic mirror 1a that transmits visible light and reflects infrared light is provided obliquely, and also serves as an optical path splitter. 4 is a light-receiving lens, and 5 (5a, 5b, 5c) is an illumination means, which is composed of, for example, a light emitting diode that emits infrared light that is insensitive to the observer. 16 is an image sensor. The light receiving lens 4 and the image sensor 16 constitute one element of the light receiving means.

The image sensor 16 is composed of a two-dimensional array of photoelectric elements, and is located at a predetermined position with respect to the light receiving lens 4 and the eyepiece 1 (the general eye point position of a photographer who does not use glasses). It is placed so that it is conjugate with the vicinity of the pupil of the eye.

Reference numeral 9 denotes a line-of-sight arithmetic processing device, which has functions for calculating optical constants of the eyeballs, correcting the line-of-sight, and also operates an infrared light emitting diode 5.
It has control functions of a, 5b, and 5c. Each element 1, 4
, 5, and 16 constitute an eyeball line of sight detection means.

101 is a photographing lens, 102 is a quick return (QR) mirror, 104 is a focusing plate, 103 is a display element, 104 is a focusing plate, 105 is a condenser lens, 106 is a penta roof prism, 107 is a sub-mirror, and 108 is a multi-point This is a focus detection device that performs focus detection by selecting multiple areas within the photographic screen. The explanation of the multi-point focus detection device is not necessary for understanding the present invention, so a brief explanation will be provided.

That is, in this embodiment, as shown in FIG. 2, a field mask 110 is arranged near the planned image forming plane of the photographing lens 101 and has a plurality of slits each determining a distance measurement area. A lens member 111 functioning as a field lens is arranged close to each other, and a set 112 of re-imaging lenses and a set 113 of photoelectric element arrays corresponding to the number of slits are arranged in sequence. The slit 110, the field lens 111, the reimaging lens set 112, and the photoelectric element array set 113 each constitute a well-known focus detection system. Reference numeral 109 denotes a camera control device, which has functions such as driving a display element in the finder, performing focus detection calculations, and driving a lens.

In this embodiment, a part of the object light transmitted through the photographing lens 101 is reflected by the QR mirror 102 to form an object image near the focusing plate 104. Focus board 10
The subject light diffused by the diffusion surface of 4 is transferred to the condenser lens 1.
05, the light is guided to the eye point E via the pentagonal roof prism 106 and the eyepiece lens 1.

Here, the display element 103 is, for example, a two-layer type guest-host type liquid crystal element that does not use a polarizing plate, and also serves as display means in the viewfinder, which is an element of the optical constant detection means of the eyeball, as shown in FIG. As shown, there are indicators for detecting the optical constants of the eyeball (indicators 51 and 52 in the figure) within the viewfinder field of view.
, 53) are displayed.

[0033] Also, a part of the subject light that has passed through the photographic lens 101 passes through the QR mirror 102, and then passes through the sub-mirror 107.
The light is reflected and guided to the aforementioned multi-point focus detection device 108 arranged at the bottom of the camera body. Further, the photographing lens 101 is extended (or retracted) by a photographic lens driving device (not shown) based on the focus detection information of the position on the subject plane selected by the multi-point focus detection device 108 based on the signal from the camera control device 109. is performed, and focus adjustment is performed.

The line of sight detecting device according to this embodiment includes a line of sight detecting means constituted by members denoted by numbers 1, 4, 5, and 16, and a line of sight detection means that calculates the optical constants of the eyeball from the eyeball image and corrects the line of sight. It is composed of a line-of-sight calculation processing device 9 that performs calculations. Note that the line-of-sight calculation processing device 9 includes an optical constant calculation means, which is a component of the eyeball optical constant detection means, an eyeball optical constant storage means, and a line-of-sight correction calculation means.

The infrared light emitting diodes 5a and 5b are arranged symmetrically with respect to the XY plane in the figure. The infrared light emitting diodes 5b and 5c are arranged symmetrically with respect to the Z-X plane in the figure. Further, the distance between the infrared light emitting diodes 5a and 5b is set to be different from the distance between the infrared light emitting diodes 5b and 5c.

In the line of sight detection means, the infrared light emitted from the infrared light emitting diodes 5 (5a, 5b, 5c) is
The light enters the eyepiece 1 and is partially reflected by the dichroic mirror 1a, illuminating the observer's eyeball 201 located near the eye point E. Further, the infrared light reflected by the eyeball 201 is reflected by the dichroic mirror 1a and is reflected by the light receiving lens 4.
An image is formed on the image sensor 16 while converging. The observer's line of sight is calculated from these eyeball image data.

Next, the line of sight detection method will be explained using FIGS. 3(A) and 3(B).

In the figure, 5a and 5b are light sources such as light emitting diodes that emit infrared light that is insensitive to the observer, and each light source is arranged approximately symmetrically in the Z direction with respect to the optical axis A. The person's eyeballs are illuminated with divergence. The infrared light emitted from the light source 5b is partially reflected on the surface of the cornea 21. Here, the position in the X-axis direction of the corneal reflected image (virtual image) d formed by the infrared light reflected by the cornea 21 is the radius of curvature r of the cornea 21.
The distance Xd of the virtual image d from the surface of the cornea 21 is 1/Xd+1/s=2/r, where s is the distance between the infrared light emitting diode 5b and the cornea 21 in the X-axis direction.
‥‥‥‥The relational expression (6) is satisfied. A portion of the infrared light reflected on the surface of the cornea 21 is collected by the light receiving lens 4 and reimaged at a position d' on the image sensor 16. At this time, the imaging magnification β1 of the corneal reflection image d on the image sensor 16 is determined by the reference position (
For example, it is expressed as a function of the distance Xt from the exit surface of the eyepiece 1 in FIG. 1B to the cornea 21 of the observer's eyeball and the radius of curvature r of the cornea 21.

Similarly, infrared light emitted from the light source 5a is partially reflected by the surface of the cornea 21. Here, a corneal reflection image (virtual image) e formed by the infrared light reflected by the cornea 21
The position of the virtual image e in the X-axis direction depends on the radius of curvature r of the cornea 21, and the distance Xe of the virtual image e from the surface of the cornea 21 is approximately the same as the distance Xd. A part of the infrared light reflected on the surface of the cornea 21 is collected by the light receiving lens 4 and reimaged at a position e' on the image sensor 16. At this time, the imaging magnification β1 of the corneal reflected image e on the image sensor 16 is the distance Xt and the corneal 2
It is expressed as a function of the radius of curvature r of 1.

Generally, the Z coordinate of the midpoint of the corneal reflection images d and e does not match the Z coordinate Zo of the center of curvature O of the cornea 21, so it is necessary to correct this by δz(Xt, r).

Furthermore, the light beams from the ends a and b of the iris 23 pass through the light-receiving lens 4 to a position a' on the image sensor 16.
, b' are imaged at the ends a and b. When the rotation angle θ of the optical axis I of the eyeball with respect to the optical axis (optical axis A) of the light-receiving lens 4 is small, and if the Z coordinates of the ends a and b of the iris 23 are Za and Zb, then the center position c of the pupil 24 is The coordinate Zc is Zc≒(Z
It is expressed as a+Zb)/2.

At this time, the cornea 21 at the center position c of the pupil 24
The distance Xc in the X-axis direction from the surface of the pupil 24 in the X-axis direction is approximately equal to the anterior chamber depth t, and the imaging magnification β2 of the center position c of the pupil 24 on the image sensor 16 is expressed as a function of the distance Xt and the anterior chamber depth t.

Further, if the Z coordinates of the positions d and e of the corneal reflection image are Zd and Ze, and the distance between the center of curvature O of the cornea 21 and the center C of the pupil 24 is LOC (≒r-t), the eyeball light From equation (1), the rotation angle θ of axis A is LOC*sinθ≒Zc-((Zd+Ze)/2+Δz
(Xt, r)) substantially satisfies the relational expression (7). For this purpose, the line of sight arithmetic processing device 9 detects the position of each feature point (corneal reflection images d, e and iris ends a, b) projected onto a part of the image sensor 16 as shown in FIG. 3(B), This determines the rotation angle θ of the optical axis A of the eyeball. In this case, equation (7) is (rt)*sinθ≒((Za′+Z
b′)/2)/β2(Xt, t) −((Zd′+Z
e′)/2+Δz(Xt, r))/β1(Xt, r)(
8) can be rewritten as Furthermore, the rotation angle θ of the eyeball is θ
≒ arcsin {(Zc′/β2(Xt, t)−
(Zg′+Δz(Xt, r))/β1(Xt, r
))/(rt)}

It can be rewritten as (9). However, Zc'≒(Za'+Zb')/2 and Zg'≒(Zd'+Ze')/2. However, the optical axis of the photographer's eyeball and the visual axis do not coincide. For this reason, the horizontal rotation angle θ of the optical axis of the photographer's eyeball
The photographer's horizontal line of sight θH is obtained by calculating the angle correction δ between the optical axis of the eyeball and the visual axis. The horizontal line of sight θH of the photographer is θH=θ−δ ‥
...(10) is what I'm looking for. Further, the absolute value of the correction angle δ between the optical axis of the eyeball and the visual axis takes a different value depending on the photographer.

Furthermore, the position Zn on the focusing board 104 that the photographer is looking at is Zn≒m*θH ‥‥
It is sought as (11). However, m is a constant determined by the camera's finder system.

As described above, the corneal radius r of the photographer's eyeball,
The value of the anterior chamber depth t and the correction angle δ between the optical axis and the visual axis of the eyeball are determined in advance, and the software of the microcomputer of the line of sight calculation processing device 9 is calculated based on a correction calculation formula for the line of sight that includes individual differences in the eyeball optical system. The line of sight of the photographer and the point of interest on the focusing board 104 are determined.

In the line of sight detection device according to the present invention, the optical constants of the photographer's eyeball optical system are determined by the optical constant detection means as follows.

FIG. 4 shows an eyeball image on the surface of the image sensor 16, showing the infrared light emitting diodes 5a, 5b, and
5c is turned on, and three corneal reflection images d', e', f' as shown in FIG. 4 are formed on the image sensor 16.

Here, the infrared light-emitting diodes 5a and 5b are arranged symmetrically with respect to the X-Y plane in FIG. 1(B), and the infrared light-emitting diodes 5b and 5c are arranged symmetrically with respect to the Z-X plane in the figure. Further, the distance between the infrared light emitting diodes 5a and 5b is set to be different from the distance between the infrared light emitting diodes 5b and 5c. For this reason, the corneal reflection image d
The interval ΔZ between ' and e' and the interval ΔY between the corneal reflection images d' and f' are also different. The intervals ΔZ and ΔY between the corneal reflection images are determined by the distance Xt from the reference position of the line of sight detection optical system (for example, the exit surface of the eyepiece 1 in FIG. 1B) to the cornea 21 of the observer's eyeball and the radius of curvature of the cornea 21. Since it is a function of r,
Distance Xt and corneal radius r are Xt=a1(r)*ΔZ**2+a2(r)*ΔZ+a
3(r) (12)r=b1(Xt)*ΔY**
It is expressed as 2+b2(Xt)*ΔY+b3(Xt) (13).

Here, coefficients a1(r), a2(r), a3
(r), b1 (Xt), b2 (Xt), and b3 (Xt) are values determined by the configuration of the line of sight detection optical system. Detect the intervals ΔZ and ΔY of the corneal reflection images, and
) The distance Xt and the corneal radius r are calculated based on the formula.

The anterior chamber depth t of the photographer's eyeball is detected by having the photographer gaze at an index within the finder system of the camera.

When the photographer looks at the index n (Z coordinate Zn on the focus plate 104) in the camera's finder system shown in FIG. −δ)≒Zc′/β2(
Xt,t)-(Zg'+Δz(Xt,r))/β1(X
t, r)

It can be rewritten as (14). For photographers whose right eye is the one looking through the viewfinder system,
The rotation angle of the optical axis A of the photographer's eyeball is the optical axis of the line of sight detection means (
Indices arranged so as to be smaller than a) For example, the indicator 51 (Z coordinate Z1 on the focus plate 104) and the indicator 5
2 (Z coordinate Z2 on the focus board 104),
Equation (14) is (rt)*(Z1/m-δ)≒Zc1'/β2(Xt
1,t) −(Zg1′+Δz(Xt1,r))
/β1(Xt1,r) (15)(rt)*(
Z2/m-δ)≒Zc2'/β2(Xt2, t)
−(Zg2′+Δz(Xt2,r))/β1(Xt2
, r) (16). (15), (
After eliminating the correction angle δ between the optical axis and the visual axis of the eyeball from formula 16), calculate the anterior chamber depth t by substituting the coordinates of the feature points of the eyeball image detected when gazing at each index. There is. The anterior chamber depth t is determined, and from this the correction angle δ between the optical axis and the visual axis of the eyeball is calculated.

FIGS. 6 and 7 are flowcharts for detecting optical constants of the eyeball, which is line-of-sight correction data, and FIG. 8 is an external view of the front of the single-lens reflex camera. In FIG. 8, 31 is a mode selection button, and 32 is an electronic dial that also serves as an optical constant detection execution switch. 33 is a release switch. A method for detecting the optical constants of the eyeball will be explained based on FIGS. 6, 7, and 8.

After turning on the camera power (not shown) (#20
0), when the photographer selects the eyeball optical detection mode with the electronic dial 32 while pressing the mode selection button 31 (#
201) The optical constants of the eyeballs stored in the line-of-sight arithmetic processing device 9 are deleted (#202), and detection of the optical constants of the photographer's eyeballs is newly started.

[0054] In addition, the indicator 5 located at the center of the finder system shown in FIG.
1 starts blinking (#203). When the photographer recognizes that the index for detecting the optical constant of the eyeball is being displayed, he operates the front stage of the release switch 33 while gazing at the index 51 (#204). Based on the signal from the release switch 33, the line-of-sight arithmetic processing device 9 turns on the infrared light-emitting diodes 5a, 5b, and 5c for line-of-sight detection, and the infrared light-emitting diodes illuminate the photographer's eyeballs (#205).

The eyeball image data at that time is input to the line-of-sight arithmetic processing device 9 (#206), and the line-of-sight arithmetic processing device 9 determines whether the eyeball image data is valid (#208). Further, at the time when the eyeball image data is input to the line-of-sight arithmetic processing device 9, the blinking display of the indicator 51 in the finder system ends (#207), and at the same time, the infrared light emitting diodes 5a, 5b, and 5c are turned off.

By the way, in current cameras, functions such as automatic focus adjustment and photometry of the photographing lens are normally activated by operating the front stage of the release switch 33, but when the mode is set to the optical constant detection mode of the eyeball, these functions are activated. It doesn't matter if the function is disabled. When the line of sight arithmetic processing device 9 determines that a corneal reflection image or an iris image cannot be detected from the eyeball image data, the blinking display of the indicator 51 in the finder system is started again (#203).
.

On the other hand, when the feature points of the corneal reflection image and the iris image are detected from the eyeball image data (#209), the line-of-sight calculation processing unit 9 calculates the corneal radius r of the photographer's eyeball and the reference position of the line-of-sight detection optical system. The distance Xt from to the cornea 21 of the observer's eyeball is calculated based on equations (12) and (13) (#210).

[0058] Also, in the line of sight arithmetic processing device 9, the above (
9) Determine whether the photographer is looking into the finder system with the right eye or the left eye based on whether θ, which is calculated from the formula, is a positive or negative value. There is. At this time, a standard value is substituted for the anterior chamber depth t.

Furthermore, when the eye of the photographer looking into the finder system becomes clear in the line-of-sight arithmetic processing device 9, the rotation angle of the optical axis A of the photographer's eyeball is made smaller with respect to the optical axis A of the line-of-sight detecting means. An indicator is selected from the remaining two indicators.

Next, if the photographer's eye is the right eye, the index 52 in the finder system shown in FIG. 5 is selected and the index 52 starts blinking in response to a signal from the camera control device 109 (#212). . The photographer recognizes that an index for detecting the optical constant of the eyeball is being displayed within the viewfinder field of view, and operates the front stage of the release switch 33 while gazing at the index 52 (#213). Based on the signal from the release switch 33, the line of sight arithmetic processing device 9 turns on the infrared light emitting diodes 5a and 5b for line of sight detection, and the infrared light emitting diodes illuminate the photographer's eyeballs (#214).

The eyeball image data at that time is input to the line-of-sight calculation processing unit 9 (#215), and the line-of-sight calculation processing unit 9 determines whether the eyeball image data is valid (#217).

Furthermore, at the time when the eyeball image data is input to the line of sight arithmetic processing device 9, the blinking display of the index 52 in the finder system ends (#216), and at the same time, the infrared light emitting diode 5
a and 5b are turned off. When the line of sight arithmetic processing unit 9 determines that a corneal reflection image or an iris image cannot be detected from the eyeball image data, the blinking display of the indicator 52 in the finder system is started again (#212).

On the other hand, when the characteristic points of the corneal reflection image and the iris image are detected from the eyeball image data (#218), the line of sight arithmetic processing unit 9 calculates the anterior chamber depth t and the correction angle between the optical axis and the visual axis of the eyeball. δ is calculated using equations (15) and (16) above.

The calculated optical constants of the eyeballs are stored in the visual line correction data storage means of the visual line arithmetic processing device 9 (#22
0). At this time, if the data of the photographer who detected the optical constants of the eyeballs is also stored in the line-of-sight calculation processing device 9 at the same time, once the optical constants of the eyeballs are detected for the camera used, it will not be necessary to do so again. It will be over.

When the optical constants of the eyeballs are stored in the line-of-sight calculation processing device 9, the indicators 51 and 52 in the viewfinder blink for a predetermined period of time to notify the photographer that the detection of the optical constants of the eyeballs has been completed. After that (#221), the mode shifts to the line-of-sight input mode (#222). In gaze input mode,
By using the previously calculated optical constants of the eyeballs, the photographer's line of sight is detected with high accuracy, and the information intended by the photographer,
For example, it becomes possible for a photographer to input the position of a subject to be photographed in focus into the camera based on the photographer's line of sight.

In this embodiment, an example was shown in which the eyeball optical constant detection mode was selected after the power was turned on, but if the photographer has already registered the eyeball optical constants in the camera, the eyeball optical constants It is also possible to immediately shift to the line-of-sight input mode without performing the detection.

When it is determined that the photographer's eye is the left eye, the indicator 53 in the finder system shown in FIG. 5 is selected, and the indicator 53 starts blinking in response to a signal from the camera control device 109. The subsequent operation is the same as when the indicator 52 is blinked.

FIG. 9(A) is a rear external view of a camera according to Embodiment 2 of the present invention, FIG. 9(B) is a viewfinder field view in Embodiment 2, and FIGS. 10 and 11 are flowcharts of Embodiment 2. be.

The line of sight detection device of this embodiment is similar to that shown in FIG. 35 in FIG. 9A is an optical constant detection switch for the eyeball, which also serves as an optical constant detection execution switch.

In this embodiment, FIG. 9(B) shows a display means in the viewfinder in which an indicator for detecting the optical constants of the eyeball, which is composed of a liquid crystal element with a backlight or a light emitting diode, is located outside the field of view of the viewfinder. Display and non-display are controlled by a camera control device (not shown).

The method for detecting the optical constants of the eyeball according to this embodiment will be explained below based on the flowcharts for detecting the optical constants of the eyeball shown in FIGS. 10 and 11.

When the photographer turns on the camera (not shown) and sets it to the gaze input mode (#230), the infrared light emitting diodes 5a and 5b for gaze detection are activated based on the signal from the gaze arithmetic processing device 9. The light turns on and starts illuminating the photographer's eyeballs. Further, when the photographer presses the eyeball optical constant detection switch 35 for a predetermined period of time (ΔT seconds), the mode shifts to the eyeball optical constant detection mode (#231). At this time, an infrared light emitting diode 5c for detecting the corneal radius r of the photographer's eyeball
lights up to generate a third corneal reflection image (#232). At the same time, if the optical constants of the eyeballs are registered in the line of sight arithmetic processing device 9, which is the optical constant storage means of the eyeballs, the optical constants of the eyeballs are deleted (#233).

Based on the signal from the camera control device, an indicator for detecting the optical constant of the eyeball located at the lower center in the horizontal direction outside the field of view of the finder lights up (#234). The photographer presses the eyeball optical constant detection switch 35 while gazing at the index (#235), and inputs the photographer's eyeball image data at that time into the line-of-sight arithmetic processing device 9 (#236). At the time when the eyeball image data is input to the line-of-sight calculation processing device 9, the indicators outside the viewfinder field of view are turned off (#237).

Next, the line of sight arithmetic processing device judges whether the eyeball image data is valid (#
238). When the line of sight arithmetic processing unit 9 determines that a corneal reflection image or an iris image cannot be detected from the eyeball image data, the indicator lights up again (#234) and retaking of the eyeball image data is started. At this time, the indicator that was once turned off by the photographer turns on again, allowing the photographer to recognize that the previous input of eyeball image data was incomplete.

On the other hand, if the eyeball image data is valid, characteristic points of the corneal reflection image and the iris image are detected from the eyeball image data (#23).
8) In the visual line arithmetic processing device 9 which also serves as an optical constant calculation means, the corneal radius r of the photographer's eyeball and the distance Xt from the reference position of the visual line detection optical system to the cornea 21 of the observer's eyeball are calculated as described in (12) above. Calculated based on formula (13) (#
239).

Furthermore, the anterior chamber depth t and the correction angle δ between the optical axis and the visual axis of the eyeball are calculated from the characteristic points of the corneal reflection image and the iris image (#230). The method for calculating the anterior chamber depth t and the correction angle δ between the optical axis and the visual axis of the eyeball is as follows.

[0077] The photographer selects the index 91 (coordinates on the focus plate 104) in the finder system of the camera shown in Fig. 9(B).
When gazing at Z, Y)), the relationship between the gazing point and each feature point of the eyeball is almost the same as the above equation (14), and (rt)*si
n(Z/m-δ)≒Zc'/β2(Xt, t)
−(Zg′+Δz(Xt, r))/β1(
Xt, r) (17) (r-t) * sin (Y/m)
≒Yc′/β2(Xt, t) −(
Yg′+Δy(Xt, r))/β1(Xt, r) (
18). However, (Zc', Yc') are the coordinates of the center of the pupil of the photographer's eyeball on the image sensor 16 surface (Zg', Yg') are the center coordinates of the two corneal reflection images on the image sensor 16 surface . Furthermore, since the rotation angle component in the vertical direction of the observer's line of sight coincides with the rotation angle component in the vertical direction of the optical axis of the eyeball, the angle correction term δ is not included in equation (18).

The anterior chamber depth t is calculated from equation (18), and the value of the anterior chamber depth t is substituted into equation (17) to calculate the correction angle δ between the optical axis and visual axis of the eyeball. Furthermore, the calculated optical constants of the eyeballs are stored in the line of sight arithmetic processing device (#241). At this time, if the data of the photographer who detected the optical constants of the eyeballs is also stored in the line-of-sight calculation processing device 9 at the same time, once the optical constants of the eyeballs are detected for the camera used, it will not be necessary to do so again. It will be over.

After storing the optical constants of the eyeballs in the line-of-sight arithmetic processing device 9 and blinking the indicator in the finder system for a predetermined period of time to notify the photographer that the detection of the optical constants of the eyeballs has been completed (# 242) The optical constant detection mode of the eyeball is canceled (#243). At the same time, the third infrared light emitting diode 5c is also turned off (#244), and the mode shifts to the line-of-sight input mode (#245). In the line-of-sight input mode, the photographer's line of sight is accurately detected by using the optical constants of the eyeballs calculated previously, and information intended by the photographer, such as the position of the subject that the photographer wants to focus on, is sent to the photographer for the shooting. input into the camera based on the person's line of sight.

In this embodiment, a camera is shown as an example of the optical device, but the present invention can be similarly applied to optical devices such as a microscope.

[0081]

According to the present invention, as described above, there are provided an optical constant detection means for detecting the optical constant of an observer's eyeball, and a line-of-sight correction calculation means for correcting a line-of-sight detection error due to individual differences in the optical constant of the eyeball. By configuring this, it is possible to achieve an optical device having a line-of-sight detection device that can automatically correct line-of-sight detection errors due to individual differences such as the size of eyeballs and can detect line-of-sight with high accuracy.

[Brief explanation of the drawing]

FIG. 1(A) is a schematic diagram of a main part of a first embodiment when the present invention is applied to a single-lens reflex camera, and FIG. 1(B) is a perspective view of a part of a main part of FIG. 1(A).

FIG. 2 is a perspective view of a part of the focus detection device of FIG. 1(A).

3(A) is an explanatory diagram of the line of sight detection principle, and (B) is an explanatory diagram of the output intensity of the image sensor in FIG. 3(A).

FIG. 4 is an explanatory diagram of an eyeball image on an image sensor.

[Figure 5] Viewfinder field of view.

[Fig. 6] Flowchart of optical constant detection of the eyeball.

[Fig. 7] Flowchart of optical constant detection of the eyeball.

FIG. 8 is a front external view of the single-lens reflex camera according to the first embodiment.

9(A) is a rear external view of a single-lens reflex camera according to a second embodiment, and FIG. 9(B) is a view within the viewfinder of the second embodiment.

FIG. 10 is a flowchart of optical constant detection of the eyeball in Example 2.

FIG. 11 is a flowchart of optical constant detection of the eyeball in Example 2.

FIG. 12 (A) is a schematic diagram of a conventional line of sight detection optical system;
B) is an output intensity diagram of the photoelectric element array of FIG. 11(A).

FIG. 13 is a schematic diagram of the main parts of a conventional single-lens reflex camera.

[Explanation of symbols]

1 Eyepiece 2
Half mirror 3 floodlight lens
4 Light receiving lens 5 Infrared light emitting diode 6 Image sensor 9
Gaze calculation processing device 21 Cornea 22
Sclera 23 Iris 24 Pupil
31 Mode selection button 32 Electronic dial
33 Release switch 35 Line of sight correction switch 101 Photographic lens 102
Flip-up mirror 103 display element
104 Focusing plate 105 Condenser lens 106 Penta roof prism 107 Sub-mirror 108
Multi-point focus detection device 109 Camera control device 110 Field of view mask 111
field lens

Claims (2)

[Claims] Claim 1: The line of sight of an observer looking into a finder system is detected by a line of sight detection means, and the optical constant of the observer's eyeball is detected by an optical constant detection means using the value obtained by the line of sight detection means. 1. An optical device having a line of sight detection device, characterized in that a line of sight detected by the line of sight detection means is corrected by a line of sight correction calculation means using a value obtained by the optical constant detection means. 2. The eyeball optical constant detection means comprises an optical constant detection execution switch, a viewfinder display means, an optical constant calculation means, and an eyeball optical constant storage means. An optical device with a line of sight detection device.