JPH06148511A - Line-of-sight detector and optical device with the same - Google Patents

Abstract

(57) [Abstract] [Purpose] A line-of-sight detection device and a line-of-sight detection device that can detect the line of sight of an observer with high accuracy by appropriately setting the imaging magnification of the line-of-sight detection optical system and the pixel pitch of the sensor. To get the optics that were there. [Structure] An eyeball of a subject is illuminated with a light beam from an illuminating unit, an eyeball image based on reflected light from the eyeball is formed on an area type sensor surface by a condenser lens, and an output signal from the sensor is obtained. When the line of sight of the eyeball is detected by the calculating means using, the condition of P / β <0.41 (mm) is satisfied, where β is the imaging magnification of the condenser lens and P is the pixel pitch of the sensor. Each element was set as follows.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a line-of-sight detection device and an optical device having the same, and more particularly to an observer (photographer) through a viewfinder system on an observation plane (focus plane) on which a subject image is formed by the photographing system. Person) observes the axis of the gazing point direction, the so-called line of sight (visual axis), using the reflected image of the eyeball obtained when the observer's eye spherical surface is illuminated, and performs various shooting operations. The present invention relates to such a visual axis detection device and an optical device including the same.

[0002]

2. Description of the Related Art Conventionally, various devices (for example, eye cameras) for detecting what position on an observation surface an observer observes, that is, for detecting a so-called line of sight (visual axis) have been proposed.

For example, in Japanese Unexamined Patent Publication No. 1-274736, a parallel light flux from a light source is projected onto the anterior segment of the eyeball of an observer, and a corneal reflection image by the reflected light from the cornea and an image forming position of the pupil are used. Seeking the visual axis.

FIG. 4 is a cross-sectional view of a main part of a single-lens reflex camera having a conventional visual axis detection device.

In the figure, reference numeral 1 is a taking lens. Two
Is a main mirror, which is obliquely installed in the photographing optical path or retreated according to the observation state of the subject image by the finder system and the photographing state of the subject image. Reference numeral 3 denotes a sub-mirror, which reflects the light flux transmitted through the main mirror 2 to a focus detection device 6 described below below the camera body. Reference numeral 4 is a shutter, and 5 is a photosensitive member, which comprises a silver salt film, a CCD or MOS type solid-state image pickup device, or an image pickup tube such as a vidicon.

Reference numeral 6 denotes a focus detection device, which includes a field lens 6a and reflection mirrors 6b and 6 arranged near the image plane.
c, a secondary imaging lens 6d, a diaphragm 6e, a line sensor 6f including a plurality of CCDs, and the like.

The focus detection device 6 is well known, for example, Japanese Patent Laid-Open No. 59-59.
The phase difference method proposed in Japanese Patent Application Laid-Open No. 107311 or Japanese Patent Application Laid-Open No. 59-107313 is used, and a plurality of areas within the observation screen (in the viewfinder field) are used as focus points, and the focus points are focused. It is configured to be detectable.

Reference numeral 7 is a focusing plate arranged on the planned image forming surface of the taking lens 1, and 8 is a penta roof prism for deflecting the finder optical path.

An optical splitter 9 having a dichroic mirror surface 9a and an eyepiece lens 11 are arranged behind the exit surface of the penta roof prism 8 and used for observing the focusing plate 7 by a photographer's eye 15. The dichroic mirror surface 9a transmits, for example, visible light and reflects infrared light.

Reference numeral 10 is a diaphragm, 12 is a condenser lens, and 14 is C.
An image sensor in which a photoelectric element array such as a CD is two-dimensionally arranged, and a photographer's eye 1 at a predetermined position with respect to the condenser lens 12.
5 is arranged so as to be conjugate with the vicinity of the pupil (iris). Reference numerals 13a and 13b (not shown) are infrared light emitting diodes which are illumination light sources and are arranged around the eyepiece lens 11.

Reference numeral 20 denotes a high-intensity superimposing LED that can be visually recognized even in a bright subject. The emitted light is reflected by the main mirror 2 via the projection prism 21 and is provided on the display portion of the focus plate 7. It is bent in the vertical direction by the minute prism array 7a, and the penta roof prism 8, the light splitter 9,
It reaches the photographer's eye 15 through the eyepiece lens 11.

The micro prism array 7a is formed in a frame shape at a plurality of positions (distance measuring points) corresponding to the focus detection area of the focusing plate 7, and is illuminated by the superimposing LEDs 20 corresponding thereto. Then, the micro prism shines in the field of view of the finder, thereby displaying the focus detection area (distance measuring point).

Reference numeral 22 denotes an aperture provided in the taking lens 1, and 23
Is a diaphragm driving device including a diaphragm driving circuit, and 106 is a focus adjusting circuit for controlling a lens driving member including a lens driving motor 24, a driving gear 26 and the like. Focus adjustment circuit 10
Reference numeral 6 drives the lens driving motor 24 by a predetermined amount based on the information on the lens driving amount from the camera side, and the photographing lens 1
Is moved to the in-focus position.

FIG. 5A is a schematic diagram of the eyeball image projected on the CCD 14 of FIG. 4, and FIG. 5B is an output intensity diagram of the CCD 14 on line AA.

In the figure, 17 is an iris, 18 is a pupil,
Reference numeral 19 (19a, 19b) is a corneal reflection image. When the observer's eye 15 is illuminated by the infrared light emitting diodes 13a, 13b, a part of the infrared light reflected by the corneal surface of the observer's eyeball causes a pair of corneal reflection images 19a, 19 having a high light intensity.
b is formed on the CCD 14.

Part of the infrared light transmitted through the cornea of the observer's eye 15 is reflected by the iris 17, and the light reaching the retina through the pupil 18 is hardly reflected. Therefore, a difference in light intensity occurs at the boundary between the iris 17 and the pupil 18 and the boundary portion is detected to calculate the center position of the pupil 18.

Where the observer is looking in the finder field of view of the camera can be obtained by detecting the rotation angle of the eyeball of the observer.

The rotation angle of the eyeball of the observer is calculated from the distance between the midpoint of the two corneal reflection images 19a and 19b and the center of the pupil 18, and this distance changes depending on the imaging magnification of the visual axis detecting optical system. Therefore, the detection accuracy of the rotation angle of the eyeball of the observer depends on the imaging magnification of the line-of-sight detection optical system.

In the Japanese Patent Laid-Open No. 1-274736, the imaging magnification of the visual axis detecting optical system is 1 or less.

[0020]

When the imaging magnification of the visual axis detection optical system at the time of visual axis detection is reduced, the amount of movement of the eyeball image formed on the area-type sensor surface with respect to the rotation of the eyeball is reduced, resulting in As a result, the amount of change in the distance between the midpoint of the corneal reflection image and the center of the pupil becomes small, and the visual axis detection accuracy decreases.

On the contrary, if the imaging magnification of the visual axis detecting optical system is increased, the visual axis detecting accuracy is increased, but the area of the light receiving portion of the area type sensor is increased, making the manufacturing difficult, and the cost of the sensor alone increases. There is a problem.

When the pixel pitch of the sensor is increased,
The movement amount between pixels of the eyeball image formed on the sensor surface with respect to the rotation of the eyeball becomes small, and the eye gaze detection accuracy decreases.

On the contrary, when the pixel pitch of the sensor is reduced, the line-of-sight detection accuracy is improved, but the number of pixels to be processed increases, and the line-of-sight detection time becomes long, which is not suitable for practical use.

In general, a finder system of a camera has a wide horizontal field of view at a normal position, and the line of sight of an observer easily moves in the horizontal direction. Therefore, it is necessary to increase the horizontal line detection accuracy of the line of sight detection device. However, the conventional line-of-sight detection device has not particularly considered such a situation.

According to the present invention, when detecting the visual line of the eyeball, the visual line can be detected with high accuracy by appropriately setting the image forming magnification of the visual line detecting optical system, the pixel pitch of the area type sensor and the like. An object is to provide a gaze detection device.

Further, according to the present invention, the line of sight can be detected with high accuracy by appropriately setting the aperture shape and area of the eyepiece lens when the photographer looks into the finder system, the pixel pitch and dimensions of the area type sensor, and the like. An object of the present invention is to provide an optical device having a line-of-sight detection device adapted to perform various shooting operations.

[0027]

The line-of-sight detection apparatus of the present invention comprises: (1-1) illuminating the eyeball of a subject with a light beam from an illumination means, and collecting an eyeball image based on the reflected light from the eyeball with a condenser lens. When an image is formed on an area-type sensor surface by means of an output signal from the sensor and the line-of-sight of the eye is detected by the calculating means using the output signal from the sensor, the image-forming magnification of the condenser lens is β, and the pixel pitch of the sensor is When P is set, each element is set so as to satisfy the condition of P / β <0.41 (mm).

Further, the optical device having the line-of-sight detecting device of the present invention is (1-2) illuminating the eyeball of the observer looking into the finder system with the light flux from the illuminating means, and the eyeball image based on the reflected light from the eyeball. Is imaged on the area type sensor surface by a condenser lens, and when the line of sight of the eye is detected by the calculating means using the output signal from the sensor, the projection area of the exit surface of the eyepiece lens of the finder system is S2 is characterized in that the condition that 0.5 ≦ S1 / S2 ≦ 1 is satisfied, where S1 is the projected area when the light receiving portion of the sensor is projected onto the exit surface of the eyepiece lens.

(1-3) The eyeball of an observer looking through the finder system is illuminated with a light beam from the illumination means, and an eyeball image based on the reflected light from the eyeball is formed on the area type sensor surface by a condenser lens. However, when the line of sight of the eyeball is detected by the calculating means using the output signal from the sensor, the opening of the finder system is formed in a substantially rectangular shape having a long side in the first direction and a short side in the second direction. The light receiving surface of the sensor is characterized by being composed of substantially rectangular pixels whose short sides are in the first direction and whose long sides are in the second direction.

[0030]

1 is a schematic view of an essential part of an optical system of a visual axis detecting device of the present invention, FIG. 2 is an enlarged explanatory view of a part of FIG. 1, and FIG. 3 is a schematic diagram of an optical device having the visual axis detecting device of the present invention. FIG.

FIG. 3 shows a case where the visual axis detection device of the present invention is applied to a single-lens reflex camera as an optical device. or,
The configuration of the optical system at this time is substantially the same as that in FIG. Therefore, in FIGS. 1 and 3, the same elements as those shown in FIG. 4 are designated by the same reference numerals.

In the present embodiment, when an observer looking into the viewfinder system of the camera pushes the front stage of the release switch (not shown) while looking at the subject to be photographed, the signal input circuit 104 detects the operation and acts as a calculation means. CPU 1
The input signal is transmitted to 00. When the CPU 100 detects that the previous operation of the release switch has been performed, the CPU 100 commands the visual axis detection circuit 101 to execute visual axis detection.

The line-of-sight detection circuit 101 is an infrared light-emitting diode (hereinafter referred to as iRED) 13a, 1 as an illumination means.
3b is turned on to illuminate the eyeball 15 of the observer. IRE
Image accumulation is performed in the CCD 14 in synchronization with the lighting time of D13a and 13b.

In FIG. 1, the observer eye 15 has an iRED1.
When illuminated by 3a and 13b, a part of the infrared light is emitted from the cornea 1
Reflected on the surface of 6 and eyeball image based on it (corneal reflection image)
19a and 19b form an image on the light receiving surface of the CCD 14 by a condenser lens 12 through an eyepiece lens 11, a light splitter 9, and a diaphragm 10 in that order. Here, the corneal reflection images 19a and 19b are iREDs 13a and 13d generated by reflection on the cornea 16.
It is a virtual image of b.

On the other hand, the infrared light transmitted through the cornea 16 is the iris 1
The image related to the iris reflected by the
1, after passing through the light splitter 9 and the diaphragm 10, an image is formed on the light receiving surface of the CCD 14 by the condenser lens 12.

Although the infrared light passing through the pupil reaches the retina, the infrared light passing through the pupil hardly returns to the CCD 14 because the retina reflects little light. In addition, a is an optical axis of the visual axis detection optical system, and a is an optical axis of the eyeball.

In general, the condenser lens 12 of the visual axis detection optical system
Is set so that the iris 17 of the observer eye 15 and the light-receiving surface of the CCD 14 are substantially conjugate when the observer eye 15 is at a predetermined position (generally, the pupil position of the finder system) from the eyepiece lens 11. ing.

The observer's eye 15 looking into the viewfinder system of the camera can freely move in the XYZ directions with respect to the eyepiece lens 11. However, if the observer's eye 15 deviates greatly from the position of the pupil of the viewfinder system. Since the image in the visual field is blurred, the shift of the observer eye 15 with respect to the optical axis (Z axis) of the eyepiece lens 11 is limited.

Therefore, the range of movement of the observer's eye 15 is shown in FIG.
As shown in, the effective light flux area of the eyepiece lens 11 (inside the solid line;
Focus on the area S1 within the range indicated by the dotted line for the area S2),
The area corresponds to approximately half the area.

Therefore, in this embodiment, assuming that the projected area of the light receiving portion of the CCD 14 onto the exit surface of the eyepiece 11 is S1, 0.5 ≦ S1 / S2 ≦ 1 (1) in the visual axis detection optical system. The imaging magnification of the optical system and the area of the light receiving portion of the CCD 14 are set so as to satisfy the above condition.

As a result, the eyeball image of the observer's eye 15 is CCD.
An image is properly formed on the 14 light receiving portions. Then, when the accumulation of the predetermined eyeball image is completed in the CCD 14, the visual axis detection circuit 101 amplifies the image signal of the CCD 14 and transmits it to the CPU 100.

In the CPU 100, the eyeball image signal is A
After the / D conversion, the feature extraction of the eyeball image is executed according to a predetermined algorithm. When the positions of the corneal reflection images 19a and 19b and the position of the center C of the pupil are obtained in a series of arithmetic processing, the rotation angle θ of the observer eye 15 is calculated based on the calculation formula described later.

Furthermore, the observation position in the finder system of the observer is calculated from the rotation angle θ of the observer eye 15.

The CPU 100 sends a focus detection start signal to the automatic focus detection circuit 103 in order to detect the focus of the taking lens 1 in the focus detection area close to the calculated observer's observation position. The automatic focus detection circuit 103 uses the subject signal in the predetermined focus detection area obtained from the focus detection device 6 as a CP.
Send to U100.

The CPU 100 calculates the focus adjustment state of the focus detection area near the observation position in the finder system of the observer, and sends the focus adjustment amount and direction for focusing the taking lens 1 to the focus adjustment circuit 106. . Focus adjustment circuit 10
Reference numeral 6 sends a drive signal to the drive motor 24 of the taking lens 1 to drive the taking lens 1 to the in-focus position.

When the CPU 100 determines that the photographic lens 1 is in focus, the CPU 100 sends a focus display signal to the display circuit 105 to execute the focus display. Focusing display is performed by turning on a predetermined superimposing LED 20.

The observer recognizes that the photographing lens 1 is focused on the subject he is gazing at, and pushes the rear stage of the release switch (not shown) in order to perform photographing, and the signal input circuit 104 outputs the release signal. It is transmitted to the CPU 100.

The CPU 100 extracts the photometric information from the photometric circuit 102 and determines the exposure value. The CPU 100 sends the determined aperture value to the aperture drive circuit 107, and at the same time, sends the shutter speed information to the shutter control circuit 108. When the main mirror 2 and the sub mirror 3 are retracted out of the photographing optical path, the shutter 4 opens and the film 5
Is exposed.

When the shutter 4 is closed and the exposure of the film 5 is completed, the CPU 100 sends a film winding signal to the motor control circuit 109 to wind the film 5.

C, which is an arithmetic circuit for calculating the line of sight
In the PU 100, when the positions of the corneal reflection images 19a and 19b and the position of the center C of the pupil are obtained, the rotation angle θ of the observer eye 15 is calculated as follows.

CCD of one set of corneal reflection images 19a, 19b
Let X1a and X1b be the positions in the X direction on the 14th surface and XC be the position of the center C of the pupil, and the X with respect to the Z axis of the observer eye 15.
Rotation angle θX

[0052]

[Equation 1] Satisfy the formula. Where β is the imaging magnification of the visual axis detection optical system,
OC is the distance between the center O of curvature of the cornea 16 and the center C of the pupil,
Pitch X is the pixel pitch of the CCD 14 in the X direction.

Now, the center C of the pupil and the corneal reflection images 19a, 1
When the equation (2) is differentiated with respect to δ with δ being the distance from the midpoint of 9b, the following equation is obtained.

[0054]

[Equation 2] In the case of a single-lens reflex camera, it is desirable that the finder screen can be divided into five or more regions in the horizontal direction (X direction) as the line-of-sight detection accuracy. In many cases, the horizontal field of view of a viewfinder system in a single-lens reflex camera is about 30 [deg]. For this reason, a line-of-sight detection accuracy smaller than about 0.11 [rad] is required.

Therefore, the center C of the pupil and the corneal reflection image 19
The software resolution of the interval δ between a and 19b is 1 [pix
el], it is necessary to satisfy dθX / dδ <0.11 (4).

Now, assuming that OC = 4.1 [mm] and θX = 0 [rad], Pitch X / β <0.41 [mm] (5) from the equations (3) and (4). ) Holds.

In this embodiment, the imaging magnification β and the pixel pitch Pitch X are set so as to satisfy the equation (5), and the line of sight is detected with high accuracy.

The pixel pitch of the CCD 14 in the X direction is set to Pi
If tch X = 0.02 [mm], the magnification β of the visual axis detection optical system may be set so as to satisfy β = 0.049. When the image forming magnification β of the line-of-sight detection optical system when the observer 15 is at a predetermined position from the eyepiece lens 11 is determined, the image forming magnification βO on the exit surface of the eyepiece lens 11 is also determined.

From the equation (1), the area SO of the light receiving portion of the CCD 14 is

[0060]

[Equation 3] May be configured to satisfy

In the present embodiment, the description has been made by using the visual axis detection accuracy in the X direction, but the calculation may be similarly performed when the visual axis detection accuracy in the Y direction is taken into consideration. Since the line-of-sight detection accuracy in the Y direction may be lower than that in the X direction, the pixel pitch in the Y direction of the CCD 14 may be set, for example.
Pitch Y should be set to 0.03 [mm]. As a result, the line-of-sight detection accuracy in the Y direction is 0.67 times that in the X direction.

As described above, in this embodiment, when the opening (finder field) of the finder system has a substantially rectangular shape with the X direction (first direction) as the long side and the Y direction (second direction) as the short side. The pixel pitch of the CCD 14 has a short side in the X direction (pitch P = 0.02 mm) and a long side in the Y direction (pitch P = 0.
03 mm).

Now, if the imaging magnification β of the visual axis detection optical system when the observer eye 15 is 20 mm away from the eyepiece lens 11 is set to be β = 0.09, the eyepiece lens 11 The imaging magnification on the exit surface is approximately βO = 0.14.

Assuming that the exit area of the eyepiece lens 11 is S2 = 16 * 10 [mm * mm], the area SO of the light receiving portion of the CCD 14 is 1.57 ≦ SO ≦ 3.14 [mm * mm] from the equation (6). ……………… (7) should be satisfied. For example, when the area of the light receiving portion of the CCD 14 is set to SO = 1.6 * 1 [mm * mm], the eyeball image can be detected with the number of pixels of 80 * 33.

In FIG. 2, the moving range of the observer's eye 15 is shown by a region (within a dotted line) S1 which is narrowed at an equal ratio in the vertical and horizontal directions with respect to the effective light beam region S2 of the eyepiece lens 11. By designing the illustrated eyepiece frame so that the observer's eye 15 does not easily move in the vertical direction with respect to the finder optical axis a, it is possible to limit the range of movement of the observer's eye in the vertical direction.

Although the visual axis detection optical system provided in the single-lens reflex camera has been described in the present embodiment, the same applies to a video camera having an electronic viewfinder, and the same applies to a screen (horizontal field of view) in the viewfinder system of the video camera. When the line-of-sight detection accuracy that can separate the area of about 18 [deg]) into three or more in the horizontal direction is required,
The line-of-sight detection optical system may be configured so as to similarly satisfy the relational expression shown by the expression (4).

[0067]

According to the present invention, when the visual axis of the eyeball is detected as described above, the visual axis can be determined by appropriately setting the imaging magnification of the visual axis detection optical system, the pixel pitch of the area type sensor, and the like. It is possible to achieve a line-of-sight detection device that can detect with high accuracy.

Further, the line of sight can be detected with high precision by appropriately setting the aperture shape and area of the eyepiece lens when the photographer looks into the finder system, the pixel pitch and dimensions of the area type sensor, and the like. An optical device having a line-of-sight detection device adapted to perform a photographing operation can be achieved.

[Brief description of drawings]

FIG. 1 is a schematic view of a main part of an optical system of a visual line detection device of the present invention.

FIG. 2 is an enlarged explanatory view of a part of FIG.

FIG. 3 is a block diagram of a main part of an optical device having a line-of-sight detection device of the present invention.

FIG. 4 is a schematic view of a main part of a single-lens reflex camera having a conventional visual line detection device.

FIG. 5 is an explanatory diagram of a part of FIG.

[Explanation of symbols]

 1 Photographic Lens 4 Shutter 5 Photosensitive Surface 6 Focus Detection Device 7 Focus Plate 8 Penta-Dach Prism 9 Optical Divider 10 Aperture 11 Eyepiece 12 Condenser Lens 13a, 13b Illumination Means 14 CCD (Sensor) 15 Observer Eye 16 Corneal 17 Iris 19a , 19b Corneal reflection image

Front page continuation (51) Int.Cl. ⁵ Identification code Office reference number FI technical display location G03B 13/02 7139-2K 7316-2K G03B 3/00 A

Claims (3)

[Claims] 1. An eyeball of a subject is illuminated with a light beam from an illuminating means, an eyeball image based on reflected light from the eyeball is formed on an area type sensor surface by a condenser lens, and an output from the sensor is provided. When the line-of-sight of the eyeball is detected by the calculating means using the signal, when the imaging magnification of the condenser lens is β and the pixel pitch of the sensor is P, P / β <0.41 (mm) A line-of-sight detection device characterized in that each element is set so as to be satisfied. 2. An eyeball of an observer looking through a finder system is illuminated with a light beam from an illumination means, and an eyeball image based on reflected light from the eyeball is formed on a sensor surface of an area type by a condenser lens, When detecting the line of sight of the eyeball using the output signal from the sensor, the projection area of the exit surface of the eyepiece of the finder system is S2, and the light receiving portion of the sensor is projected onto the exit surface of the eyepiece lens. An optical device having a line-of-sight detection device, which satisfies a condition of 0.5 ≦ S1 / S2 ≦ 1 where S1 is a projected area at that time. 3. An eyeball of an observer looking through a finder system is illuminated with a light beam from an illumination means, and an eyeball image based on reflected light from the eyeball is formed on a sensor surface of an area type by a condenser lens. When detecting the line of sight of the eyeball by the calculating means using the output signal from the sensor, the opening of the finder system is formed in a substantially rectangular shape having a long side in the first direction and a short side in the second direction. The optical device having the line-of-sight detection device, wherein the light-receiving surface of is composed of substantially rectangular pixels whose short sides are in the first direction and whose long sides are in the second direction.