JP2016087291A - Pupil / gaze measuring device and illumination system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a pupil and visual line measurement device capable of measuring at least either a pupil diameter or a visual line direction of a person to be measured with glasses with a high degree of precision, and an illumination system using it.SOLUTION: A pupil and visual line measurement device 8 includes an eyeball irradiation part 1 for irradiating light to eyeballs 11 of a person to be measured 10 wearing glasses 5, an optical system 2a through which reflected light from the eyeballs 11 passes, an optical system control part 2c for controlling the optical system 2a so that pupils 12 of the eyeballs 11 are in focus, but lenses of the glasses 5 are not in focus, an image pickup device 2b for receiving the reflected light passing through the optical system 2a and acquiring image data on the eyeballs 11, and an image processing part 3a for calculating at least either a pupil diameter r or a visual line direction d of the person to be measured 10 based on the image data on the eyeballs 11 acquired by the image pickup device 2b.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a pupil / line-of-sight measurement device that measures at least one of a line-of-sight direction and a pupil diameter, and an illumination system using the same.

  Conventionally, when the line-of-sight direction is measured by image processing, the pupil center, the Purkinje image (the reflection image on the cornea), and both are generally used. In particular, when it is necessary to measure the line-of-sight direction with high accuracy, both the pupil center and the Purkinje image are used.

  A technique related to this is disclosed in the following Patent Document 1. According to the line-of-sight measurement method disclosed in Patent Document 1, information on the positional relationship between the pupil center and the Purkinje image and information on the shape of the pupil and the Purkinje image are processed by a computer. Thereby, even if a to-be-measured person wears spectacles, a gaze direction can be measured.

JP 2006-95008 A

  However, according to the technique disclosed in Patent Document 1 above, when measuring the line-of-sight direction of a person wearing glasses, the image of the light reflected on the lens of the eyeglasses has a pupil diameter or line-of-sight direction. Measurements may be adversely affected. More specifically, when the disturbance light image reflected on the surface of the spectacle lens overlaps the pupil or Purkinje image, it may be difficult to accurately recognize the pupil center or Purkinje image. In that case, there arises a problem that the line-of-sight direction of the measurement subject wearing glasses cannot be measured with high accuracy. This problem also occurs with respect to the pupil diameter. Therefore, according to the technique disclosed in Patent Document 1, neither the pupil diameter nor the line-of-sight direction of the measurement subject wearing glasses can be measured with high accuracy.

  The present invention has been made in view of the above problems. An object of the present invention is to provide a pupil / line-of-sight measuring device capable of measuring at least one of the pupil diameter and the line-of-sight direction of a measurement subject wearing glasses with high accuracy and an illumination system using the same. is there.

  A pupil / line-of-sight measurement device according to a first aspect of the present invention includes an eyeball irradiation unit that irradiates light to the eyeball of a measurement subject wearing glasses, an optical system through which reflected light reflected by the eyeball passes, An imaging element that receives the reflected light that has passed through the optical system and acquires image data of the eyeball, and a pupil diameter and a line-of-sight direction of the subject based on the image data of the eyeball acquired by the imaging element Of the optical system and the imaging device so that the image processing unit for calculating at least one of the image processing unit and the pupil of the eyeball is in focus but the lens of the eyeglass is not in focus. And an optical system controller that controls at least one of them.

  A pupil / line-of-sight measurement device according to a second aspect of the present invention includes an eyeball irradiation unit that irradiates light to the eyeball of a measurement subject wearing glasses, an optical system through which reflected light reflected by the eyeball passes, An imaging element that receives the reflected light that has passed through the optical system and acquires image data of the eyeball, and a pupil diameter and a line-of-sight direction of the subject based on the image data of the eyeball acquired by the imaging element An image processing unit that calculates at least one of the image processing unit, a display unit that displays the image of the eyeball based on the image data of the eyeball acquired by the imaging device, and a measurer that displays the image on the display unit While viewing the image of the eyeball being in focus, the pupil of the eyeball is in focus but at least one of the optical system and the image sensor so that the eyeglass lens is not in focus. An operation section or may control one, and a.

  The pupil / line-of-sight measurement device according to the third aspect of the present invention includes a position fixing unit that fixes the position of the face of the measurement subject in a state of wearing glasses, a position relative to the position fixing unit, and the position fixing unit. An eyeball irradiating unit whose position and posture are set so as to irradiate light to the eyeball of the measurement subject whose position of the face is fixed, an optical system through which reflected light reflected by the eyeball passes, and the optical Based on the image sensor that receives the reflected light that has passed through the system and acquires image data of the eyeball, and the eyeball image data acquired by the image sensor, out of the pupil diameter and line-of-sight direction of the subject An image processing unit that calculates at least one of the image processing unit, and the optical system and the imaging element are focused on the pupil of the eyeball of the measurement subject whose position of the face is fixed by the position fixing unit. Suit It is but are set so that out of focus the lens of the glasses.

  An illumination system of the present invention includes the pupil / line-of-sight measurement device according to the first to third aspects described above and a lighting device that irradiates light to an environment where the measurement subject exists, and the pupil / line-of-sight measurement device Controls the illumination mode of the luminaire based on at least one of the pupil diameter and the line-of-sight direction calculated by the image processing unit.

  According to the present invention, it is possible to measure with high accuracy at least one of the pupil diameter and the line-of-sight direction of a measurement subject wearing glasses.

It is a figure which shows schematic structure of the pupil and eyes | visual_axis measuring apparatus and lighting fixture in the illumination system of Embodiment 1 of this invention. It is a figure for demonstrating the state in which the disturbance light reflected by the lens of spectacles was contained in image data, when the image data of an eyeball was acquired in the pupil and gaze measuring device of an embodiment of the invention. For explaining the state in which the image of disturbance light reflected by the eyeglass lens included in the image data is blurred after the eyeball image data is acquired in the pupil / line-of-sight measurement device according to the embodiment of the present invention. FIG. It is a figure which shows an example of the luminance distribution acquired along the predetermined | prescribed virtual line of the image data of the eyeball acquired by the pupil and eyes | visual_axis measuring apparatus of embodiment of this invention. It is a figure for demonstrating the state in which only the image of the pupil and the image of the Purkinje image were left among the image data of the eyeball acquired by the pupil and visual line measuring device of embodiment of this invention. It is a figure which shows the other example of the luminance distribution acquired along the predetermined | prescribed virtual line of the image data of the eyeball acquired by the pupil and eyes | visual_axis measuring apparatus of embodiment of this invention. It is a flowchart for demonstrating the pupil and eyes | visual_axis measurement process in the pupil and eyes | visual_axis measuring apparatus of embodiment of this invention. FIG. 8A is a diagram showing the relationship between the size of the diaphragm and the depth of field in the optical system, and FIG. 8A shows the depth of field when the diaphragm is relatively small (F value is relatively large). FIG. 8B illustrates that the depth of field is relatively shallow when the aperture is relatively large (the F value is relatively small). It is a figure for doing. FIG. 9A is a diagram showing the relationship between the focal length and the depth of field in the optical system, and FIG. 9A shows the relative depth of field when the aperture diameter is the same and the focal length is relatively large. FIG. 9 (b) illustrates that the depth of field is relatively shallow when the aperture diameter is the same and the focal length is relatively small. FIG. It is a flowchart for demonstrating the pupil diameter and gaze direction calculation process in the pupil and gaze measurement apparatus of embodiment of this invention. It is a figure for demonstrating the method of calculating a pupil diameter and a gaze direction using the partial image of a pupil, and the partial image of a Purkinje image. It is a flowchart for demonstrating the lighting fixture control process in the pupil and eyes | visual_axis measuring apparatus of embodiment of this invention. It is a figure which shows the 1st example of the usage pattern of the illumination system of embodiment of this invention. It is a figure which shows the 2nd example of the usage pattern of the illumination system of embodiment of this invention. It is a figure which shows the 3rd example of the usage pattern of the illumination system of embodiment of this invention. It is a figure which shows the 4th example of the usage pattern of the illumination system of embodiment of this invention. It is a figure which shows schematic structure of the pupil and eyes | visual_axis measuring apparatus of Embodiment 2 of this invention. It is a figure which shows schematic structure of an example of the pupil and eyes | visual_axis measuring apparatus of Embodiment 3 of this invention. It is a figure which shows schematic structure of the other example of the pupil and eyes | visual_axis measuring apparatus of Embodiment 3 of this invention.

  Hereinafter, a pupil / line-of-sight measurement device and an illumination system according to an embodiment of the present invention will be described in detail with reference to the drawings. In the following plurality of embodiments, parts having the same reference numerals have the same configuration unless otherwise specified, even if there is a difference in shape on the drawings. , And exhibit the same function.

(Embodiment 1)
The illumination system 20 including the pupil / line-of-sight measuring device 8 according to Embodiment 1 of the present invention will be described with reference to FIG.

  The illumination system 20 according to the present embodiment includes an illumination fixture 50 that irradiates light to an environment where the pupil / line-of-sight measurement device 8 and the measurement subject 10 exist. In the present embodiment, the pupil / gaze measuring device 8 controls the illumination mode of the luminaire 50 based on at least one of the calculated pupil diameter r and the gaze direction d. That is, the illumination system 20 can change the mode of illumination by the luminaire 50 based on the state of the measurement subject 10 estimated from at least one of the pupil diameter r and the line-of-sight direction d. As a lighting mode of the lighting fixture 50 controlled by the pupil / line-of-sight measuring device 8, brightness, color temperature, light distribution, and the like are conceivable. However, the change of the lighting mode of the lighting fixture 50 controlled by the pupil / line-of-sight measurement device 8 is not limited as long as the lighting state of the lighting fixture 50 is changed so that some lighting characteristics are different. Also good.

  The pupil / line-of-sight measurement device 8 of the present embodiment is configured so that the face 15 of the measurement subject 10 wearing the glasses 5, particularly the chin, is fixed to the contact portion 4 of the position fixing unit 64. The image data of the eyeball 11 is acquired. However, the pupil / line-of-sight measurement device 8 of the present embodiment can automatically detect the eyeball 11 and adjust the focus, so that the position fixing unit 64 is not an essential component. This point will be described later.

  In the present embodiment, the acquired image data of the eyeball 11 includes an image of the skin around the eyeball 11, an image of the white eye, an image of the iris 13, and an image of the pupil 12. However, in addition to the image data of the eyeball 11, an image of the frame of the glasses 5 is also acquired. Although the image of the lens of the glasses 5 is basically not acquired, only the disturbance light O reflected on the lens of the glasses 5 may be included in the image data.

  In FIG. 1, left and right images are distinguished from each other as left glasses 5L and right glasses 5R, left eyeball 11L and right eyeball 11R, left iris 12L and right iris 12R, left eyeball 11L and right eyeball 11R. The left and right images may be acquired simultaneously as one image, but each of the left and right images may be acquired as two separate images. When each of the left and right eyeballs 11 is separately imaged, the pupil / line-of-sight measurement device 8 for the left eyeball 11L and the pupil / line-of-sight measurement device 8 for the right eyeball 11R are provided separately. preferable. In this case, the lighting mode of the luminaire 50 is controlled using the average values of the pupil diameter r and the gaze direction d of the left and right eyeballs calculated by the left and right pupil / gaze measuring devices 8 respectively. In the following description, however, the left and right eyeball images are not distinguished in order to simplify the description.

  The pupil / line-of-sight measuring device 8 according to the present embodiment includes an eyeball irradiation unit 1, an imaging unit 2, and a control unit 3. The control unit 3 causes the eyeball irradiation unit 1 to irradiate light toward the measurement subject 10 and creates image data based on the reflected light from the measurement target 10 received by the imaging unit 2. Further, the control unit 3 calculates the pupil diameter r and the line-of-sight direction d based on the created image data.

  The eyeball irradiation unit 1 irradiates light to the eyeball 11 of the measurement subject 10 in the state of wearing the glasses 5. The light irradiated by the eyeball irradiation unit 1 may be visible light as long as it has a small effect on the person to be measured 10, but is preferably infrared light other than visible light and having a small adverse effect on the human body. Furthermore, from the viewpoints of both the ease of image acquisition by the image sensor 2b and the small adverse effect on the human body, the light emitted by the eyeball irradiation unit 1 is more preferably a near infrared ray.

  The imaging unit 2 includes an optical system 2a, an imaging element 2b, and an optical system control unit 2c. The reflected light reflected by the eyeball 11 passes through the optical system 2a. The imaging element 2b receives the reflected light that has passed through the optical system 2a and acquires image data of the eyeball 11.

  The optical system 2a has a mechanism that realizes three functions of pan, tilt, and zoom. The optical system control unit 2c has a mechanism for realizing the above-described three functions of the optical system 2a so that the image data of the eyeball 11 is automatically acquired based on the image data captured by the image sensor 2b. Can be controlled. Therefore, according to the pupil / line-of-sight measurement device 8 of the present embodiment, the image data of the eyeball 11 is acquired without operating the pupil / line-of-sight measurement device 8 by the measurer.

  Further, the optical system 2a has a mechanism for realizing an autofocus function for automatically focusing on the pupil 12. The optical system control unit 2c controls a mechanism that realizes the autofocus function of the optical system 2a so that the pupil 12 is automatically focused based on the image data of the eyeball 11 captured by the imaging device 2b. . That is, according to the pupil / line-of-sight measuring device 8 of the present embodiment, the eyeball 11 is automatically focused on the pupil 12 and focused on the pupil 12 without any operation of the pupil / line-of-sight measuring device 8 by the measurer. Are automatically acquired by the image sensor 2b. However, when the Purkinje image P is used, the optical system control unit 2c controls the optical system 2a so that the cornea of the eyeball 11 in addition to the pupil 12 is in focus.

  The pupil / line-of-sight measurement device 8 includes a position / posture changing mechanism 120 that changes the postures and positions of the eyeball irradiation unit 1 and the imaging unit 2. Therefore, the optical system control unit 2c can change the postures and positions of the eyeball irradiation unit 1 and the image sensor 2b based on the image data of the eyeball 11 captured by the image sensor 2b. Thereby, the reflected light from the body part around the glasses 5 or the eyeball 11 of the light irradiated by the light irradiation unit does not overlap the pupil 12. Further, the optical system control unit 2c can change the postures and positions of the eyeball irradiation unit 1 and the imaging device 2b so that the shadow of the body part around the glasses 5 or the eyeball 11 does not overlap the pupil 12. . That is, according to the pupil / line-of-sight measurement device 8 of the present embodiment, the image data of the eyeball 11 in a state in which the adverse effects due to the body parts around the glasses 5 or the eyeball 11 are eliminated without the operation of the measurer are automatically performed. To be acquired. The position / orientation changing mechanism 120 may change the attitude and position of one of the eyeball irradiation unit 1 and the imaging device unit 2.

  The optical system 2a of the present embodiment is a lens system called a telephoto lens that can form a state where the depth of field X is extremely shallow. When the optical system 2a and the image sensor 2b of the present embodiment magnify the periphery of the pupil 12, the depth of field X is larger than the distance between the cornea of the eyeball 11 and the lens of the glasses 5. It is configured to be a small value. Therefore, the depth of field X can be changed to a value smaller than the distance between the cornea and the lens of the glasses 5 in a state where the cornea of the eyeball 11 and the pupil 12 are in focus. Therefore, in the image of the eyeball 11 acquired by the image sensor 2b, the image of the disturbance light O reflected on the lens of the glasses 5 can be blurred.

  The focus may be optimally adjusted to any position of the iris 13, the pupil 12, the cornea, and the vicinity thereof, but at least the pupil 12 needs to be focused. Since the pupil 12 is a hole surrounded by the iris 13, in this specification, when the pupil 12 is in focus, the iris 13 is also in focus. In addition, when the Purkinje image P is used to calculate the line-of-sight direction d, it is necessary to focus on the cornea in addition to the pupil 12 and the iris 13. Therefore, when the Purkinje image P is used, the depth of field X needs to be smaller than the distance between the cornea of the eyeball 11 and the lens of the glasses 5. However, when the Purkinje image P is not used, the depth of field X may be a value smaller than the distance between the pupil 12 and the lens of the glasses 5.

  The optical system control unit 2c according to the present embodiment focuses on the lens of the glasses 5 in a state where the pupil 12 (or the pupil and the cornea) is in focus in addition to the general imaging function described above. The optical system 2a and the image sensor 2b can be controlled so as not to be present. Specifically, the optical system control unit 2c changes at least one of the stop and the focal length of the optical system 2a in a state where the pupil 12 (or the pupil and the cornea) is in focus. Thereby, the optical system control unit 2 c sets the depth of field X to a value smaller than the distance between the pupil 12 (or the cornea) and the lens of the glasses 5. Therefore, the imaging device 2b can acquire image data of the eyeball 11 in a state where the light reflected from the lens of the eyeglass 5 of the light irradiated from the eyeball irradiation unit 1 is blurred.

  In the present embodiment, the optical system control unit 2c once focuses on the pupil 12 in a state where the depth of field X is larger than the distance between the pupil 12 (or cornea) and the lens of the glasses 5. . Thereafter, the optical system control unit 2c changes the object field while changing at least one of the aperture and the focal length of the optical system 2a so that the pupil 12 (or the pupil and the cornea) is kept in focus. The depth X is made smaller than the distance between the pupil 12 and the lens of the glasses 5. Therefore, it is possible to efficiently form a state where the pupil 12 is in focus but the eyeglass 5 is not in focus. At this time, if it is necessary that both the cornea of the eyeball 11 and the pupil 12 are in focus, the depth of field X is preferably smaller than the distance between the cornea and the lens of the glasses 5. .

  In the present embodiment, a value of the depth of field X smaller than a predetermined distance between the pupil 12 (or the cornea) and the lens of the glasses 5 is stored. In addition, the optical system control unit 2c reduces the depth of field X until it matches the stored value in a state where the pupil 12 (or cornea and pupil) is in focus. Information for controlling the optical system 2a is incorporated in the optical system control unit 2c in advance.

  In the present embodiment, the depth of field X is made smaller than the distance between the pupil 12 (or the cornea) and the lens of the glasses 5 in order to blur the disturbance light O more reliably. However, in the state where the pupil 12 (or the pupil and the cornea) is in focus by shifting the boundary of the in-focus range from the glasses 5 side to the pupil 12 side without changing the depth of field X. A state in which the lens of the glasses 5 is out of focus may be formed. In this case, the depth of field X may be larger than the distance between the pupil 12 (or the cornea) and the lens of the glasses 5.

  The control unit 3 includes a start switch 9, an image processing unit 3a, and a display / illumination control unit 3b. The activation switch 9 activates the pupil / line-of-sight measuring device 8, and when the activation switch 9 is turned on, the eyeball irradiation unit 1 emits light. The image processing unit 3a processes the image of the eyeball 11 acquired by the imaging unit 2, and calculates the pupil diameter r and the line-of-sight direction d. The display / illumination control unit 3b controls the illumination mode of the luminaire 50, which will be described later, based on the calculated pupil diameter r and line-of-sight direction d.

  The display / illumination control unit 3 b may control the display unit 7 that displays the image data of the eyeball 11. In this case, the display / illumination control unit 3 b can display the image of the eyeball 11, the pupil diameter r and the line-of-sight direction d of the eyeball 11 on the display unit 7. However, since the pupil / line-of-sight measurement device 8 of the present embodiment is used in the illumination system 20 that does not require a measurer, the display unit 7 that displays the image data of the eyeball 11 may not be controlled. . That is, the display / lighting control unit 3b may control only the lighting mode of the lighting fixture 50 in the present embodiment.

  Next, a method of acquiring the partial image data of the pupil 12 and the partial image data of the Purkinje image P with high accuracy by blurring the disturbance light O reflected on the lens of the glasses 5 with reference to FIGS. Will be explained. 2 to 6 show image images of the eyeball 11 and the glasses 5 acquired by the image sensor 2b. 4 to 6, the relationship between the brightness value of the reflected light incident on the image sensor 2 b and the position along the X axis is shown in a graph. The brightness of the reflected light incident on the image sensor 2b has a flat portion. The flat portion is considered to be light reflected by the same color portion of the eyeball 11. For example, since the pupil 12 is black, the flat portion having the lowest luminance is considered to be an image of the pupil 12. Further, the Purkinje image P is considered to be the tip of a pointed portion where the brightness of reflected light is highest. Therefore, as can be seen from FIGS. 4 to 6, which part of the graph is the Purkinje image P, the disturbance light O, the iris 13, or the pupil 12 depending on the luminance of the reflected light incident on the image sensor 2 b. Can be easily identified.

  4 to 6, brightness data along each of a plurality of virtual lines parallel to the X axis is acquired. Thereby, among the image data of the eyeball 11, partial image data of the pupil 12 and partial image data of the Purkinje image P can be created.

  In general, as shown in FIG. 2, when the disturbance light O reflected on the lens of the glasses 5 overlaps the image data of the iris 13, the outer periphery of the iris 13 may not be clear. Therefore, the pupil / line-of-sight measurement device 8 of the present embodiment blurs only the image data of the disturbance light O as shown in FIG. Thereby, it becomes possible to specify the outer periphery of the iris 13.

  As can be seen from FIGS. 4 and 6, the image data of the disturbance light O has almost the same luminance as the image of the Purkinje image P as indicated by a one-dot chain line if it is not blurred. However, since the pupil / line-of-sight measurement device 8 of the present embodiment blurs the image data of the disturbance light O, as can be seen from the hatched area of the symbol K shown in FIG. 4 and FIG. The brightness is reduced as compared with a non-blurred state. As a result, the Purkinje image P acquired by the image sensor 2b has a luminance value greater than the upper threshold value UT. On the other hand, the disturbance light O acquired by the imaging device 2b has a luminance value smaller than the upper threshold value UT. Therefore, the image data of the portion whose luminance is larger than the upper threshold value UT can be specified as the partial image data of the Purkinje image P.

  As can be seen from FIGS. 4 and 6, the luminance of the image of the pupil 12 has a luminance value smaller than the lower threshold LT. Therefore, the image data of the portion whose luminance is smaller than the lower threshold LT can be specified as the partial image data of the pupil 12. In this embodiment, as can be seen from FIGS. 4 and 6, the pupil diameter r is the maximum value of the width along the X-axis direction of the partial image data of the pupil 12.

  If only the pupil 12 and the Purkinje image P remain in the image shown in FIG. 4 and other portions have the same luminance as the iris 13, an image as shown in FIG. 5 is obtained. That is, the pupil / line-of-sight measuring device 8 of the present embodiment performs image processing for distinguishing only the pupil 12 and the Purkinje image P from other images, as shown in FIG.

  As described above, as shown in FIG. 6, when the disturbance light O reflected on the lens of the glasses 5 overlaps the image data of the pupil 12, the outer periphery of the pupil 12 may not be clear. In this case, the pupil diameter r may not be measured with high accuracy. Also, the center of the pupil may not be clear. In these cases, the line-of-sight direction d cannot be measured. Therefore, by only blurring the image of the disturbance light O, the image of the disturbance light O in the portion blurred on the image sensor 2b and the partial image of the pupil 12 are distinguished. The image processing unit 3 a recognizes the image of the pupil 12 by excluding the blurred image that overlaps the image of the pupil 12. Therefore, the image processing unit 3a calculates the pupil diameter r and the line-of-sight direction d of the measurement subject 10 in a state where the adverse effect of the disturbance light O reflected on the lens of the glasses 5 on the image of the pupil 12 is eliminated. Can do. Therefore, the pupil diameter r and the line-of-sight direction d can be calculated with high accuracy.

  4 and 6, in order to clearly distinguish the partial image of the pupil 12 and the Purkinje image P from the disturbance light O, it is necessary to adjust the degree to which the luminance of the disturbance light O is reduced. Further, the values of the upper threshold value UT and the lower threshold value LT are set to values that can distinguish between the luminance of the partial image of the pupil 12 and the Purkinje image P and the luminance of the disturbance light O, respectively. Specifically, the hatched area indicated by the symbol K in the graphs of FIG. 4 and FIG. 6 does not straddle both the upper threshold value UT and the lower threshold value LT.

  Next, a control process executed in the pupil / gaze measurement apparatus 8 will be described with reference to FIGS.

  First, pupil / line-of-sight measurement processing will be described with reference to FIG.

  In step S <b> 1, the control unit 3 determines whether or not the activation switch 9 of the pupil / line-of-sight measurement device 8 is in the ON state. If it is determined in step S1 that the start switch 9 is not in the ON state, the control unit 3 stands by in that state. If it is determined in step S1 that the activation switch 9 is in the ON state, the control unit 3 causes the eyeball irradiation unit 1 to irradiate near infrared light. Thereby, the image sensor 2b receives the reflected light of the near infrared light. Thereafter, the image pickup unit 2 searches for the eyeball 11 of the person to be measured 10 using the pan / tilt / zoom functions of the optical system 2a. At this time, the optical system control unit 2c performs control to change the direction of the optical axis of the optical system 2a in step S3, and in step S4, the image of the eyeball model stored in advance is captured. The image captured by the element 2b is compared.

  In step S4, the optical system control unit 2c does not have a portion in which the image of the eyeball model stored in advance and the image of the actual eyeball 11 captured by the image sensor 2b coincide in shape. There is. In this case, the optical system control unit 2c repeats the processing from step S1 to step S4. On the other hand, in step S4, there may be a case where the image of the eyeball model stored in advance in advance and the image of the actual eyeball 11 captured by the image sensor 2b have a portion that matches in shape. In this case, the optical system control unit 2c assumes that the eyeball 11 of the person to be measured 10 has been detected, and executes the process of step S5.

  In step S5, the optical system control unit 2c controls the optical system 2a so as to focus on the pupil 12. A so-called autofocus function is used. Specifically, by moving the lens position of the optical system 2a back and forth along the optical axis, the luminance of the partial image of the pupil 12 acquired by the imaging device 2b becomes the highest state or a state close thereto. If so, the movement of the lens of the optical system 2a is stopped. In step S6, the control unit 3 determines whether or not the pupil 12 is in focus in the image acquired by the image sensor 2b. That is, it is determined whether or not the luminance of the partial image of the pupil 12 is higher than a predetermined value.

  In the image acquired by the image sensor 2b, the control unit 3 repeats the processes of Steps S1 to S6 when the pupil 12 is not in focus. On the other hand, if the pupil 12 (or the pupil and the cornea) is in focus in the image acquired by the image sensor 2b, the control unit 3 executes the process of step S7. In step S <b> 7, the control unit 3 reflects the reflected light from the eyeglasses 5 of the light irradiated by the eyeball irradiation unit 1 or the body part around the eyeball 11 on the actual image of the eyeball 11 acquired by the imaging device 2 b. It is determined whether or not they overlap. In step S7, the control unit 3 determines whether the shadow of the body part around the eyeglasses 5 or the eyeball 11 overlaps the actual image of the eyeball 11 acquired by the image sensor 2b. This determination is performed by comparing the actual image of the eyeball 11 acquired by the image sensor 2b with the model image of the eyeball on which shadow or reflected light overlaps. Specifically, the control unit 3 determines that the shadow or the reflected light overlaps the image of the pupil 12 if there is a portion where the luminance greatly changes in the actual image of the pupil 12. That is, the color of the pupil 12 is close to a single color, and thus the luminance distribution should be almost flat. If not, the control unit 3 determines whether the reflected light or shadow is in the pupil 12 image. Is considered to have been affected.

  In step S7, when at least one of the reflected light and the shadow overlaps the actual image of the eyeball 11 acquired by the image sensor 2b, the control unit 3 executes the process of step S8. To do. In step S8, the control unit 3 sends a control command to the optical system control unit 2c based on the image data of the eyeball 11 imaged by the imaging device 2b. Thereby, the optical system control unit 2c is configured so that the image of the light reflected by the eyeglasses 5 or the body part around the eyeball 11 irradiated by the eyeball irradiation unit 1 overlaps the image of the pupil 12. And the attitude and position of the image pickup unit 2 are changed. In step S <b> 8, the optical system control unit 2 c determines the postures of the eyeball irradiation unit 1 and the imaging unit 2 so that the shadow image of the body part around the glasses 5 or the eyeball 11 does not overlap the image of the pupil 12. And change position. The processes in step S7 and step S8 are repeated until the reflected light and shadow images are not included in the image of the pupil 12 acquired by the image sensor 2b. Specifically, the postures and positions of the eyeball irradiation unit 1 and the imaging unit 2 are sequentially changed according to a plurality of predetermined posture and position change patterns.

  In step S7, when neither the reflected light image nor the shadow image of the body part overlaps the image of the pupil 12, the control unit 3 executes the process of step S9.

  In step S9, the control unit 3 sends a control command for blurring the disturbance light O of the image acquired by the image sensor 2b to the optical system control unit 2c. Thereby, the optical system control unit 2c controls the optical system 2a so that the depth of field X is set to a predetermined value or less, or the depth of field X is maintained to be a predetermined value or less. Specifically, for example, the control unit 3 causes the optical system control unit 2c to set the size of the stop of the optical system 2a so that the depth of field X is smaller than the distance between the pupil 12 and the lens of the glasses 5. Adjust the height. A method of changing the depth of field X will be described later.

  Next, in step S10, the control unit 3 is a partial image of the disturbance light O in the image of the actual eyeball 11 acquired by the imaging device 2b in a state where the pupil 12 (or the pupil and the cornea) is in focus. It is determined whether or not it is blurred. In step S10, the partial image of the disturbance light O in the actual image of the eyeball 11 acquired by the imaging device 2b may not be blurred while the pupil 12 (or the pupil and the cornea) is in focus. In this case, the control part 3 repeats the process of step S6-step S10.

  On the other hand, in step S10, when the disturbance light O of the image acquired by the image sensor 2b is blurred while the pupil 12 is in focus, the control unit 3 executes the process of step S13. . In step S11, the control unit 3 calculates the pupil diameter r and the line-of-sight direction d using the partial image data of the pupil 12 and the partial image data of the Purkinje image P acquired by the imaging device 2b. Thereafter, in step S12, the control unit 3 controls the lighting device 50 in the lighting mode using the calculated pupil diameter r and line-of-sight direction d.

  According to the above control method, the optical system control unit 2c controls the optical system 2a so that a state in which both the pupil 12 and the lens 5 are in focus is once formed. In other words, the optical system control unit 2c once images both the pupil 12 and the lens of the glasses 5 in a state where the depth of field X is larger than the distance between the pupil 12 (or the cornea) and the lens of the glasses 5. The element 2b is imaged. Thereafter, the optical system control unit 2c controls the optical system 2a so that the pupil 12 (or the pupil and the cornea) is in focus and the lens of the glasses 5 is not in focus. That is, the optical system control unit 2c makes the depth of field X smaller than the distance between the pupil 12 and the lens of the glasses 5 in a state where the pupil 12 (or cornea) is in focus. Therefore, from the beginning of the measurement, the pupil 12 is not focused with the depth of field X being extremely small. Therefore, the pupil 12 (or the pupil and the cornea) is in focus, but a state in which the eyeglass 5 is not in focus can be formed to some extent efficiently.

  As shown in FIGS. 8 and 9, the depth of field X is a range of distance on the subject side where the image acquired by the image sensor 2 b seems to be in focus. As shown in FIG. 8A, when the focal length f is the same and the aperture diameter D of the diaphragm 201 is relatively small (F value = f / D is relatively large), the depth of field X Is relatively deep. On the other hand, as shown in FIG. 8B, when the focal length f is the same and the aperture diameter D of the diaphragm 201 is relatively large (F value = f / D is relatively small), the object field The depth X is relatively shallow. Therefore, the depth of field X can be reduced by increasing the aperture diameter D of the diaphragm 201 while the focal length f is the same. Further, as shown in FIG. 9A, the depth of field X is relatively deep when the focal length f is relatively large. On the other hand, as shown in FIG. 9B, when the focal length f is relatively small, the depth of field X is relatively shallow. Therefore, the depth of field X can be reduced by reducing the focal length f. In the present embodiment, the optical system control unit 2c changes at least one of the aperture diameter D and the focal length f in the optical system 2a in order to change the depth of field X.

  In the present embodiment, the optical system control unit 2c controls only the optical system 2a to focus on the pupil 12 (or pupil and cornea) of the eyeball 11, but the lens of the glasses 5 is focused. Form a mismatched state. However, the optical system control unit 2c controls the image sensor 2b to form a state in which the pupil 12 (or pupil and cornea) of the eyeball 11 is in focus but the lens of the glasses 5 is not in focus. May be. When the image sensor 2b is moved along the optical axis, the focal length f can be changed, and therefore the depth of field X can be changed. That is, the optical system control unit 2c only needs to control at least one of the optical system 2a and the imaging device 2b in order to change the depth of field X. In any case, the optical system controller 2c only needs to be able to form a state in which the pupil 12 (or pupil and cornea) of the eyeball 11 is in focus but the lens of the glasses 5 is not in focus.

  Next, pupil diameter / gaze direction calculation processing will be described with reference to FIG. In the following description, the pupil diameter / gaze direction calculation process will be described under the precondition that both the pupil 12 and the cornea of the eyeball 11 are in focus.

  In the pupil / line-of-sight measurement device 8 of the present embodiment, first, in step S21, the image processing unit 3a has a luminance smaller than a predetermined luminance threshold LT shown in FIG. A portion having the same is recognized as partial image data of the pupil 12. In the present embodiment, in step S <b> 22, the image processing unit 3 a recognizes the image data of the eyeball 11 as partial image data of the Purkinje image reflected by the cornea of the measurement subject 10. The partial image of the Purkinje image is a portion having a brightness larger than a specific brightness threshold UT larger than a predetermined brightness threshold LT shown in FIG. That is, the image processing unit 3a recognizes the partial image of the pupil 12 and the partial image of the Purkinje image P.

  Next, the image processing unit 3a calculates the pupil diameter r using the partial image data of the pupil 12 in step S23. In the present embodiment, in FIG. 6, when the virtual line parallel to the X axis is moved along the Y axis direction, the distance of the portion where the partial image data of the pupil 12 and the virtual line overlap is the longest. The value is the pupil diameter r. Note that. The pupil diameter r is preferably measured in the horizontal direction (lateral diameter) of the pupil 12 in consideration of measurement errors caused by eyelid shadows or the like.

  Thereafter, in step S24, the image processing unit 3a calculates the line-of-sight direction d using the partial image data of the pupil 12 and the partial image data of the Purkinje image P.

  In order to calculate the line-of-sight direction d, first, the spatial coordinates of the center T of the pupil 12 are obtained.

  FIG. 11 illustrates a lens center 222 of the optical system 2a and a planar coordinate system 26 of an image acquired by the image sensor 2b that is located at a focal distance f from the lens center 222. The circle 12 a is the outer periphery of the image of the pupil 12, the center point t of the circle 12 a is the image of the center T of the pupil 12, and p is the Purkinje image acquired on the planar coordinate system 26. The plane coordinates (tu, tv) of the center point t can be grasped from the captured image. Therefore, the spatial coordinates (Tx, Ty, T z) of the center T of the pupil 12 are calculated by the following formula.

  Tx = (Tz / f) · tu, Ty = (Tz / f) · tv, and Tz is substantially equal to the distance between the lens center 222 and the eyeball 11. Here, f is the focal length of the optical system 2a. The symbol “•” indicates a scalar product.

  Specifically, a technique disclosed in Japanese Patent Laid-Open No. 2005-253778 may be used as a method for calculating the spatial coordinates of the center T of the pupil 12. However, the spatial coordinates of the center T of the pupil 12 may be calculated by any method. For example, the technique disclosed in JP-A-8-140937 may be used.

  When calculating the line-of-sight direction d, in this embodiment, the image processing unit 3a uses the values of (i) to (iii) that it knows to calculate the spatial coordinates of the Purkinje image P as a triangle. Calculated according to the principle of surveying.

(I) Distance of a virtual line segment 111-222 connecting the light source 111 of the eyeball irradiation unit 1 and the lens center 222 shown in FIG. 11 (ii) A virtual connection of the light source 111 of the eyeball irradiation unit 1 and the Purkinje image P Angle θ1 formed by line segment P-111 and virtual line segment 111-222
(Iii) Angle θ2 formed by a virtual line segment P-222 connecting the lens center 222 and the Purkinje image P and a virtual line segment 111-222
The radius R of the virtual corneal sphere model 14a including the cornea 14 is assumed to be a known constant value (for example, 7.7 mm).

  The value of the radius R is equal to the distance of the virtual line segment SP that is the distance between the center S of the virtual corneal sphere model 14a and the Purkinje image P. The direction from the Purkinje image P toward the center S of the virtual corneal sphere model 14a is on the extension line of the virtual line segment PM connecting the midpoint M between the light source 111 and the lens center 222 of the eyeball irradiation unit 1 and the Purkinje image P. It is in. Therefore, the spatial coordinates of the center S of the virtual corneal sphere model 14a are easily calculated.

  Thereafter, the image processing unit 3a uses the spatial coordinates of the center T of the pupil 12 and the spatial coordinates of the center S of the virtual corneal sphere model 14a to connect the center T of the pupil 12 and the center S of the corneal sphere model 14a. Is calculated as the line-of-sight direction d.

  As a technique for calculating the line-of-sight direction d, any technique such as the technique disclosed in Japanese Patent Laid-Open No. 2006-95008 or Japanese Patent Laid-Open No. 2003-79777 may be used.

  When the shape specificity of the eyeball 11 adversely affects the measurement of the pupil diameter r and the line-of-sight direction d, the values of the pupil diameter r and the line-of-sight direction d may be corrected by using a virtual eyeball model. The process for removing the adverse effects of eyelash shadows is a commonly performed process. That is, when binarizing the image of the pupil 12, if a short break is found in the image of the pupil 12 in the horizontal direction (the X-axis direction described above), the break is complemented by estimation from surrounding images. Is done.

  The calculation process of the pupil diameter r and the line-of-sight direction d in the present embodiment is as described above. In particular, in the present embodiment, in order to improve the measurement accuracy of the line-of-sight direction d, the line-of-sight direction d is calculated using the partial image data of the pupil 12 and the partial image data of the Purkinje image P. However, any process may be executed as long as it can calculate the pupil diameter r and the line-of-sight direction d. For example, the image processing unit 3 a can calculate both the pupil diameter r and the line-of-sight direction d using only the partial image data of the iris 13. In the present specification, it is assumed that the iris 13 is also in focus when the pupil 12 is in focus. In this case, an elliptic model is created from the partial image of the iris 13. The line-of-sight direction d may be calculated based on the five parameters of the center coordinates, the long axis, the short axis, and the inclination of the axis of the elliptic model. Similarly, both the pupil diameter r and the line-of-sight direction d may be calculated from only the partial image of the pupil 12.

  According to the above control, the periphery of the pupil 12 is enlarged and photographed with the telephoto lens in order to improve the measurement accuracy of the pupil diameter r and the line-of-sight direction d. At that time, using the shallowness of the depth of field X, the image reflected by the lens of the glasses 5 is blurred in the image acquired by the imaging device 2b. Thereby, the bad influence by the reflected image on the lens of the glasses 5 can be reduced by simple image processing called binarization. As a result, the accuracy of measurement of the pupil diameter r and the line-of-sight direction d is further improved.

  Finally, an example of the lighting fixture control process according to the present embodiment will be described with reference to FIG.

  In step S41, the control unit 3 determines whether or not the pupil diameter r is smaller than a predetermined value. If it is determined in step S41 that the pupil diameter r is not smaller than the predetermined value, the control unit 3 considers that the surrounding environment of the measurement subject 10 is not too bright. Therefore, the control part 3 performs the process of step S43. On the other hand, if it is determined in step S41 that the pupil diameter r is smaller than the predetermined value, the control unit 3 determines whether the person to be measured 10 is glaring or not because the surrounding environment is too bright. Is considered thin. Thereby, in step S42, the control part 3 performs the process which makes the illumination intensity of the lighting fixture 50 small, and changes the luminescent color of the lighting fixture 50 to a dark color. That is, the control unit 3 changes the illumination mode of the luminaire 50 so as to reduce the burden on the eye of the measurement subject 10 whose pupil diameter r is excessively small because it is too bright. However, the brightness and color of the light emitted by the luminaire 50 may be controlled independently.

  Next, in step S43, the control unit 3 determines whether or not the pupil diameter r is larger than a specific value. If it is determined in step S43 that the pupil diameter r is not larger than the specific value, the control unit 3 considers that the surrounding environment of the measurement subject 10 is not too dark. Thereby, the control part 3 performs the process of step S45. On the other hand, if it is determined in step S43 that the pupil diameter r is larger than the specific value, the control unit 3 determines that the person to be measured 10 has an eye because the surrounding environment of the person to be measured 10 is too dark. It is considered as being too wide open. Thereby, the control part 3 performs the process which changes the luminescent color of the lighting fixture 50 to a bright color with the illuminance of the lighting fixture 50 large in step S44. That is, the control unit 3 changes the illumination mode of the luminaire 50 so as to reduce the burden on the measurement subject 10 whose pupil diameter r is excessively large because it is too dark. However, also in this case, the brightness and color of the light emitted from the lighting device 50 may be controlled independently.

  Thereafter, in step S45, the control unit 3 determines whether or not the line-of-sight direction d is stable. For example, if the line-of-sight direction d passes through a certain range of areas, the control unit 3 determines that the line-of-sight direction d is stable. If it is determined in step S45 that the line-of-sight direction d is stable, the control unit 3 considers that the concentration of the person to be measured 10 is appropriate and executes the process of step S41. On the other hand, in step S45, it may be determined that the line-of-sight direction d is not stable. For example, there may be a case where the time during which the line-of-sight direction d does not pass through a certain range of region continues for a predetermined time or more. In this case, the control unit 3 considers that the concentration of the person being measured 10 is not appropriate. Thereby, the control part 3 performs the process which blinks the lighting fixture 50 in step S46. That is, the control unit 3 causes the lighting device 50 to blink in order to alert the measurement subject 10 determined that the line-of-sight direction d is not stable. In this case, the emission color of the luminaire 50 may be a color that alerts the person to be measured 10 such as red or yellow.

  As shown in FIG. 13, the first example of the illumination system 20 is installed as a control device in which the pupil / line-of-sight measurement device 8 controls the backlight of the display unit of the car navigation system 50 a of the automobile 70 as the illumination fixture 50. Has been. Moreover, the pupil / gaze measuring device 8 also controls the interior lamp 50b. In the present embodiment, the person to be measured 10 is a driver of the automobile 70.

  In a situation where the lighting system 20 is used, when the state in which the line-of-sight direction d does not pass through a region within a certain range on the front side continues for a predetermined time or longer, the driver who is the measurement subject 10 Or you may be driving aside. In that case, the measurement subject 10 can be alerted by blinking the lighting fixture 50 or lighting the lighting fixture 50 brighter.

  Similarly, when a large fluctuation is observed in the pupil diameter r, it is considered that the driver of the car is drowsy. Even in such a case, it is possible to promote awakening by increasing the illuminance of the luminaire 50, or to lower the color temperature and relax the driver of the car.

  As shown in FIG. 14, in the second example of the illumination system 20 of the present embodiment, the pupil / line-of-sight measuring device 8 as described above controls the backlight of the liquid crystal display unit LED 150 as the illumination fixture 50. It is installed as a control device. The liquid crystal display unit LED 150 is a display unit of a computer such as a tablet terminal or a portable terminal. The person to be measured 10 is looking at the visual recognition object 100. According to this illumination system 20, the illumination mode by the backlight that greatly affects the state of the person being measured 10 can be changed based on the state of the person being measured 10. For example, if the pupil diameter r is too large, the brightness of the backlight as the lighting apparatus 50 is increased, and if the pupil diameter r is too small, the brightness of the backlight as the lighting apparatus 50 is decreased. Thereby, the degree of eye fatigue of the person to be measured 10 who is looking at the visual recognition object 100 can be reduced.

  As shown in FIG. 15, in the third example of the illumination system 20 of the present embodiment, the pupil / line-of-sight measuring device 8 as described above controls the illumination fixture 50 installed on the desk as the illumination fixture 50. It is a control device. The person to be measured 10 is a student who is studying while looking at an image displayed on the visual recognition object 100 in the room.

  In this case, if the pupil diameter r is excessively small, the concentration of the subject 10 on studying may be reduced. At this time, according to the lighting system 20, the measurement subject 10 can be alerted by blinking the lighting fixture 50 or lighting the lighting fixture 50 brighter. Specifically, when the pupil diameter r is smaller than a predetermined value, the lighting fixture 50 is blinked or the brightness of light emitted from the lighting fixture 50 is increased. Also in this case, the color of the light emitted from the luminaire 50 may be changed to a bright color (or high color temperature).

  Conversely, if the pupil diameter r is excessively large, the person to be measured 10 may be excited in a tension state. In this case, in order to relieve tension, the brightness of the light emitted from the lighting device 50 may be reduced. Furthermore, in order to calm the person to be measured 10, the color of the light emitted from the lighting device 50 may be changed to a dark color (or low color temperature).

  As shown in FIG. 16, the third example of the illumination system 20 according to the present embodiment is a luminaire in which the pupil / line-of-sight measuring device 8 as described above is attached to the ceiling surface as the luminaire 50. . A person to be measured 10 is a person watching a television as a visually recognized object 100 in a drawing room. The pupil / line-of-sight measurement device 8 of the present embodiment controls the illumination mode of the luminaire 50 attached to the ceiling surface by wireless communication. The pupil / line-of-sight measuring device 8 according to the present embodiment is built in a remote controller of a television (or a lighting fixture) and placed on a table in the living room.

  When the pupil diameter r is excessively small because the pupil / line-of-sight measurement device 8 of the present embodiment is too bright, the illuminance of the luminaire 50 is reduced so as to reduce the burden on the eye of the measurement subject 10. . If it is determined that the measured person 10 is gazing at the center of the TV screen based on the measured line-of-sight direction, the light emitted from the TV is regarded as too bright and the TV brightness is adjusted. It may be lowered. In addition, the pupil / gaze measurement device 8 of the present embodiment increases the illuminance of the luminaire 50 so as to reduce the burden on the measurement subject 10 when the pupil diameter r is excessively large because it is too dark. . Thereby, the degree of eye fatigue of the television viewer as the person to be measured 10 is reduced. Note that the visual recognition object 100 may be placed on a table as shown in FIG.

  Although the pupil / line-of-sight measurement device 8 according to the first embodiment is described as controlling the luminaire 50, the pupil / radius measurement device 8 simply outputs data of the pupil diameter r and the line-of-sight direction d without controlling the luminaire 50. It may be used as a device. In this case, the pupil / line-of-sight measuring device 8 may have a display unit 7 for displaying data on the pupil diameter r and the line-of-sight direction d. In addition, the pupil diameter r and the line-of-sight direction d data output from the pupil / line-of-sight measurement device 8 may be displayed by another device.

(Embodiment 2)
Next, a pupil / line-of-sight measurement apparatus according to Embodiment 2 of the present invention will be described with reference to FIG.

  As shown in FIG. 17, the pupil / line-of-sight measurement device 8 according to the second embodiment includes an eyeball irradiation unit 1, an optical system 2a, an image sensor 2b, an image processing unit 3a, a display unit 7, and an operation unit 2d. Yes. The eyeball irradiation unit 1 irradiates light to the eyeball 11 of the measurement subject 10 in the state of wearing the glasses 5. The reflected light reflected by the eyeball 11 passes through the optical system 2a. The imaging element 2b receives the reflected light that has passed through the optical system 2a and acquires image data of the eyeball 11. The image processing unit 3a calculates at least one of the pupil diameter r and the line-of-sight direction d of the person to be measured 10 based on the image data of the eyeball 11 acquired by the imaging device 2b. These are exactly the same as in the first embodiment.

  The pupil / line-of-sight measurement device 8 of the present embodiment includes a display unit 7 that is not necessary in the pupil / line-of-sight measurement device of the above-described embodiment. The display unit 7 displays an image of the eyeball 11 based on the image data of the eyeball 11 acquired by the imaging element 2b. The display unit 7 is controlled by a display / illumination control unit 3 b built in the control unit 3. The display / illumination control unit 3 b displays the image data of the eyeball 11 on the display unit 7. In the present embodiment, it is not essential for the display / lighting control unit 3b to control the lighting fixture, but the lighting fixture may be controlled as necessary.

  In the pupil / line-of-sight measurement device 8 of the embodiment, an operation unit 2d that is not necessary in the pupil / line-of-sight measurement device of the above embodiment is provided in the imaging unit 2. The operation unit 2d is configured so that the measurer 200 is focused on the pupil 12 (or the pupil and the cornea) of the eyeball 11 while viewing the image of the eyeball 11 displayed on the display unit 7. This is for controlling the optical system 2a so that it is not in focus. That is, in the present embodiment, the measurer 200 is focused on the pupil 12 (or the pupil and the cornea) of the eyeball 11 instead of the optical system control unit 2c of the first embodiment. The lens is out of focus. The operation unit 2d is, for example, for changing each of the aperture diameter D and the focal length f of the diaphragm. The measuring person 200 changes the form of the optical system 2a by operating the operation unit 2d, and adjusts the focus to the pupil 12 (or the pupil and the cornea) of the person to be measured 10 or decreases the depth of field X. Thus, the image of the lens of the glasses 5 can be blurred.

  The pupil / line-of-sight measurement device 8 of the present embodiment also improves the accuracy of calculation of at least one of the pupil diameter r and the line-of-sight direction d. Further, according to the pupil / line-of-sight measurement device 8 of the present embodiment, the measurer 200 checks whether the pupil 12 (or the pupil and the cornea) is in focus and whether the disturbance light O is blurred. However, the pupil diameter r and the line-of-sight direction d can be recognized. Therefore, it is possible to prevent deficiencies in measurement of the pupil diameter r and the line-of-sight direction d due to malfunction of the pupil / line-of-sight measurement device 8 or the like.

  The display / illumination control unit 3b preferably displays the pupil diameter r and the line-of-sight direction d calculated by the image processing unit 3a on the display unit 7. According to this, the measurer 200 can grasp the pupil diameter r and the line-of-sight direction d while changing the image of the eyeball 11 displayed on the display unit 7 with the operation unit 2d.

  However, when the display unit 7 is not provided, the display / illumination control unit 3b may store the pupil diameter r and the line-of-sight direction d calculated by the image processing unit 3a in a recording medium. Further, the display / illumination control unit 3b may output to another computer. In these cases, the pupil diameter r and the line-of-sight direction d are processed in another device.

  Similarly to the first embodiment, the display / lighting control unit 3b may control the lighting device 50 in the present embodiment. In this case, how the pupil diameter d and the line-of-sight direction d change according to the change in the illumination mode of the luminaire 50 indicates how the measurer 200 displays the pupil diameter r and the line of sight displayed on the display unit 7. This can be grasped by a change in the value of the direction d.

(Embodiment 3)
As shown in FIGS. 18 and 19, the pupil / line-of-sight measuring device 8 of the third embodiment includes a position fixing unit 64, an eyeball irradiation unit 1, an image pickup unit 2 (an optical system 2a, an image pickup element 2b), and an image. A processing unit 3a is provided.

  The pupil / line-of-sight measurement device 8 of the third embodiment includes a position fixing unit 64 that is not necessary in the pupil / line-of-sight measurement device of the above-described embodiment. The position fixing unit 64 fixes the position of the face 15 of the measurement subject 10 with the glasses 5 on. The position of the eyeball irradiation unit 1 is fixed with respect to the position fixing unit 64, and the position and posture are set so that the eyeball 11 of the measurement subject 10 whose position of the face 15 is fixed by the position fixing unit 64 is irradiated with light. Yes. However, the position fixing unit 64 can finely adjust the shape, posture, and the like so that an image of the eyeball 11 can be acquired according to individual differences in the size and shape of the face 15 of the measurement subject 10. It is configured.

  The position fixing unit 64 includes the contact unit 4 with which the face 15 of the measurement subject 10 contacts. The position fixing unit 64 includes an eyeball irradiation unit 1, an image pickup unit 2 (optical system 2 a and image pickup device 2 b), and a connection unit 6 that connects the housing of the image processing unit 3 a and the contact unit 4. In FIG. 18, the contact portion 4 is in contact with the forehead of the person to be measured 10, but in FIG. 19, it is in contact with the jaw of the person to be measured 10. The contact part 4 may contact any position on the face 15 of the person to be measured 10. The connecting unit 6 has rigidity that maintains the relative position of the face 15 with respect to the eyeball irradiation unit 1, the imaging unit 2 (the optical system 2a, the imaging device 2b), and the image processing unit 3a. Anything may be used.

  Further, the image processing unit 3a calculates at least one of the pupil diameter r and the line-of-sight direction d of the measurement subject 10 based on the image data of the eyeball 11 acquired by the imaging device 2b. These points are the same as in the above embodiment.

  In the pupil / line-of-sight measurement device 8 of the third embodiment, the optical system 2a is different from the optical system 2a of the pupil / line-of-sight measurement device 8 of the above-described embodiment. In the optical system 2a of the present embodiment, the pupil 12 (or the pupil and the cornea) of the eyeball 11 of the measurement subject 10 whose position of the face 15 is fixed by the position fixing unit 64 is in focus. In addition, the optical system 2a of the present embodiment is set so that the lens of the glasses 5 is not in focus. The optical system 2a of the present embodiment is the same as the optical system 2a of the above-described embodiment in that the reflected light reflected by the eyeball 11 passes. The imaging element 2b receives the reflected light that has passed through the optical system 2a and acquires image data of the eyeball 11.

  The above configuration also improves the accuracy of calculation of at least one of the pupil diameter r and the line-of-sight direction d. The pupil / line-of-sight measurement device 8 of the present embodiment has a simple structure, but the accuracy of calculation of the pupil diameter r and the line-of-sight direction d is improved.

  The pupil / line-of-sight measurement device 8 of the present embodiment includes a display unit 7 that is not necessary in the pupil / line-of-sight measurement device of the above-described first embodiment. The display unit 7 displays an image of the eyeball 11 based on the image data of the eyeball 11 acquired by the imaging element 2b. The display unit 7 is controlled by a display / illumination control unit 3 b built in the control unit 3. The display / illumination control unit 3 b displays the image data of the eyeball 11 on the display unit 7. Therefore, the measurer 200 can finely adjust the shape and posture of the position fixing unit 64 while viewing the image of the eyeball 11 displayed on the display unit 7. Therefore, even if there are individual differences in the size and shape of the face 15 of the person 10 to be measured, an image of the eyeball 11 in focus on the pupil 12 (or the pupil and cornea) can be easily acquired by the pupil / line-of-sight measuring device 8. Is done.

  Also in the present embodiment, it is preferable that the display / illumination control unit 3b displays the pupil diameter r and the line-of-sight direction d calculated by the image processing unit 3a on the display unit 7. According to this, the measurer 200 can grasp the pupil diameter r and the line-of-sight direction d while viewing the image of the eyeball 11 displayed on the display unit 7.

  Also in the present embodiment, when the display unit 7 is not provided, the display / illumination control unit 3b stores the pupil diameter r and the line-of-sight direction d calculated by the image processing unit 3a in a recording medium. It may be. Further, the display / illumination controller 3b may output to another computer. In this case, the pupil diameter r and the line-of-sight direction d are processed in another device.

  Similarly to the first embodiment, the display / lighting control unit 3b may control the lighting device 50 in the present embodiment. In this case, how the pupil diameter d and the line-of-sight direction d change according to the change in the illumination mode of the luminaire 50 indicates how the measurer 200 displays the pupil diameter r and the line of sight displayed on the display unit 7. This can be grasped by a change in the value of the direction d.

(Embodiment 4)
The pupil / line-of-sight measurement device according to the present embodiment includes the pupil / line-of-sight measurement device 8 according to any of Embodiments 1 to 3 provided for each of the right eyeball 11 and the left eyeball 11. It is. According to this, since the state of the left and right eyeballs 11 of the person to be measured 10 can be grasped simultaneously, the state of the person to be measured 10 can be grasped with higher accuracy.

(Other)
In addition to the contraction and expansion of the pupil 12 in response to the light entering the eye, the muscles are contracted and expanded, and many studies have also been made that a sense of comfort or discomfort appears in the movement of the pupil 12 through the autonomic nerve. . The pupil / line-of-sight measurement device 8 of the present embodiment described above can also be used for such research.

  Hereinafter, features of the pupil / line-of-sight measurement device 8 of each embodiment and effects obtained thereby will be described.

  (1) The pupil / line-of-sight measuring device 8 of the first embodiment includes an eyeball irradiation unit 1, an optical system 2a, an image sensor 2b, and an optical system control unit 2c. The eyeball irradiation unit 1 irradiates light to the eyeball 11 of the measurement subject 10 in the state of wearing the glasses 5. The reflected light reflected by the eyeball 11 passes through the optical system 2a. The imaging element 2b receives the reflected light that has passed through the optical system 2a and acquires image data of the eyeball 11. The image processing unit 3a calculates at least one of the pupil diameter r and the line-of-sight direction d of the measurement subject 10 based on the image data of the eyeball 11 acquired by the imaging element 2b. The optical system control unit 2c controls at least one of the optical system 2a and the image sensor 2b so that the pupil 12 of the eyeball 11 is in focus but the lens of the glasses 5 is not in focus. .

  According to the above configuration, the image data of the eyeball 11 is acquired by the imaging device 2b in a state where the disturbance light O reflected on the lens of the glasses 5 is blurred. When the image of the blurred part overlaps the image of the pupil 12, the image processing unit 3a can distinguish between the image of the blurred part and the image of the pupil 12 by the image processing. Therefore, the image processing unit 3a recognizes the image of the pupil 12 by excluding the image of the blurred portion that overlaps the image of the pupil 12. Thereby, the image processing unit 3a eliminates the adverse effect of the disturbance light O reflected on the lens of the spectacles 5 on the image of the pupil 12, and at least of the pupil diameter r and the line-of-sight direction d of the measurement subject 10. Either one can be calculated. As a result, the accuracy of calculation of at least one of the pupil diameter r and the line-of-sight direction d is improved.

  (2) The pupil / line-of-sight measurement device 8 of the second embodiment includes an eyeball irradiation unit 1, an optical system 2a, an imaging element 2b, an image processing unit 3a, a display unit 7, and an operation unit 2d. The eyeball irradiation unit 1 irradiates light to the eyeball 11 of the measurement subject 10 in the state of wearing the glasses 5. The reflected light reflected by the eyeball 11 passes through the optical system 2a. The imaging element 2b receives the reflected light that has passed through the optical system 2a and acquires image data of the eyeball 11. The image processing unit 3a calculates at least one of the pupil diameter r and the line-of-sight direction d of the person to be measured 10 based on the image data of the eyeball 11 acquired by the imaging device 2b. The display unit 7 displays an image of the eyeball 11 based on the image data of the eyeball 11 acquired by the imaging element 2b. The operation unit 2d can control at least one of the optical system 2a and the image sensor 2b. Thereby, the measurer 200 forms a state where the eyeball 11 of the eyeball 11 is in focus but the lens of the eyeglass 5 is not in focus while viewing the image of the eyeball 11 displayed on the display unit 7. To do.

  The above configuration also improves the accuracy of calculation of at least one of the pupil diameter r and the line-of-sight direction d.

  (3) The pupil / line-of-sight measurement device 8 of the third embodiment includes a position fixing unit 64, an eyeball irradiation unit 1, an optical system 2a, an image sensor 2b, and an image processing unit 3a. The position fixing unit 64 fixes the position of the face 15 of the measurement subject 10 with the glasses 5 on. The position of the eyeball irradiation unit 1 is fixed with respect to the position fixing unit 64, and the position and posture are set so that the eyeball 11 of the measurement subject 10 whose position of the face 15 is fixed by the position fixing unit 64 is irradiated with light. Yes.

  The reflected light reflected by the eyeball 11 passes through the optical system 2a. The imaging element 2b receives the reflected light that has passed through the optical system 2a and acquires image data of the eyeball 11. The image processing unit 3a calculates at least one of the pupil diameter r and the line-of-sight direction d of the measurement subject 10 based on the image data of the eyeball 11 acquired by the imaging element 2b. The optical system 2 a and the image sensor 2 b are in focus on the pupil 12 of the eyeball 11 of the measurement subject 10 whose position of the face 15 is fixed by the position fixing unit 64, but not in focus on the lens of the glasses 5. Is set to

  The above configuration also improves the accuracy of calculation of at least one of the pupil diameter r and the line-of-sight direction d.

  (4) The image processing unit 3 a may recognize a portion of the image data of the eyeball 11 having a luminance smaller than a predetermined luminance threshold LT as the partial image data of the pupil 12. In this case, it is preferable that the image processing unit 3a calculates at least one of the pupil diameter r and the line-of-sight direction d using the partial image data of the pupil 12.

  According to said structure, the bad influence which the image of disturbance light O has on the image of the pupil 12 can be reliably excluded by the image process using a predetermined threshold value.

  (5) A precondition that the cornea of the eyeball 11 is in focus in addition to the pupil 12 may be satisfied. In this case, the image processing unit 3a reflects the portion of the image data of the eyeball 11 that has a luminance greater than the predetermined luminance threshold UT larger than the predetermined luminance threshold LT by the cornea of the measurement subject 10. It may be recognized as partial image data of the image P. In this case, it is preferable that the image processing unit 3a calculates the line-of-sight direction d using the partial image data of the pupil 12 and the partial image data of the Purkinje image P.

  According to said structure, by using the Purkinje image P from which the bad influence resulting from the disturbance light O was excluded, the gaze direction d of the to-be-measured person 10 is computable with higher precision.

  (6) The optical system 2a preferably has a mechanism that realizes three functions of pan, tilt, and zoom. In this case, the optical system control unit 2c realizes the above-described three functions of the optical system 2a so that the image data of the eyeball 11 is automatically acquired based on the image data captured by the image sensor 2b. It is preferable to control the mechanism.

  According to said structure, the image data of the eyeball 11 are acquired automatically.

  (7) It is preferable that the optical system 2a has a mechanism for realizing an autofocus function for automatically focusing on the pupil 12. In this case, the optical system control unit 2c is a mechanism that realizes the autofocus function of the optical system 2a so that the pupil 12 is automatically focused based on the image data of the eyeball 11 imaged by the imaging device 2b. Is preferably controlled.

  According to said structure, the focus of the image of the eyeball 11 can be automatically adjusted on the image pick-up element 2b.

  (8) The pupil / line-of-sight measurement device 8 further includes a position / orientation changing mechanism 120 that changes the attitude and position of at least one of the eyeball irradiation unit 1 and the imaging unit 2 including the optical system 2a and the imaging device 2b. It is preferable to provide. In this case, it is preferable that the optical system control unit 2c change the postures and positions of the eyeball irradiation unit 1 and the imaging unit 2 based on the image data of the eyeball 11 captured by the imaging device 2b. Thereby, the reflected light of the light irradiated by the eyeball irradiation unit 1 on the spectacles 5 or the body part around the eyeball 11 does not overlap the pupil 12, and the shadow of the body part around the spectacles 5 or the eyeball 11 does not overlap the pupil. 12 is not overlapped.

  According to said structure, the image of the eyeball 11 is acquirable in the state which excluded the bad influence by the spectacles 5 of the light with which the eyeball irradiation part 1 irradiated, or the body part around the eyeball 11 was excluded. In addition, the image of the eyeball 11 can be acquired in a state in which the adverse effects due to the shadow of the body part around the glasses 5 or the eyeball 11 are eliminated.

  (9) It is preferable that the optical system control unit 2c controls the optical system 2a so that both the pupil 12 and the lenses of the glasses 5 are in focus. Thereafter, the optical system controller 2c preferably controls the optical system 2a so that the pupil 12 is kept in focus and the lens of the glasses 5 is not in focus.

  According to the above configuration, it is possible to efficiently form a state where the pupil 12 is in focus but the eyeglass 5 is not in focus.

  (10) The pupil / line-of-sight measurement device 8 according to any one of the first to third embodiments is preferably a pupil / line-of-sight measurement device provided for each of the right eyeball 11 and the left eyeball 11.

  According to said structure, since the state of the left and right eyeballs 11 of the to-be-measured person 10 can be grasped simultaneously, the state of the to-be-measured person 10 can be grasped with higher accuracy.

  (11) The illumination system 20 includes the pupil / line-of-sight measurement device 8 according to any one of the first to third embodiments, and the illumination fixture 50 that irradiates light to the environment where the measurement subject 10 exists. In this case, it is preferable that the pupil / gaze measuring device 8 controls the illumination mode of the luminaire 50 based on at least one of the pupil diameter r and the gaze direction d calculated by the image processing unit 3a.

  According to said structure, the aspect of the illumination by the lighting fixture 50 can be changed based on the state of the to-be-measured person 10 estimated from at least any one of the pupil diameter r and the gaze direction d.

  (12) The luminaire 50 is preferably a backlight of a display device. According to this, based on the state of the person to be measured 10, it is possible to adjust the aspect of illumination by the backlight of the visual recognition object 100 that greatly affects the state of the person 10 to be measured.

DESCRIPTION OF SYMBOLS 1 Eyeball irradiation part 2a Optical system 2b Image pick-up element 2c Optical system control part 3a Image processing part 5 Glasses
7 Display section
8 pupil / line-of-sight measuring device 10 person to be measured 11 eyeball 12 pupil 20 lighting system 50 lighting fixture 64 position fixing unit 200 measuring person

Claims (12)

An eyeball irradiation unit that irradiates light to the eyeball of the measurement subject in the state of wearing glasses;
An optical system through which reflected light reflected by the eyeball passes;
An image sensor that receives the reflected light that has passed through the optical system and acquires image data of the eyeball;
An image processing unit that calculates at least one of the pupil diameter and the line-of-sight direction of the person to be measured based on the image data of the eyeball acquired by the imaging device;
An optical system control unit that controls at least one of the optical system and the imaging element so that the pupil of the eyeball is in focus but the lens of the eyeglass is not in focus. Pupil / line-of-sight measurement device. An eyeball irradiation unit that irradiates light to the eyeball of the measurement subject in the state of wearing glasses;
An optical system through which reflected light reflected by the eyeball passes;
An image sensor that receives the reflected light that has passed through the optical system and acquires image data of the eyeball;
An image processing unit that calculates at least one of the pupil diameter and the line-of-sight direction of the person to be measured based on the image data of the eyeball acquired by the imaging device;
A display unit that displays the image of the eyeball based on the image data of the eyeball acquired by the imaging device;
While the measurer looks at the image of the eyeball displayed on the display unit, the optical system and the eyepiece are in focus so that the pupil of the eyeball is in focus but the lens of the eyeglass is not in focus. A pupil / line-of-sight measurement device comprising: an operation unit capable of controlling at least one of the imaging elements. A position fixing unit for fixing the position of the face of the measurement subject in the state of wearing glasses;
An eyeball irradiating unit in which a position and a posture are set so as to irradiate light to the eyeball of the measurement subject whose position with respect to the position fixing unit is fixed and the position of the face is fixed by the position fixing unit;
An optical system through which reflected light reflected by the eyeball passes;
An image sensor that receives the reflected light that has passed through the optical system and acquires image data of the eyeball;
An image processing unit that calculates at least one of the pupil diameter and the line-of-sight direction of the measurement subject based on the image data of the eyeball acquired by the imaging device;
The optical system and the image sensor are in focus on the pupil of the eyeball of the measurement subject whose position of the face is fixed by the position fixing unit, but the lens of the glasses is not in focus. Pupil / line-of-sight measuring device set to. The image processing unit
Of the image data of the eyeball, a portion having a luminance smaller than a predetermined luminance threshold is recognized as the partial image data of the pupil,
The pupil / line-of-sight measurement device according to claim 1, wherein at least one of the pupil diameter and the line-of-sight direction is calculated using partial image data of the pupil. Under the precondition that the eye cornea is in focus in addition to the pupil,
The image processing unit
Of the image data of the eyeball, a portion having a luminance greater than a specific luminance threshold greater than the predetermined luminance threshold is recognized as partial image data of a Purkinje image reflected by the cornea,
The pupil / gaze measurement apparatus according to claim 4, wherein the gaze direction is calculated using the partial image data of the pupil and the partial image data of the Purkinje image. The optical system has a mechanism that realizes three functions of pan, tilt, and zoom,
The optical system control unit controls a mechanism that realizes the three functions of the optical system so that the image data of the eyeball is automatically acquired based on the image data captured by the image sensor. The pupil / gaze measurement apparatus according to claim 1. The optical system has a mechanism for realizing an autofocus function for automatically focusing on the pupil;
The optical system control unit controls a mechanism that realizes the autofocus function of the optical system so that the pupil is automatically focused based on image data of the eyeball imaged by the imaging device. The pupil / gaze measurement apparatus according to claim 1. A position and orientation changing mechanism that changes the orientation and position of at least one of the eyeball irradiating unit and the imaging unit including the optical system and the imaging element;
Based on the image data of the eyeball imaged by the image sensor, the optical system control unit reflects light reflected by the eyeball irradiation unit on the eyeglasses or a body part around the eyeball. The posture and position of the eyeball irradiation unit and the imaging unit are changed so that a shadow of a body part around the glasses or the eyeball does not overlap the pupil. The pupil / gaze measuring device described.   The optical system control unit controls the optical system so that a state where both the pupil and the eyeglass lens are in focus is formed, and then the state where the pupil is in focus is maintained. The pupil / line-of-sight measurement apparatus according to claim 1, wherein the optical system is controlled so that the lens of the spectacles is out of focus.   A pupil / line-of-sight measuring device according to claim 1 provided for each of a right eyeball and a left eyeball. The pupil / gaze measuring apparatus according to any one of claims 1 to 10,
A lighting fixture that irradiates light to an environment in which the measurement subject exists, and
The pupil / line-of-sight measuring device controls an illumination mode of the lighting fixture based on at least one of the pupil diameter and the line-of-sight direction calculated by the image processing unit. The lighting system according to claim 11, wherein the lighting fixture is a backlight of a display device.

Images