WO2007125794A1 - Data measuring device and data measuring method - Google Patents

Abstract

Provided are a data measuring device and a data measuring method capable of accurately measuring data on a pupil of a human body in a non-contact manner with the human body. The data measuring device (1) includes: an image capturing unit (5) for capturing a dynamic image around an eye of the human body; and a data analysis unit (10) for extracting the pupil by matching with a template for each frame of the dynamic image captured by the image capturing unit (5) and measuring the data on the pupil according to a change by time. The data analysis unit (10) updates the template for each frame by using the data on the pupil of the frame for which data measurement has been already performed. The data on the pupil is formed by a pixel value around the pupil and pupil diameter data.

Description

 Specification

 Data measuring apparatus and data measuring method

 Technical field

 The present invention relates to a data measuring device and a data measuring method, and more particularly to a data measuring device and a data measuring method for measuring data related to a pupil of a living body.

 Background art

 Conventionally, an apparatus for measuring data related to a pupil of a living body has been proposed for diagnostic purposes such as medical diagnosis. As such a data measuring device, there has been proposed a measuring device provided with various measuring means and diagnostic means using data relating to the pupil of a living body.

[0003] For example, Patent Document 1 detects the amount of change in a living body part such as an eye blink and a pupil diameter from a moving image of a living body. A diagnostic device to diagnose is described.

[0004] Patent Document 2 describes a person state in which a person's face is photographed using a CCD camera and infrared light, and a person's state is determined by extracting the pupil position or shape of the person from the photographed image. A detection device is described.

[0005] In Patent Document 3, a change in the pupil area (pupil diameter) of a living body when a stimulus is applied by infrared light is detected by a goggle-shaped measuring tool equipped with an infrared light source and an infrared CCD camera. In addition, a pupil-to-light reaction measuring instrument for relaxation evaluation that evaluates the relaxation of a living body from the detection result is described.

 Patent Document 1: Japanese Patent Laid-Open No. 7-124126

 Patent Document 2: Japanese Patent Laid-Open No. 7-249197

 Patent Document 3: Japanese Patent Laid-Open No. 2005-143684

 Disclosure of the invention

 Problems to be solved by the invention

[0006] However, in the invention described in Patent Document 1, although the amount of change in the living body part is detected from the moving image of the living body, no specific means for acquiring the moving image force and living body data of the living body has been described. [0007] Further, in the invention described in Patent Document 2, the extraction of the pupil position or the pupil shape from the photographed image is performed by a binary key process, so that the state of the person can be accurately determined immediately under the influence of noise. There was a problem that I could not.

 [0008] In addition, the invention described in Patent Document 3 needs to be provided with a special device in combination with an infrared light source and an infrared CCD camera. In addition, it is necessary for the subject to wear goggles at the time of measurement, and there is a problem that a physiological and psychological burden is placed on the living body!

 Furthermore, there is a template matching method as one method for extracting parts such as eyes, pupils, and mouths from biological images such as human face images. In the template matching method, a representative image called a template is prepared, the two are compared while scanning the template position on the image data to be sought, and the best match is extracted. This is a method of detecting the location of the site to be extracted. However, the template is typically representative and fixed, and the template size and content (shape, color, etc.) are not changed during scanning, so the template may be misaligned with the image being inspected (for example, bare skin) If the eye is made into a template and the make-up eye is to be inspected, etc.), or if it doesn't exist (for example, a black eye template is used to extract the blue eye part), the detection accuracy is It will fall. When the inspection target is a moving image, the positional relationship between the subject and the camera changes three-dimensionally for each frame, so the size of the detection target in the moving image data changes, and the template cannot follow the change. When the pupil is the detection target, the camera itself and the subject's movement itself are constantly changing due to the amount of light and physiological changes. Therefore, in the case of performing template matching with high accuracy on an object such as a pupil that changes by itself, it is difficult to use the conventional method of fixing the template.

 [0009] The present invention provides a data measurement apparatus and a data measurement method that improve detection accuracy by updating template information using information obtained from a detection result of a target in a frame image with moving image data. With the goal.

 Means for solving the problem

[0010] In order to solve the above problem, the invention described in claim 1 is

 An image capturing unit for capturing a first image by capturing a living body having an eye area;

Extracting a first eye region from the first image; Detecting a pupil from the first eye region with a predetermined template relating to the pupil, measuring data relating to the detected pupil;

 Data processing to update a predetermined template with measured data 'Analysis unit, The image capturing unit captures the living body and inputs a second image,

 The data processing / analysis unit

 Extract the second eye area from the second image,

 Detect second eye area pupil with updated template

 It is characterized by that.

 [0011] The invention according to claim 2 is the data measuring device according to claim 1,

 The data relating to the pupil is a pixel value at the center of the pupil and diameter data of the pupil.

 [0012] The invention according to claim 3 is the data measuring device according to claim 1 or claim 2, wherein the data processing unit 'analysis unit is provided for each frame. By acquiring the distance between a plurality of points whose positional relationship does not change in time series and correcting the data regarding the pupil for each frame according to the relative relationship of the distance between the plurality of points, The data regarding the pupil is a relative value with respect to a predetermined reference value.

 [0013] The invention according to claim 4 is the data measuring device according to any one of claims 1 to 3, wherein the image photographing unit is A distance sensor or a displacement sensor for measuring a distance or displacement from a living body is provided, and the data analysis unit calculates an absolute value of data related to the pupil using the distance or displacement from the living body as a parameter. To do.

 [0014] The invention according to claim 5 is the data measuring device according to claim 4, wherein the distance sensor or the displacement sensor includes a plurality of imaging elements, and The shape is measured.

[0015] The invention described in claim 6 is the data measuring device described in claim 5, wherein the image photographing unit is a visible camera. [0016] The invention described in claim 7

 Capturing a first image by imaging a living body having an eye area;

 Extracting the first image force first eye region;

 Detecting a pupil from the first eye region with a predetermined template for the pupil; measuring data about the detected pupil;

 Updating the predetermined template with the measured data;

 Photographing the living body and inputting a second image;

 Extracting a second eye region from the second image;

 The second eye region force is also detected by the updated template.

[0017] The invention according to claim 8 is the data measurement method according to claim 7, wherein the data relating to the pupil is a pixel value of a pupil center and pupil diameter data. It is characterized by.

 The invention's effect

 [0018] According to the invention described in claim 1 or claim 7, it is possible to customize the pupil template for each living body and improve the extraction accuracy of the pupil.

[0019] According to the invention of claim 2 or claim 8, it is possible to create or update a pupil template.

[0020] According to the invention described in claim 3 of the claim, it is possible to make the data related to the pupil a unified value for all frames.

 According to the invention described in claim 4, it is possible to calculate an absolute value of data relating to the pupil.

 [0021] According to the invention described in claim 5, it is possible to simplify the apparatus configuration.

 [0022] According to the invention described in claim 6, it is possible to simplify the device configuration and reduce the manufacturing cost.

 Brief Description of Drawings

FIG. 1 is a block diagram showing an overall structure of a data measurement device according to the present embodiment. FIG. 2 is a conceptual diagram of processing for extracting an eye region from face image data.

FIG. 3 is a conceptual diagram showing pupil template creation processing.

 FIG. 4 is a graph showing transition of pupil pixel values and iris pixel values in each frame.

FIG. 5 is a graph showing transition of pupil pixel values and iris pixel values in each frame.

FIG. 6 is a graph showing transition of pupil pixel values and iris pixel values in each frame.

FIG. 7 is a graph showing the transition of the minimum pixel value in each frame.

[8] FIG. 8 is a conceptual diagram showing extraction processing of pupil diameter data using pixel values.

FIG. 9 is a graph showing the transition of diameter data in each frame.

[10] It is a conceptual diagram showing the process of correcting the diameter data using the distance between multiple points in each frame.

 FIG. 11 is a flowchart showing a method for extracting pupil and diameter data by template matching.

 FIG. 12 is a flowchart showing a method for updating a pupil template.

Explanation of symbols

 1 Data measurement device

 2 External device

 3 Control unit

 4 External communication section

 5 Image capture unit

 6 Lighting section

 7 User interface section

 8 IZO part

 9 Memory section

 10 Data processing 'Analysis Department

 11 Parameter setting / Management section

 12 Data storage unit

 13 Display

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block configuration diagram of the data measuring apparatus 1 according to the present embodiment. As shown in FIG. 1, an external device 2 is connected to the data measuring device 1 via a network that can communicate with each other, and the measurement results of the data measuring device 1 can be transmitted to the external device 2. .

 The network in the present embodiment is not particularly limited as long as it means a communication network capable of data communication. For example, the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), a telephone line network ISDN (Integrated Services Digital Network) network, CATV (Cable Television) line, optical communication line, etc. Moreover, it is good also as a structure which can communicate not only by wire but by radio | wireless.

 The external device 2 is constituted by a personal computer or the like, and is preferably installed in a place where some kind of consulting and diagnosis can be received. In addition, the external device 2 may be configured as an Internet site from which consulting information can be obtained, or as a mobile terminal such as a consultant, doctor, or clerk. Also, the external device 2 may be configured as a data server for a home health management system.

[0028] As shown in FIG. 1, the data measuring device 1 includes a control unit 3, an external communication unit 4, an image capturing unit 5, an illumination unit 6, a user interface unit 7, an IZO unit 8, a memory unit 9, and data processing. An analysis unit 10, a parameter setting / management unit 11, a data storage unit 12, and a display unit 13 are provided.

 [0029] The control unit 3 includes a CPU and a RAM, and drives and controls each component of the data measurement device 1. Since the data measuring device 1 of the present embodiment also handles moving images, it is desirable that the control unit 3 be configured with a chip that can control the operation as fast as possible.

 [0030] The external communication unit 4 is configured to be able to perform information communication with the external device 2 by wired or wireless communication means. Note that the data measuring apparatus 1 of the present embodiment handles moving image data, and therefore preferably has a communication mode capable of high-speed transmission as much as possible.

[0031] The image capturing unit 5 captures a moving image around the subject's eyes, such as a camera module or other camera attached to a CCD camera, digital still camera, CMOS camera, video camera, mobile phone, or the like. Consists of. In addition, the image shooting unit 5 may be shifted in color shooting or monochrome shooting, but the following will be described in the case of monochrome shooting. The

 [0032] Although an infrared camera can be used as the image photographing unit 5, it is desirable to photograph with a visible camera in the present embodiment. When using a visible camera, it is desirable to use a camera with high sensitivity in the red region, focusing on the contrast of the captured image. The sensitivity in the red region can be relatively increased by installing an optical filter with high transmittance in that band. Further, it may be used except for a camera filter provided with an infrared cut filter.

 In addition, the image capturing unit 5 of the present embodiment is configured as a distance sensor or a displacement sensor for detecting the distance or displacement between the image capturing unit 5 and the subject. The existing distance sensor or displacement sensor can be used. As a result, the image capturing unit 5 associates the distance and displacement between the image capturing unit 5 and the subject with each captured frame and stores them in the data storage unit 12 or the parameter setting / management unit 11. It has become.

 [0034] In addition, the distance and displacement from the subject can be determined by configuring the camera as the image capturing unit 5 with two eyes and using images (areas between the eyebrows, nose, and the other eye) captured by each power camera. It can also be obtained from measuring the 3D shape. In addition, by measuring the distance between two or more characteristic points, the absolute distance per pixel can be obtained without using a special sensor. This makes it possible to detect the absolute value of data relating to the pupil without providing a distance sensor or a displacement sensor.

 [0035] The illumination unit 6 is an arbitrary component of the data measuring device 1, and can irradiate around the eyes of a subject with a light source when the surrounding environment is dark at the time of photographing. As the light source, white to incandescent (color temperature around 3000K) visible light can be used. It is also possible to use an infrared light source as the light source. Further, by diffusing and illuminating light by the illumination unit 6, direct reflection of light on the surface of the subject's eyes can be mitigated, and the accuracy of image processing can be improved. Further, it is desirable that the illumination unit 6 has a mechanism that can vary the light intensity of the light source.

 [0036] The illumination unit 6 has a mechanism capable of irradiating illumination light for stimulus addition.

As the illumination light for applying the stimulus, it is preferable to use strobe (flash) light that is visible light and stable in intensity and irradiation time. In this case, the intensity of the irradiated light is kept constant over time. The As a result, it is possible to measure the pupil response by applying a stimulus to the eye of the subject at the time of photographing by the image photographing unit 5. In addition, it is desirable that the flash (flash) light can adjust the light emission time if the light emission timing is reached.

 [0037] The user interface unit 7 is composed of a keyboard, a mouse, a trackball, and the like. The user interface unit 7 allows a user to input an instruction, and can transmit the status and request of the data measurement device 1 to the user. Note that since it is desirable to have a device configuration with less burden on the user, the touch panel may be configured integrally with the display unit 13.

 [0038] The IZO unit 8 is configured to be able to connect a portable device such as a CF card, an SD card, or a USB memory card. In addition, a port for accessing an external device 2 such as Ethernet (registered trademark) can be connected. As a result, it is possible to input various parameters and data necessary for setting the operation of the data measuring device 1 and to transmit the measurement results to the external device 2.

 [0039] The memory unit 9 is composed of RAM, ROM, DIMM, etc., and the data processing 'analysis unit 10 etc. transfers data required by the data storage unit 12 etc. Device 1 is designed to operate at high speed and stability. In addition, the memory unit 9 of this embodiment needs to have a capacity that can execute moving image processing in real time without dropping frames.

 [0040] Data processing The analysis unit 10 measures a time-series change of data related to the pupil of the living body by analyzing the moving image captured by the image capturing unit 5.

 First, as shown in FIG. 2, the data processing / analysis unit 10 extracts the eye region from the face image data. For eye region extraction, conventional methods using eye template matching and positional relationship between the eye periphery can be applied. At this time, an eye area in an arbitrary frame may be manually set, and the area may be used as an eye area in another frame. Also, set the eye area wider so that the eyes and iris do not protrude from other frames.

 [0042] Further, the data processing 'analysis unit 10 is adapted to create a pupil template.

The pupil template is created using data such as the size of the pupil, the average pixel value of the pupil, and the average pixel value of the iris, which are obtained by analyzing moving images of a plurality of people's eye regions in advance. example For example, as shown in FIG. 3, the size of the template is set so that the pupil can be accommodated, and a circle corresponding to the pupil is arranged at the center. Then, a pixel template is created by setting the pixel values of the circumference and the inside to the pixel values of the pupils of the plurality of people and the pixel values of the outside of the circle to the average of the pixel values of the irises of the plurality of people. be able to.

It should be noted that the pupil template may be obtained from image data when the pupil is contracted. As a result, the detection accuracy of the pupil center coordinates is increased even for image data in which the pupil is contracted.

 [0044] Further, the data processing 'analysis unit 10 extracts the pupil by performing the template matching of the pupil for each frame of the moving image with respect to the eye region from which the facial image data force has been extracted. At this time, when the difference between the image data of the eye area of each frame and the pupil template is minimized, it is determined that the center of the pupil. Thereby, the pupil center coordinates can be detected.

 In the template matching of the pupil, the search area of the next frame may be narrowed down using the pupil center coordinates extracted in the previous frame under the assumption that the moving distance of the subject (pupil) between the frames is small. . For example, the size of the search area of the next frame can be set separately, and the “pupil center coordinater setting range” of the previous frame can be searched in that frame.

Note that since distortion of the optical system of the image capturing unit 5 affects the accuracy of template matching, it is desirable to capture a lattice chart and acquire distortion correction image data before measurement.

 [0046] Here, when the subject opens his eyes, the feature amount of the difference between the image data of each frame and the template is as shown in the graphs of Figs. On the other hand, when the subject closes his eyes, the feature quantity of the difference between the image data of each frame and the template is as shown in the graph in FIG. Thus, when the subject blinks or closes his eyes at the time of photographing, the minimum value of the difference from the template and the position of the pupil are irrelevant, and it is necessary to exclude these data from the moving image data.

FIG. 7 is a graph showing the transition of the minimum value of the difference from the template in each frame when light stimulation is given to the pupil three times with a certain period of illumination light for stimulus addition. Figure 7 As shown, the minimum value appears as a discontinuous value due to blinking of the subject. Although the minimum value transition is shown in FIG. 7, the pupil center coordinate appears as a discontinuous value when the subject blinks or closes his eyes even in the graph showing the pupil center coordinate transition, etc. .

 [0048] Therefore, the data processing 'analysis unit 10 extracts image data of a part in which transitions such as pixel values and pupil center coordinates are discontinuous from the moving image data, and when the subject blinks, Is excluded as image data when is closed. For example, a predetermined threshold value is separately set for the minute amount (variation amount) in the graph of FIG. 7, and when the differential amount is equal to or greater than the predetermined threshold value, it is possible to determine “blink”. Also, the graph showing the transition of the minimum value in Fig. 7 is smoothed multiple times, the difference between the minimum value before smoothing and after smoothing is taken for each frame, and the image data when the difference increases is blinked. It can also be excluded as image data when the eyes are closed.

 [0049] In the above example, a force using a light stimulus as a stimulus to be applied to the subject. The stimulus to be applied to the subject may be sound, smell, smell, or touching the subject. For example, plosives, explosive sounds, incense, and aromas are possible. In addition, the subject's stress may be alleviated or the subject may be stressed. For example, hot springs, footbaths, hot air, and cold air can be considered.

 [0050] In addition, the "blink" data obtained when extracting the image data when the subject blinks or when the eyes are closed is the power of diverting the data to other analyzes as "blink count" data. The data may be transferred to the external device 2. It is also possible to extract changes at the time of stimulus response as data by searching for the start point, apex and end point position of the change in pixel values and pupil center coordinates at the time of stimulus response. The start position of the pupil response due to light stimulation can be detected by, for example, storing the light emission timing of stimulation light in association with moving image data and using the light emission timing data during image processing. In addition, the search of the start point, the apex point, and the end point position of the change at the time of stimulus response can be performed using other known methods.

[0051] Further, the data processing 'analysis unit 10 starts with the pupil center coordinates obtained by matching the template of the pupil as the starting point, and ends the pupil in at least one of the left, right, up, down, and diagonal directions for each frame of the moving image. By searching for a part, pupil diameter data is extracted. At this time, as shown in FIG. 8, the pixel value at the search position is in the range from 0 to “pixel value at the pupil center coordinate + threshold value” using the pixel value at the pupil center coordinate and the threshold value set separately. If it is outside, it can be determined to be outside the pupil. Pupil diameter data can also be obtained from the end point force of the pupil thus obtained. In addition, when extracting the pupil diameter data, the pixel value force average value and standard deviation amount around the center of the pupil are calculated when searching for the end of the pupil, and ranges such as `` average value + 3 times the standard deviation '' It may be performed by providing a lens and considering it in the pupil. This diameter data may be averaged as a single feature value as shown in FIG. 9, which may be a separate feature value based on the diameter data in the horizontal direction or the diameter data in the vertical direction. Fig. 9 shows the pupil diameter data in each frame. The pupil diameter data changes depending on the stimulus response.

 [0053] Further, for each frame of the moving image, the data processing 'analysis unit 10 updates the pupil template based on the pixel value at the center of the pupil and the diameter data of the pupil obtained by the processing of the immediately preceding frame. This is applied to pupil extraction of the next frame. The data used for updating the pupil template may be data obtained by averaging not only the data obtained in the immediately preceding frame but also the data obtained in all the previous frames.

 [0054] By updating the pupil template in this way, the pupil template is customized for each subject, so that the pupil extraction accuracy is improved.

 [0055] In addition, if the difference between the templates before and after the update is equal to or less than a predetermined threshold value set separately, the data processing 'analysis unit 10 considers that a template suitable for the subject has been created, Thereafter, the template update can be terminated. As a result, unnecessary template creation steps can be omitted, and the process can be performed quickly.

 [0056] It should be noted that the data processing 'analysis unit 10 does not use the updated data when updating the pupil template, if the immediately preceding frame is image data when blinking or closing eyes. To do. Also, when averaging the data of all frames, do not use the image data when blinking or closing eyes.

[0057] Further, the data processing ' analysis unit 10 performs a correction to make the data related to the pupil of each frame relative to a predetermined reference value. In other words, the pupil diameter data obtained by the above processing is the data obtained as relative values in pixel units within one frame. Data and not a uniform value between frames. Therefore, the data processing 'analysis unit 10 obtains the distance between a plurality of points whose positional relationship does not change in time series such as the corner of the eye and the top of each eye in each frame, and calculates the relative distance between these points between each frame. Accordingly, the pupil diameter data is corrected so that the pupil diameter data is unified between the frames. For example, as shown in FIG. 10, the distance between the corner of the eye and the eye is D, the distance Dn between the corner of the eye and the eye is obtained in the nth frame, and the correction amount DnZD is obtained. By correcting the diameter data of the pupil, it is possible to make the diameter data of the pupil uniform with other frames.

 [0058] Further, the data processing 'analysis unit 10 measures the absolute value of pupil diameter data. That is, the diameter data of the pupil obtained by the above processing is a relative value in pixel units, and the absolute value of the diameter data varies depending on the distance between the image capturing unit 5 and the subject. Therefore, the data processing / analysis unit 10 uses the distance sensor or the displacement sensor as the image capturing unit 5 to measure and store the distance and displacement between the image capturing unit 5 and the subject in each frame as parameters. The absolute distance is now calculated!

 [0059] In measuring the absolute value of the diameter data, a reference object (patch, sticker, etc.) with a known size and length is placed around the subject, and one image is drawn from the size of the object on the image. The absolute value of the length per element may be calculated. It is also possible to take an image using a subject that can measure the length of the subject, such as a scaled seal, and calculate the absolute length per pixel in the same way. In addition, the length and size of the area around the eyes, such as the eyelashes of the subject, the nose, and the distance from the corners of the eyes to the eyes, can be separately measured and given as reference parameters.

 [0060] The parameter setting / management unit 11 is configured to be able to set parameters necessary for processing and control of the data measuring device 1, and manages the set parameters. ing.

 [0061] The data storage unit 12 is configured by an HDD or the like, and includes image data input from the outside, moving image data captured by the image capturing unit 5, and a moving image that has been subjected to image processing by the data processing / analyzing unit 10. It manages and holds image data or temporary data during image processing.

[0062] The display unit 13 may be a CRT, liquid crystal, organic EL, plasma, or projection display. In addition to displaying the moving image data being processed by the data processing / analysis unit 10 or the moving image data held by the data storage unit 12, the data processing / analysis unit 10 also relates to the state of each component of the data measuring device 1. Information and information given from the external device 2 are displayed. It is also possible to adopt a configuration that also functions as the user interface unit 7 such as a touch panel.

[0063] Of the above components, control unit 3, external communication unit 4, IZO unit 8, memory unit 9, data processing / analysis unit 10, parameter setting / management unit 11, data storage unit 12, and display The unit 13 can be configured as a general personal computer, and the data measuring device 1 can be configured by attaching the image capturing unit 5, the illumination unit 6, and the user interface unit 7 thereto.

 Next, the data measurement method of the present invention using the above-described data measurement apparatus 1 will be described with reference to the flowcharts of FIG. 11 and FIG.

 First, when the surrounding environment is dark, the illumination unit 6 illuminates the subject's eyes and the image photographing unit 5 photographs the subject's eyes with a moving image. At this time, the illumination unit 6 may irradiate illumination light for stimulus addition.

 Next, as shown in FIG. 2, the data processing / analysis unit 10 extracts an eye area from the face image data photographed by the image photographing unit 5.

 Subsequently, the data processing 'analysis unit 10 creates a pupil template. The pupil template is created using data such as pupil size, pupil pixel value average, and iris pixel value average obtained in advance by analyzing moving images of a plurality of human eye regions.

 [0066] Subsequently, the data processing 'analysis unit 10 extracts the pupil by performing template matching of the pupil for each frame of the moving image for the extracted eye region of the face image data force. Figure 11 shows a flowchart of template matching and pupil extraction methods.

[0067] As shown in FIG. 11, when face image data is input to the data processing 'analysis unit 10 (step S1), the data processing' analysis unit 10 extracts the eye region (step S2) and creates it. Template matching is performed using the pupil template (step S3). The center of the pupil is when the difference between the image data of the eye area of each frame and the pupil template is minimized. By determining, pupil center coordinates are detected (step S4). At this time, the data processing / analysis unit 10 extracts image data of a part of the moving image data in which transitions such as pixel values and pupil center coordinates are discontinuous. Excluded as image data when closed. Next, the pupil diameter data is obtained by searching for the edge of the pupil in at least one of the left, right, up, down, and diagonal directions for each frame of the moving image, starting from the pupil center coordinates obtained by matching the template of the pupil. Is extracted (step S5). Subsequently, it is determined whether or not there is a next frame (step S6). If there is a next frame, steps S1 to S5 are repeated, and if there is no next frame, the process is terminated.

 [0068] Further, the data processing 'analysis unit 10 updates the pupil template for each frame of the moving image. FIG. 12 shows a flowchart of a method for updating the pupil template.

 [0069] As shown in FIG. 12, when the first frame of the moving image is input to the data processing 'analysis unit 10 (step S11), the data processing / analysis unit 10 extracts the eye region (step S12), Template matching is performed using the created pupil template (step S13). Thereby, pupil center coordinates are detected (step S14), and pupil diameter data is extracted (step S15). Next, the data processing / analysis unit 10 updates the pupil template based on the data obtained by the processing of the immediately preceding frame or the average data of all the frames (step S16).

 [0070] At this time, the data processing 'analysis unit 10 determines whether or not the difference between the templates before and after the update is equal to or less than a predetermined threshold value that is set separately. If it is updated and the value is below the predetermined threshold, template matching can be performed without updating the template thereafter.

 Here, the update of the template will be described in detail. When updating the template, as described above, the pupil diameter data obtained by pupil extraction and the pixel values around the pupil center are averaged and calculated. Pixel values and pixel values outside the pupil are used.

The pixel value outside the pupil is calculated by averaging pixel values in a predetermined area outside the area determined to be outside the pupil, that is, the area inside the pupil calculated from the diameter data of the pupil. The area to be averaged is the range of the portion surrounded by concentric circles outside the diameter of the pupil by a predetermined number of pixels or distance. The predetermined number of pixels or distance is set by the parameter setting unit of the image processing apparatus. Or, the partial force other than the white-eye area of the eye is inside the pupil. You can use the area where the area is deleted as the extra-pupil area (resulting in an iris area), with the pixel values in the area averaged.

 [0071] Subsequently, when the next frame is input to the data processing / analysis unit 10 (step S17), the data processing / analysis unit 10 extracts the eye region (step S18), and the generated pupil template is used. Template matching is performed again (step S19). Then, using the updated template, pupil center coordinates are detected (step S20), and pupil diameter data is extracted (step S21). Next, it is determined whether or not there is a next frame (step S22) .If there is a next frame, the pupil template is updated again (step S16), and if there is no next frame, the process ends. To do.

 [0072] Subsequently, the data processing 'analysis unit 10 performs correction so that pupil diameter data is unified between the frames. That is, the diameter data of the pupil is a relative value between the frames.

 [0073] Further, the data processing 'analysis unit 10 measures the absolute value of the diameter data of the pupil. That is, the absolute distance per pixel is calculated using the distance and displacement between the image capturing unit 5 and the subject in each frame measured and stored by the distance sensor or displacement sensor as the image capturing unit 5 as parameters.

 Next, the data processing / analyzing unit 10 transmits the measurement result to the external device 2.

 As described above, according to the data measuring apparatus 1 and the data measuring method according to the present embodiment, the pupil template is updated using the data of the frame in which the data measurement has already been performed, so that the pupil is determined for each living body. This template can be customized to improve the pupil extraction accuracy.

 In addition, it is possible to create or update a pupil template using the pixel value at the center of the pupil and the diameter data of the pupil.

 [0077] In addition, when data relating to the pupil is measured for each frame of the moving image, the data is only a relative value within one frame, but the relative relationship of the distances between multiple points whose positional relationship does not change in time series By correcting the data of each frame according to the above, it becomes possible to make the data relating to the pupil a uniform value among all the frames.

[0078] Further, when data about the pupil is measured in each frame of the moving image, the data is Although it is only a relative value within one frame in pixel units, the absolute distance per pixel is calculated by using the distance or displacement from the living body as a parameter, and the absolute value of data related to the pupil is calculated. It becomes possible.

 [0079] In addition, since the three-dimensional shape of a living body can be measured using only image data without using a distance sensor or a displacement sensor, and the absolute value of data relating to the pupil can be calculated, the apparatus configuration can be simplified. It becomes possible to do.

Further, by irradiating the living body with light at the time of imaging, it is possible to accurately perform imaging by the image capturing unit and improve the measurement accuracy of data related to the pupil.

[0081] In addition, since a living body is imaged using a visible light source and a visible camera, the device configuration without using a special device such as an infrared light source or an infrared force mela is simplified to reduce manufacturing costs. It becomes possible to do.

[0082] The data measuring device 1 of the present embodiment can be used after the concentration of subjects during personal computer work and to determine the degree of fatigue. It can also be used to determine the subject's psychological stress level by adding stimuli other than light stimuli (sound, aroma, incense, footbath, etc.). Furthermore, it can also be used to measure the degree of emotion and attention of a subject who is awarding a movie, a TV program, or news.

As described in detail above, according to the data measuring device and the data measuring method of the present invention, it is possible to measure data relating to the pupil of the living body with high accuracy without contacting the living body.

Claims

The scope of the claims  [1] An image photographing unit for photographing a living body having an eye region and acquiring a first image;  Extracting a first eye region from the first image;  Detecting a pupil from the first eye region with a predetermined template relating to the pupil, measuring data relating to the detected pupil;  Data processing to update a predetermined template with measured data 'Analysis unit, The image capturing unit captures the living body and inputs a second image,  The data processing / analysis unit  Extract the second eye area from the second image,  Detect second eye area pupil with updated template  A data measuring device characterized by that. 2. The data measuring apparatus according to claim 1, wherein the data relating to the pupil is a pixel value at the center of the pupil and diameter data of the pupil. [3] The data processing 'analysis unit acquires a distance between a plurality of points whose positional relationship does not change in time series for each frame, and the pupil for each frame according to a relative relationship of the distances between the plurality of points. 3. The data measuring device according to claim 1, wherein the data relating to the pupil of each of the frames is set to a relative value with respect to a predetermined reference value by correcting the data relating to the frame. . [4] The image photographing unit includes a distance sensor or a displacement sensor that measures a distance or displacement from a living body, and the data processing / analyzing unit relates to the pupil using the distance or displacement from the living body as a parameter. The data measurement device according to any one of claims 1 to 3, wherein an absolute value of data is calculated. 5. The data measuring apparatus according to claim 4, wherein the distance sensor or the displacement sensor is constituted by a plurality of image sensors and measures a three-dimensional shape of a living body. 6. The data measuring device according to claim 5, wherein the image photographing unit is a visible camera.  [7] acquiring a first image by imaging a living body having an eye region; Extracting the first image force first eye region; Detecting a pupil from the first eye region with a predetermined template for the pupil; measuring data about the detected pupil; Updating the predetermined template with the measured data; Photographing the living body and inputting a second image; Extracting a second eye region from the second image; Detecting the second eye region force pupil with the updated template; A data measurement method characterized by comprising:  8. The data measurement method according to claim 7, wherein the data relating to the pupil is a pixel value at the center of the pupil and diameter data of the pupil.