JP2024153324A - Biometric determination device and biometric determination method - Google Patents

Abstract

To provide an organism determination device and an organism determination method capable of determining whether or not a subject is an organism based on an eye image, with a simple constitution.SOLUTION: The organism determination device is a device for adding an organism determination function to an external device which performs authentication. The organism determination device comprises: a light source which generates infrared light; a camera which captures an image of a subject with an optical axis different from the light source; and a controller which controls the light source and the camera. The controller detects an eye of the subject from the image captured with the camera, determines whether or not light is absorbed into the pupil of the eye of the subject, and determines the subject as an organism when determining that the light is absorbed into the pupil of the eye of the subject.SELECTED DRAWING: Figure 3

Description

The present invention relates to a liveness determination device and liveness determination method.

In a facial recognition device, security can be improved by determining whether the subject captured by the camera is a living body in order to prevent impersonation. In addition, if a mask is worn on the face, it becomes difficult to determine whether the subject is a living body using the characteristics of the facial skin surface.

There are methods for determining whether a person is alive by using characteristics of the eyeball. For example, the method described in Patent Document 1 uses images captured by illuminating the subject on the same axis as the camera's optical axis to detect retinal reflexes and determine whether a person is alive.

JP 2005-040591 A

The technology described in Patent Document 1 can determine whether a person is alive by utilizing retinal reflex. However, to align the optical axis of the camera and the light source, a mechanism is required to arrange multiple mirrors and lenses to reflect the light, which increases the thickness and weight. Since a biometric determination device to be mounted on a smartphone or a stationary authentication device is required to be thin and lightweight, a biometric determination method using a different mechanism from that of Patent Document 1 is desired. In other words, there is a demand for a biometric determination device and a biometric determination method that can determine whether a person is alive with a simpler configuration than the technology described in Patent Document 1.

The present invention has been made to solve the above problems. That is, one of the objects of the present invention is to provide a liveness determination device and liveness determination method that can determine whether or not a subject is a live person from an image of the eye with a simple configuration.

In order to solve the above problem, the living body determination device of the present invention is a living body determination device that includes a light source that generates infrared light, a camera that captures an image of a subject with an optical axis different from that of the light source, and a controller that controls the light source and the camera, and the controller is configured to detect the eye of the subject from the image captured by the camera, determine whether or not light is absorbed by the pupil of the eye of the subject, and determine that the subject is living if it is determined that light is absorbed by the pupil of the eye of the subject.

The living body determination method of the present invention is a living body determination method using a light source that generates infrared light, a camera that captures an image of a subject with an optical axis different from that of the light source, and a controller that controls the light source and the camera, and the controller detects the subject's eye from the image captured by the camera, determines whether or not light is being absorbed by the pupil of the subject's eye, and determines that the subject is living if it is determined that light is being absorbed by the pupil of the subject's eye.

According to the present invention, it is possible to determine whether or not an object is a living body from an image of an eye using a simple configuration. Note that the effects described here are not necessarily limited to those described herein, and may be any of the effects described in this disclosure.

FIG. 1 is a diagram illustrating an example of a hardware configuration of a biometric determination device according to the first embodiment. FIG. 2 is a diagram showing the positional relationship between the biometric determination device according to the first embodiment and a subject. FIG. 3 is a functional block diagram showing the configuration of the biometric determination device according to the first embodiment and its peripheral devices and external devices. FIG. 4 shows an example infrared image of a real eye and a histogram of pixels within the inscribed rectangle of the iris region. FIG. 5 shows an example infrared image of a print of an eye and a histogram of pixels within the inscribed rectangle of the iris region. FIG. 6 is a functional block diagram showing the configuration of a living body determination device according to the second embodiment, and its peripheral devices and external devices. FIG. 7 shows an example infrared image of a real eye and a histogram of pixels within the inscribed rectangle of the iris region. FIG. 8 is a functional block diagram showing the configuration of a living body determination device according to the third embodiment, and its peripheral devices and external devices. FIG. 9 is a functional block diagram showing the configuration of a living body determination device according to the fourth embodiment, and its peripheral devices and external devices. FIG. 10 is a flowchart showing the operation of the liveness determination unit of the liveness determination device according to the fourth embodiment.

Each embodiment of the present invention will be described in detail below with reference to the drawings. The biometric determination device according to each embodiment is intended to be installed in a facility (such as a store or office) where consent to capturing images has been obtained for the purpose of determining whether a subject is a living body or not.

Furthermore, such a biometric determination device can be connected to the face recognition device described above, and by adding a biometric determination function to the face recognition device, it is possible to prevent impersonation of a person. However, the technology is not limited to the above, and it can be connected to any device other than a face recognition device, particularly a device that needs to determine whether or not a person is a living body (for example, a drowsy driving monitoring system). Alternatively, it may be used as a stand-alone biometric determination device.

<<First embodiment>>
First, the configuration of a liveness determination device according to a first embodiment of the present invention will be described with reference to Figures 1 to 3. The liveness determination device according to the first embodiment generally has a configuration for illuminating the face of a subject with a light source, and determining whether or not light is absorbed by the pupil from pixel values of the eye area in a face image captured (picked up) by a camera to perform liveness determination.

Figure 1 is a hardware configuration diagram of a biometric determination device 1 according to a first embodiment of the present invention. The biometric determination device 1 includes a controller 11 as a control unit that controls the entire device. The hardware of the controller 11 includes a CPU (Central Processing Unit) 111H, a ROM (Read Only Memory) 112H, a RAM (Random Access Memory) 113H, a camera control unit 114H, a light source control unit 115H, and an output unit 116H. Specific examples of each of these blocks will be described later.

As shown in FIG. 1, the biometric determination device 1 also includes a camera 12 that is connected to the camera control unit 114H in the controller 11 and is capable of capturing an image of a subject and acquiring image information of the subject, and a light source that is connected to the light source control unit 115H and illuminates the subject.

It is preferable that the camera 12 and light source 13 described above are capable of capturing and illuminating infrared light. Using infrared light can prevent the subject from feeling uncomfortable, improving usability. In addition, for example, even if the subject's iris is black in the visible light range, it is possible to determine whether the subject is alive with high accuracy. For example, an infrared LED (light-emitting diode) can be used as the light source 13.

In one non-limiting example, the camera 12 A/D converts an analog image signal captured through optical elements such as a lens and an aperture (not shown) and an image sensor to digital data, and outputs the digital image data to the camera control unit 114H.

The CPU 111H reads and executes various programs stored in the ROM 112H or the RAM 113H. Specifically, the functions of each part of the biometric determination device 1 are realized by the CPU 111H executing the programs.

ROM112H is a storage medium for storing the programs executed by CPU111H and various parameters required for execution.

RAM 113H is a storage medium that serves as a working area for storing various information that is temporarily used by CPU 111H. RAM 113H also functions as a temporary storage area for data used by CPU 111H.

The biometric determination device 1 may be configured to include multiple CPUs 111H, ROMs 112H, and RAMs 113H.

The camera control unit 114H includes an input/output interface (not shown) and inputs (acquires) image data including two-dimensional information from the camera 12 for each frame, and supplies the data to the CPU 111H, etc.

The light source control unit 115H includes an input/output interface (not shown) and controls the light source 13. In this case, if there are multiple light sources, the light source control unit 115H may control each light source independently. Data can be transmitted and received between the light source control unit 115H and the light source 13 using protocols such as USB (Universal Serial Bus), I2C (Inter-Integrated Circuit), SPI (Serial Peripheral Interface), and UART (Universal Asynchronous Receiver Transmitter).

The output unit 116H outputs the results of processing by the CPU 111H to a peripheral device (e.g., a display or speaker) or an external device (e.g., a face recognition device). The results of processing by the CPU 111H can be stored in the ROM 112H or the RAM 113H.

The hardware configuration of the biometric determination device 1 is not limited to the configuration shown in FIG. 1. For example, the CPU 111H, the ROM 112H, and the RAM 113H may be provided separately from the biometric determination device 1. In that case, the biometric determination device 1 may be realized using a general-purpose computer (e.g., a server computer, a personal computer, a smartphone, etc.).

In addition, multiple computers can be connected via a network, with each computer sharing the functions of each part of the biometric determination device 1. On the other hand, one or more of the functions of the biometric determination device 1 can also be realized using dedicated hardware.

Figure 2 is a diagram showing the positional relationship between the biometric determination device 1 and the subject. The camera 12 and light source 13 are installed a width W apart, and the camera 12 takes a picture of the subject from a position at a distance D. The biometric determination device 1 is installed at the height of a person's face, for example, on a stand or a wall. The optical axis ax1 of the camera 12 and the optical axis ax2 of the light source 13 are different. Although Figure 2 shows the camera 12 and light source 13 installed inside the biometric determination device 1, they may be installed outside the biometric determination device 1 via an interface or the like. Also, although Figure 2 shows only one light source, the light source 13 may be composed of multiple light sources.

Figure 3 is a block diagram showing the functional configuration of the biometric determination device 1 and its periphery. The biometric determination device 1 is connected to a peripheral device 2 and an external device 3.

The biometric determination device 1 includes the camera 12 and light source 13 described above in FIG. 1, and, as functions of the controller 11 (CPU 111H, etc.) in FIG. 1, an imaging control unit 101, an eye detection unit 102, a pupil absorption determination unit 103, a biometric determination unit 104, a communication unit 105, and an external interface 106.

The shooting control unit 101 controls the timing of shooting by the camera 12 and the timing of illumination by the light source 13. The shooting control unit 101 starts shooting by receiving an input of shooting timing from the peripheral device 2, for example, via the external interface 106 described below. The shooting control unit 101 first illuminates the light source 13 and shoots with the camera while it is illuminated. The shooting may be performed continuously over multiple frames, or only one frame may be shot. In this example, an example of shooting continuously at, for example, 30 frames/second will be described. When shooting only one frame, the illumination of the light source 13 need only last for one frame.

The timing of shooting that the shooting control unit 101 receives from the peripheral device 2 is input by the user's desired operation, for example, by touch buttons arranged on the display 21 equipped with a touch panel, or by buttons on the remote control 23. As another example, the device can automatically start shooting when sound is detected by the microphone 24, or when the presence of a human is detected by the human sensor 25.

In addition, instead of a human sensor, it is also possible to constantly take pictures with the camera 12, and when a human is detected in the image taken by the camera 12, the camera 12 can output the image. The process of detecting a human from the image taken by the camera 12 may be performed by the camera 12 or by the image taking control unit 101. Deep learning can be used as a method of detecting a human from an image taken by the camera 12. When using deep learning, this can be achieved by having a deep neural network learn, for example, face images of multiple people in advance. In addition, when the biometric determination device 1 is used in a fixed environment, a person can also be detected using the difference between the images of the previous frame and the current frame. For example, each pixel of the previous frame and the current frame can be subtracted, and if there are a certain number of pixels whose subtraction result is equal to or greater than a certain value, it can be determined that a person has appeared in the image.

The camera 12 captures an image of a subject according to the capture timing received from the capture control unit 101, and outputs the captured image to the eye detection unit 102. At this time, all captured images may be output to the eye detection unit 102, or may be output to the eye detection unit 102 only when a human is detected in the image as described above.

The eye detection unit 102 detects the eyes of the subject from within the image input from the camera 12. Eye detection can be achieved, for example, by using deep learning. When using deep learning, this can be achieved by having a deep neural network learn the eyes of multiple people in advance. Pattern matching can also be used. As another method, when the position and direction of the subject's eyes and the position and direction of the camera 12 are fixed, it is also possible to detect a specific position in the image captured by the camera 12 as the eye, without performing the eye detection process described above.

The eye detection unit 102 also outputs an image of the eye area detected from the image input from the camera 12 to the pupil absorption determination unit 103. At this time, it may detect both the left and right eyes and output an image of both eye areas, or it may detect only one of the left and right eyes and output an image of one eye area.

The eye detection unit 102 cuts out an area that includes at least the iris and pupil. The cutout may be a circumscribing rectangle or inscribing rectangle around the iris area as shown in FIG. 4, or a circle that fits the iris area. When cutting out a circumscribing rectangle or inscribing rectangle, for example, the detected eye position may be used as the center to cut out an area with a certain percentage of the distance between the eyes as its width and height. When cutting out a circle, the detected eye position may be used as the center to cut out an area with a certain percentage of the distance between the eyes as its radius.

As another method, the outline of the iris region may be obtained using an edge filter such as a Laplacian filter, and the inscribed rectangle or circumscribed rectangle of the outline, or the inside of the outline, may be cut out.
The pupil absorption determination unit 103 determines whether or not light is absorbed by the pupil and outputs the determination result. The pupil absorption determination unit 103 receives the eye image output by the eye detection unit 102 as an input and determines whether or not light is absorbed by the eye image.

Whether or not light is absorbed by the pupil can be determined by whether or not there are dark pixels in the image (dark pixels that indicate that light is absorbed by the pupil). Figure 4 shows an example of an infrared image of the vicinity of a real human eye and a histogram of the inscribed rectangle of the iris region. Figure 5 shows an example of an infrared image of the vicinity of a real human eye and a histogram of the inscribed rectangle of the iris region. In the case of an infrared image of a real eye, the pupil area becomes dark by absorbing infrared light, so there are pixels with dark pixel values in the histogram, but in the case of a printout, there are no dark pixels. This is because in the case of a printout, even the black parts reflect the light from the light source and a certain amount of light returns to the camera. In this method, the pupil absorption determination unit 103 scans the pixels in the eye area and can determine that light is absorbed by the pupil if there are a certain number or more pixels below a certain value.

Another method for determining whether light is being absorbed by the pupil is to make a determination from the shape of the histogram. Specifically, for example, first the moving average of the histogram is calculated, and the resulting moving average is differentiated to find the maximum value. It can then be determined that light is being absorbed by the pupil from the number of maximum values found and their positions. For example, this can be determined based on conditions such as there being one or more maximum values below a pixel value of 50 and one or more maximum values between pixel values 100 and 200.

As another method, the gradient, kurtosis, skewness, half-width, etc. of the graph can be calculated, and a predictive model such as a decision tree can be used to determine that light is being absorbed by the pupil. In this case, the gradient, etc. can be calculated multiple times by dividing the horizontal axis of the graph. As yet another method, deep learning can be used. When using deep learning, this can be achieved by having a deep neural network learn the inscribed rectangle of the iris area of a real human in advance.

When the eye detection unit 102 detects both eyes, the pupil absorption determination unit 103 determines whether light is being absorbed by the pupil of each eye independently, and outputs the determination results for each eye independently.

The living body determination unit 104 determines whether the subject is a living body or not based on the determination result output by the pupil absorption determination unit 103, and outputs the determination result.

If the pupil absorption determination unit 103 determines that light is being absorbed by the pupil, the living body determination unit 104 determines that the subject is living.

When the eye detection unit 102 outputs an image in which the areas of both eyes have been cut out, the live body determination unit 104 determines whether or not the subject is a live body based on the determination results for both eyes output by the pupil absorption determination unit 103. At this time, the pupil absorption determination unit 103 may determine that the subject is a live body when the pupils of both eyes are absorbing light, or may determine that the subject is a live body when only the pupil of one eye is absorbing light. When determining that the subject is a live body when only the pupil of one eye is absorbing light, it can also determine that the subject is a live body even if, for example, one eye is closed (wink) or noise is mixed into the image of one eye.

The biometric determination unit 104 can also use multiple frames to determine whether or not a subject is a living organism. When multiple frames are used to make a determination, if the pupil absorption determination unit 103 determines that the pupil is absorbing light in at least a certain number of frames among the multiple frames, it can determine that a subject is a living organism. This makes it possible to determine that a subject is a living organism even if the subject is blinking or noise is mixed into the image of a certain frame, for example.

The communication unit 105 communicates with the external device 3 (see FIG. 3). The communication unit 105 transmits the result of the determination by the biometric determination unit 104 to the external device 3.

In the example shown in FIG. 3, the communication unit 105 is configured to perform wireless communication with the external communication unit 31 of the external device 3. Here, the communication means (method) of the communication unit 105 can be wireless communication such as WiFi or Bluetooth (registered trademark). As another example, the communication unit 105 can communicate with the external device 3 via a wired connection. Thus, the biometric determination device 1 can transmit the determination result as to whether the subject is a living body or not to the external device 3 such as a server via the communication unit 105, and cause the external device 3 to perform face recognition, etc.

The external communication unit 31 transmits the data received from the communication unit 105 to the external device 3. The data received by the external communication unit 31 is, for example, the result of the determination by the liveness determination unit 104. In addition, the functions provided by the external device 3 may be provided within the liveness determination device 1.

The external device 3 is preferably an authentication device that requires the result of a biometric determination of the subject. Note that the configuration of such an authentication device is publicly known, so a detailed description thereof will be omitted. In addition, the external device 3 may be various devices other than an authentication device, so long as they require the result of a biometric determination of the subject.

The communication unit 105 can also transmit images captured by the camera 12 to the external device 3. This allows authentication to be performed without separately capturing an image of the subject, when the external device 3 is an image authentication device.

In this example, the external interface 106 transmits and receives electrical signals to and from the peripheral device 2 via a wired cable.

As shown in FIG. 3, the peripheral device 2 includes a display 21, a speaker 22, a remote control 23, a microphone 24, and a human presence sensor 25, and each block (device) is connected to the external interface 106 via a wired cable.

The external interface 106 can also transmit the result of the determination made by the biometric determination unit 104 to the peripheral device 2. For example, the result of the determination made by the biometric determination unit 104 can be displayed on the display 21. In another example, an alarm can be sounded by the speaker 22 based on the result of the determination made by the biometric determination unit 104.

According to the first embodiment, it is possible to determine whether or not a subject is a living body from an image of the eye using an infrared image of the subject. Furthermore, the living body determination device 1 according to the first embodiment does not require a special mechanism for aligning the optical axes of the camera and the light source. In other words, the living body determination device 1 according to the first embodiment can determine whether or not a subject is a living body from an image of the eye with a simple configuration. This allows the living body determination to be performed with a thin and lightweight device, improving the security of the authentication device as the external device 3.

<<Second embodiment>>
A living body determination device 1A according to a second embodiment of the present invention will be described. In the second embodiment, a configuration example in which a corneal reflex determination unit is additionally provided based on the configuration of the living body determination device 1 according to the first embodiment will be described. Note that components having the same configurations and functions as those in the first embodiment are given the same reference numerals, and detailed descriptions thereof will be omitted.

The biometric determination device 1A according to the second embodiment utilizes the phenomenon that the cornea reflects light when a relatively strong light is irradiated onto the eyeball. This reflected image is generally known as the first Purkinje image.

Figure 6 is a functional configuration diagram of a liveness determination device 1A according to the second embodiment, which includes a corneal reflex determination unit 107. As shown in Figure 6, in the liveness determination device 1A, the corneal reflex determination unit 107 is connected after the eye detection unit 102 and before the liveness determination unit 104. Therefore, in the second embodiment, the functions of the eye detection unit 102 and the liveness determination unit 104 differ somewhat from those in the first embodiment, and therefore will be referred to below using similar reference numerals as the eye detection unit 102A and the liveness determination unit 104A.

The eye detection unit 102A in the second embodiment has the same basic function as the eye detection unit 102 in the first embodiment, and cuts out the detected eye area from the image input from the camera 12.

On the other hand, the eye detection unit 102A adds the cut-out eye area to the pupil absorption determination unit 103 and outputs it to the corneal reflex determination unit 107.

The corneal reflex determination unit 107 determines whether the cornea is reflecting light and outputs the determination result. The corneal reflex determination unit 107 receives the image of the eye output by the eye detection unit 102A as input and determines whether light is being reflected by the image of the eye.

Whether the cornea is reflecting light can be determined by whether or not there are bright pixels in the image (bright pixels that indicate that the cornea is reflecting light). Figure 7 shows an example of an infrared image of the area near a real human eye and a histogram of the inscribed rectangle of the iris region. In the case of an infrared image of a real eye, the cornea reflects infrared light, making some parts brighter, so there are pixels with bright pixel values in the histogram, but in the case of a printout, there are no bright pixels. With this method, the corneal reflex determination unit 107 scans the pixels in the eye region, and can determine that the cornea is reflecting light if there are a certain number of pixels above a certain value.

Another method for determining whether the cornea is reflecting light is to make a determination from the shape of the histogram. Specifically, for example, first the moving average of the histogram is calculated, and the resulting moving average is differentiated to find the maximum value. It can then be determined that the cornea is reflecting light from the number of maximum values found and their positions. For example, it can be determined that there is at least one maximum value between pixel values 100 and 200, and at least one maximum value above pixel value 230.

As another method, the gradient, kurtosis, skewness, half-width, etc. of the graph can be calculated, and a predictive model such as a decision tree can be used to determine that light is being absorbed by the pupil. In this case, the gradient, etc. can be calculated multiple times by dividing the horizontal axis of the graph. As yet another method, deep learning can be used. When using deep learning, this can be achieved by having a deep neural network learn the inscribed rectangle of the iris area of a real human in advance.

When the eye detection unit 102A detects both eyes, the corneal reflex determination unit 107 determines whether the cornea of each eye is reflecting light independently, and outputs the determination results independently for each eye.

The liveness determination unit 104A determines whether the subject is a live organism based on the determination results output by the pupil absorption determination unit 103 and the corneal reflex determination unit 107, and outputs the determination result. When the pupil absorption determination unit 103 and the corneal reflex determination unit 107 output the determination results for both eyes independently, the liveness determination unit 104A first performs liveness determination for each of the same eyes output by the pupil absorption determination unit 103 and the corneal reflex determination unit 107, and then outputs a liveness determination result according to the liveness determination results for the left and right eyes. At this time, the subject may be determined to be a live organism only when both the left and right eyes are determined to be live organisms, or may be determined to be a live organism only when only one of the left and right eyes is determined to be live organisms.

When performing a liveness determination for the same eye output by the pupil absorption determination unit 103 and the corneal reflex determination unit 107, it may be determined that a living body is present when both the pupil absorption determination unit 103 determines that the pupil is absorbing light and the corneal reflex determination unit 107 determines that the cornea is reflecting light are satisfied, or it may be determined that a living body is present when either one of the conditions is satisfied.

Furthermore, according to the second embodiment, as in the first embodiment, the liveness determination unit 104 can use multiple frames to determine whether or not a subject is a live person.

The biometric determination device 1A according to the second embodiment has the effect based on the configuration of the first embodiment described above, and by determining the corneal reflex, it is possible to perform biometric determination with higher accuracy.

<<Third embodiment>>
A biometric determination device 1B according to a third embodiment of the present invention will be described. In the third embodiment, a configuration example in which a face determination unit 108 is additionally provided based on the configuration of the biometric determination device 1A according to the second embodiment will be described. Note that components having the same configurations and functions as those in the second embodiment are given the same reference numerals, and detailed descriptions thereof will be omitted.

Figure 8 is a functional configuration diagram of a biometric determination device 1B according to the third embodiment, which includes a face determination unit 108. As shown in Figure 8, in the biometric determination device 1B, the face determination unit 108 is connected after the camera 12 and before the eye detection unit 102A and the external interface. For this reason, in the third embodiment, the functions of the camera 12, eye detection unit 102A, external interface 106, peripheral device 2, display 21, and speaker 22 are somewhat different from those in the second embodiment, so hereinafter, similar reference numerals will be used to refer to the camera 12B, eye detection unit 102B, external interface 106B, peripheral device 2B, display 21B, and speaker 22B.

The camera 12B of the third embodiment has the same basic functions as the camera 12 of the first and second embodiments, and captures an image of a subject according to the capture timing received from the capture control unit 101.

On the other hand, camera 12B outputs the captured image to face determination unit 108 instead of eye detection unit 102B.

The face determination unit 108 calculates the size and orientation of the subject's face included in the image captured by the camera 12. If the calculated size of the subject's face is equal to or larger than a certain value and the orientation of the subject's face is equal to or smaller than a certain value based on the optical axis of the camera (for example, the angle between the orientation of the face and the optical axis of the camera is equal to or smaller than a predetermined threshold angle), the image captured by the camera 12 is output to the eye detection unit 102B. Limiting the size of the subject's face limits the distance between the camera 12B and the subject, and when the distance is large, the image captured by the camera 12 is not output to the eye detection unit 102B. When the distance between the subject and the camera 12B is large, it becomes difficult to observe the reflection of light at the cornea and the absorption of the pupil from the image captured by the camera 12B. By limiting the distance between the subject and the camera 12B, it is possible to prevent a decrease in the accuracy of the live body determination. Also, when the orientation of the face is larger than a certain value, it becomes difficult to observe the reflection of light at the cornea and the absorption of light at the pupil from the image captured by the camera 12B. By limiting the orientation of the subject's face, it is possible to prevent a decrease in the accuracy of biometric determination.

If the calculated size of the subject's face is less than a certain size, the face determination unit 108 outputs information including the fact that the face is small to the external interface 106B.

The external interface 106B is connected to the peripheral device 2B, and outputs the information received from the face determination unit 108, including the fact that the face is small, to either or both of the display 21B and speaker 22B included in the peripheral device 2B.

When display 21B receives information including the fact that the face is small, it instructs the subject to move closer to camera 12B. For example, it can display a sentence such as "Please move closer" on the screen of display 21B, or an illustration instructing the subject to move closer.

If speaker 22B receives information that includes information that the subject has a small face, it instructs the subject to move closer to camera 12B. For example, it can emit a voice message saying, "Please move closer to the camera."

If the calculated orientation of the subject's face relative to the optical axis of the camera is greater than a certain level, the face determination unit 108 outputs information including the fact that the orientation of the face is large to the external interface 106B.

The external interface 106B is connected to the peripheral device 2B, and outputs information received from the face determination unit 108, including information about the large face orientation, to either or both of the display 21B and speaker 22B included in the peripheral device 2B.

When display 21B receives information including the fact that the subject's face is facing forward, it instructs the subject to face the camera 12B. For example, it can display a sentence such as "Please face forward" on the screen of display 21B, or an illustration instructing the subject to face forward.

When speaker 22B receives information that indicates the subject's face is facing a large direction, it instructs the subject to face camera 12B. For example, it can emit a voice message saying, "Please face the camera."

The face determination unit 108 may calculate only the size of the subject's face included in the image captured by the camera 12. In this case, if the calculated size of the subject's face is equal to or larger than a certain value, the image captured by the camera 12B is output to the eye detection unit 102B.

Face determination unit 108 may also calculate only the orientation of the subject's face contained in the image captured by camera 12. In this case, if the orientation of the subject's face is equal to or smaller than a certain value relative to the optical axis of the camera, the image captured by camera 12B is output to eye detection unit 102B.

According to the third embodiment, by limiting the size and orientation of the subject's face, it is possible to prevent a decrease in the accuracy of biometric determination. In addition, when a face recognition device is connected to the biometric determination device 1B, it is expected that a decrease in the accuracy of face recognition can be prevented by limiting the size and orientation of the subject's face.

<<Fourth embodiment>>
In the fourth embodiment, an example of performing a live body determination using two frames of images in which the illumination intensity of the light source is changed while using the configuration of the live body determination device 1A according to the second embodiment as a base will be described. Note that components having the same configurations and functions as those in the second embodiment are given the same reference numerals, and detailed descriptions thereof will be omitted.

Figure 9 is a functional configuration diagram of a biometric determination device 1C according to the fourth embodiment. In the fourth embodiment, the functions of the light source 13, the photography control unit 101, and the biometric determination unit 104 are somewhat different from those in the second embodiment, so hereinafter they will be referred to as the light source 13C, the photography control unit 101C, and the biometric determination unit 104C using similar reference numerals.

The imaging control unit 101C controls the illumination intensity of the light source 13C in addition to the functions of the imaging control unit 101 in the second embodiment. The imaging control unit 101C controls the imaging so that frames in which the light source 13C irradiates strong light (e.g., infrared light) and frames in which the light source 13C irradiates weak light (e.g., infrared light) are alternately captured. For example, the imaging control unit 101C controls the imaging so that frames in which the light source 13C irradiates light (e.g., infrared light) at a first intensity and frames in which the light source 13C irradiates light (e.g., infrared light) at a second intensity that is weaker than the first intensity are alternately captured.

In a frame in which the light source 13C strongly irradiates light, the light source 13C is controlled so that the pupil absorption judging unit 103 judges that the light is absorbed by the pupil in the case of a real eye by a predetermined judging method and a predetermined judging criterion, and the corneal reflex judging unit 107 judges that the cornea is reflecting the light in the case of a real eye (the illumination intensity of the light source 13C (the intensity of the light generated by the light source 13C) is controlled). In a frame in which the light source 13C weakly irradiates light, the light source 13C is controlled so that the pupil absorption judging unit 103 does not judge that the light is absorbed by the pupil in the case of a real eye, and the corneal reflex judging unit 107 does not judge that the cornea is reflecting the light in the case of a real eye by a predetermined judging method and a predetermined judging criterion (the illumination intensity of the light source 13C (the intensity of the light generated by the light source 13C) is controlled). The predetermined judging method is, for example, any of the judging methods described above, and the predetermined judging criterion is a criterion suitable for the judgment.

In a frame in which the light source 13C weakly irradiates light, the light source 13C may be controlled by a predetermined judgment method and a predetermined judgment criterion so that the light intensity is such that the pupil absorption judgment unit 103 judges that the light is absorbed by the pupil in the case of a real eye, and the corneal reflex judgment unit 107 does not judge that the cornea is reflecting light even in the case of a real eye. Also, in a frame in which the light source 13C weakly irradiates light, the light source 13C may not be irradiated. In the following example, an example will be described in which the light source 13C is controlled by a predetermined judgment method and a predetermined judgment criterion so that the light intensity is such that the pupil absorption judgment unit 103 does not judge that the light is absorbed by the pupil in the case of a real eye, and the corneal reflex judgment unit 107 does not judge that the cornea is reflecting light even in the case of a real eye.

The imaging control unit 101C outputs information on the irradiation intensity of the captured frame to the biometric determination unit 104C.

The light source 13C has its light irradiation intensity and irradiation timing controlled by the imaging control unit 101C.

The living body determination unit 104C performs a living body determination using the determination results of the pupil absorption determination unit 103 and the corneal reflex determination unit 107 for two frames, one in which the light source 13C is strongly irradiated and the other in which the light source 13C is weakly irradiated. FIG. 10 is a flowchart showing one specific example of such processing. The processing of the living body determination unit 104C will be described below with reference to FIG. 10.

The biometric determination unit 104C starts processing when a frame with a strong irradiation intensity obtained from the imaging control unit 101C is detected.

In step 401, the biometric determination unit 104C starts processing for a frame with a strong irradiation intensity obtained from the imaging control unit 101C, and therefore receives the determination result of the pupil absorption determination unit 103 for the frame with strong irradiation.

If the pupil absorption determination unit 103 determines that light is absorbed by the pupil in a strongly illuminated frame (step 402, YES), the biometric determination unit 104C proceeds to step 403. On the other hand, if the pupil absorption determination unit 103 determines that light is not absorbed by the pupil (step 402, NO), the biometric determination unit 104C proceeds to step 410.

In step 403, the biometric determination unit 104C receives the determination result of the corneal reflex determination unit 107 for the strongly illuminated frame.

If the corneal reflex determination unit 107 determines that the cornea is reflecting light in a strongly illuminated frame (step 404, YES), the biometric determination unit 104C proceeds to step 405. On the other hand, if the corneal reflex determination unit 107 determines that the cornea is not reflecting light (step 404, NO), the biometric determination unit 104C proceeds to step 410.

In step 405, the biometric determination unit 104C receives the determination result of the pupil absorption determination unit 103 for the weakly illuminated frame. Since the imaging control unit 101C controls the light source 13C to alternately capture frames with strong illumination and frames with weak illumination, in step 405, it is possible to receive the determination result of the pupil absorption determination unit 103 for the weakly illuminated frame.

If the pupil absorption determination unit 103 determines that light is absorbed by the pupil in the weakly illuminated frame (step 406, YES), the biometric determination unit 104C proceeds to step 410. On the other hand, if the pupil absorption determination unit 103 determines that light is not absorbed by the pupil (step 406, NO), the biometric determination unit 104C proceeds to step 407.

In step 407, the biometric determination unit 104C receives the determination result of the corneal reflex determination unit 107 for the weakly illuminated frame.

If the corneal reflex determination unit 107 determines that the cornea is reflecting light in the weakly illuminated frame (step 408, YES), the biometric determination unit 104C proceeds to step 410. On the other hand, if the corneal reflex determination unit 107 determines that the cornea is not reflecting light (step 408, NO), the biometric determination unit 104C proceeds to step 409.

In step 409, the biometric determination unit 104C determines that the subject is a biometric subject.

In step 410, the biometric determination unit 104C determines that the subject is not a biometric subject.

The processing for frames with strong illumination (S401 to S404) and the processing for frames with weak illumination (S405 to S408) may be interchanged. Furthermore, if the processing is for the same frame, the processing for the judgment results of the pupil absorption judgment unit 103 (S401 and S402, S405 and S406) and the processing for the judgment results of the corneal reflex judgment unit 107 (S403 and S404, S407 and S408) may be interchanged.

Also, multiple pairs of frames with strong and weak illumination may be used to perform liveness determination.

According to the fourth embodiment, a more accurate live body determination can be performed by alternately switching between frames that strongly illuminate the light source 13C and frames that weakly illuminate the light source 13C. For example, even if a print that reproduces pupil absorption and corneal reflex can be prepared by applying a special paint to the print or processing (drilling holes) specific locations on the print, it will be determined that pupil absorption and corneal reflex occur even in frames that are weakly illuminated, so that such a print can be determined not to be a live body.

<<Modifications>>
The present invention is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

The present invention can also be configured as follows:

[1]
A light source that generates infrared light;
a camera that captures an image of a subject with an optical axis different from that of the light source;
A controller for controlling the light source and the camera;
A liveness determination method using
The controller
Detecting the eyes of a subject from an image captured by the camera;
determining whether light is being absorbed by the pupil of the subject's eye;
determining that the subject is a living body when it is determined that light is absorbed by the pupil of the subject's eye;
Biometric determination method.

[2]
The method for determining a live body according to [1],
By the controller,
determining whether a cornea of the subject's eye is reflecting light;
determining that the subject is a living body when it is determined that light is absorbed by the pupil of the subject's eye and that the cornea of the subject's eye is reflecting light;
Biometric determination method.

1, 1A, 1B, 1C... biometric determination device, 2, 2B... peripheral device, 3... external device, 11... controller, 12... camera, 13, 13C... light source, 111H... CPU, 112H... ROM, 113H... RAM, 114H... camera control unit, 115H... light source control unit, 116H... output unit, 101, 101C... photography control unit, 102, 102A, 102B... eye detection unit, 103... pupil absorption determination unit, 104, 104A, 104C... biometric determination unit, 105... communication unit, 106, 106B... external interface, 107... corneal reflex determination unit, 108... face determination unit

Claims (13)

A light source that generates infrared light;
a camera that captures an image of a subject with an optical axis different from that of the light source;
A controller for controlling the light source and the camera;
A biometric determination device comprising:
The controller:
Detecting the eyes of the subject from an image captured by the camera;
determining whether light is being absorbed by the pupil of the subject's eye;
determining that the subject is a living body when it is determined that light is absorbed by the pupil of the subject's eye;
It was configured as follows:
Biometrics device. The liveness determination device according to claim 1,
The controller:
determining whether a cornea of the subject's eye is reflecting light;
determining that the subject is a living body when it is determined that light is absorbed by the pupil of the subject's eye and that the cornea of the subject's eye is reflecting light;
It was configured as follows:
Biometrics device. The liveness determination device according to claim 1,
The controller:
The light source is configured to be controllable so as to irradiate the infrared light with either a first intensity or a second intensity that is weaker than the first intensity;
The controller:
from a first image captured by the camera in a state in which the light source irradiates the infrared light at the first intensity and a second image captured by the camera in a state in which the light source irradiates the infrared light at the second intensity that is weaker than the first intensity;
determining whether a pupil of the subject's eye in the first image is absorbing light;
determining whether a cornea of the subject's eye in the first image is reflecting light;
determining whether a pupil of the subject's eye in the second image is absorbing light;
determining whether a cornea of the subject's eye in the second image is reflecting light;
determining that the subject is a living subject when it is determined that the pupil of the subject's eye in the first image is absorbing light, and the cornea of the subject's eye in the first image is reflecting light, and the pupil of the subject's eye in the second image is not absorbing light, and the cornea of the subject's eye in the second image is not reflecting light;
It was configured as follows:
Biometrics device. The liveness determination device according to claim 1,
The controller:
The light source is configured to be controllable so as to be in either a first state in which the infrared light is irradiated at a predetermined intensity or a second state in which the infrared light is not irradiated,
The controller:
from a first image captured by the camera when the light source is in the first state and a second image captured by the camera when the light source is in the second state,
determining whether a pupil of the subject's eye in the first image is absorbing light;
determining whether a cornea of the subject's eye in the first image is reflecting light;
determining whether a pupil of the subject's eye in the second image is absorbing light;
determining whether a cornea of the subject's eye in the second image is reflecting light;
determining that the subject is a living subject when it is determined that the pupil of the subject's eye in the first image is absorbing light, and the cornea of the subject's eye in the first image is reflecting light, and the pupil of the subject's eye in the second image is not absorbing light, and the cornea of the subject's eye in the second image is not reflecting light;
It was configured as follows:
Biometrics device. The liveness determination device according to claim 1,
The controller:
A liveness determination is performed using a plurality of images captured by the camera at different times.
It was configured as follows:
Biometrics device. The liveness determination device according to claim 1,
A peripheral device including at least one of a display and a speaker;
The controller:
Detecting the size of the subject's face from the image captured by the camera;
when the size of the face is equal to or smaller than a certain size, instructing the subject to move closer to the camera by the peripheral device;
It was configured as follows:
Biometrics device. The liveness determination device according to claim 1,
A peripheral device including at least one of a display and a speaker;
The controller:
Detecting the orientation of the subject's face from the image captured by the camera;
instructing the subject to face the front of the camera by the peripheral device when the orientation of the face is equal to or greater than a certain value with respect to the optical axis of the camera;
It was configured as follows:
Biometrics device. The liveness determination device according to claim 1,
The controller:
determining whether light is being absorbed in the pupil of the subject's eye based on the presence of dark pixels indicative of light being absorbed in the pupil;
It was configured as follows:
Biometrics device. The liveness determination device according to claim 1,
The controller:
determining whether light is being absorbed by the pupil based on the shape of the histogram of the subject's eye;
It was configured as follows:
Biometrics device. The liveness determination device according to claim 2,
The controller:
determining whether the cornea of the subject's eye is reflecting light based on whether there is a bright pixel indicating that the cornea is reflecting light;
It was configured as follows:
Biometrics device. The liveness determination device according to claim 2,
The controller:
determining whether the cornea is reflecting light or not based on the shape of the histogram of the subject's eye;
It was configured as follows:
Biometrics device. A light source that generates infrared light;
a camera that captures an image of a subject with an optical axis different from that of the light source;
A controller for controlling the light source and the camera;
A liveness determination method using
The controller
Detecting the eyes of a subject from an image captured by the camera;
determining whether light is being absorbed by the pupil of the subject's eye;
determining that the subject is a living body when it is determined that light is absorbed by the pupil of the subject's eye;
Biometric determination method. The liveness determination method according to claim 12,
The controller
determining whether a cornea of the subject's eye is reflecting light;
determining that the subject is a living body when it is determined that light is absorbed by the pupil of the subject's eye and that the cornea of the subject's eye is reflecting light;
Biometric determination method.

Images