HK1227139B - Methods and systems for biometric authentication - Google Patents

Description

Method and system for biometric authentication

Information about divisional applications

This application is a divisional application. The parent application is an invention patent application filed on July 2, 2013, with application number 201310276024.5, and titled “Method and System for Texture Features for Biometric Verification.”

Technical Field

The present invention relates to biometric authentication based on images of the eyes.

Background Art

It is often necessary to restrict access to a property or resource to specific individuals. Biometric recognition systems can be used to verify the identity of an individual to grant or deny access to a resource. For example, an iris scanner can be used by a biometric security system to identify an individual based on the unique structure in their iris.

Summary of the Invention

This specification describes techniques related to biometric authentication based on images of an eye. Generally speaking, one aspect of the subject matter described in this specification can be embodied in a method that includes obtaining one or more image regions from a first image of an eye, wherein each of the image regions includes a view of a respective portion of the eye's vasculature outside of a limbal boundary of the eye. The method can further include applying a number of distinct filters to each of the image regions to generate a plurality of respective descriptors for the regions. The number of distinct filters can include a plurality of convolution filters that are each configured to describe one or more aspects of the eye's vasculature and that, in combination, describe the visible eye vasculature in a feature space. The method can further include determining a match score based on the generated descriptors and based on one or more descriptors associated with a second image of the eye's vasculature.

In general, one aspect of the subject matter described in this specification can be embodied in a system comprising a module configured to obtain one or more image regions from a first image of an eye, wherein each of the image regions includes a view of a respective portion of the eye's vasculature outside of a limbal boundary of the eye. The system can further include means for applying a number of distinct filters to each of the image regions to generate a plurality of respective descriptors for the regions. The number of distinct filters can include a plurality of convolutional filters, each configured to describe one or more aspects of the eye's vasculature and, in combination, describing the visible eye vasculature in a feature space. The system can further include a module configured to determine a match score based on the generated descriptors and based on one or more descriptors associated with a second image of the eye's vasculature.

In general, one aspect of the subject matter described in this specification can be embodied in a system comprising a data processing apparatus and a memory coupled to the data processing apparatus. The memory has stored thereon instructions that, when executed by the data processing apparatus, cause the data processing apparatus to perform operations comprising obtaining one or more image regions from a first image of an eye, wherein each of the image regions comprises a view of a respective portion of the eye's vasculature outside a limbal boundary of the eye. The operations can further comprise applying a number of distinct filters to each of the image regions to generate a plurality of respective descriptors for the regions. The number of distinct filters can comprise a plurality of convolution filters, each configured to describe one or more aspects of the eye's vasculature and, in combination, describing the visible eye vasculature in a feature space. The operations can further comprise determining a match score based on the generated descriptors and based on one or more descriptors associated with a second image of the eye's vasculature.

In general, one aspect of the subject matter described in this specification can be embodied in a non-transitory computer-readable medium storing software comprising instructions executable by a processing device that, upon execution, cause the processing device to perform operations comprising obtaining one or more image regions from a first image of an eye, wherein each of the image regions comprises a view of a respective portion of the eye's vasculature outside a limbal boundary of the eye. The operations may further comprise applying a plurality of distinct filters to each of the image regions to generate a plurality of respective descriptors for the regions. The plurality of distinct filters may comprise a plurality of convolutional filters, each configured to describe one or more aspects of the eye's vasculature and, in combination, describing the visible eye vasculature in a feature space. The operations may further comprise determining a match score based on the generated descriptors and based on one or more descriptors associated with a second image of the eye's vasculature.

These and other embodiments may each optionally include one or more of the following features. The visible eye vasculature may be described by some of the generated descriptors based on co-occurrence statistics of image data elements in one or more image regions. The visible eye vasculature may be described by some of the generated descriptors based on information-theoretic statistics of image data elements in one or more image regions. The visible eye vasculature may be described locally or globally by some of the generated descriptors based on one or more non-convolutional statistical derivatives of the image data elements. One of the convolution filters may be a Gabor filter. One of the convolution filters may be a wavelet transform. One of the plurality of filters may be a nonlinear filter. The nonlinear filter may be a trained neural network. Applying the plurality of distinct filters to each of the image regions to generate a plurality of corresponding descriptors for the regions may include combining the corresponding outputs of one or more of the filters. Applying the plurality of distinct filters to each of the image regions to generate a plurality of respective descriptors for the regions may include calculating respective magnitudes of outputs of one or more of the filters. Determining a match score may include: combining the respective descriptors for each of the regions into respective vectors for the regions; comparing, for one or more of the regions of the first image, the respective vectors for the regions with vectors derived from descriptors of corresponding regions of the second image to generate respective similarity scores; and determining the match score based at least on the generated similarity scores. A determination may be made as to whether the match score exceeds a value, wherein the value is based at least on a robust threshold in a neighborhood of a three-dimensional receiver operating curve generated from a sensitivity analysis. The image data element may be one of a voxel, a pixel, a ray, or a red, green, or blue channel value. The image region may include one or more continuous or discontinuous image data elements. Obtaining the one or more image regions may include tiling a region to obtain a smaller region.

Certain embodiments of the present invention may be implemented to achieve none, one, or more of the following advantages. Some embodiments may provide security by reliably authenticating individuals. Some embodiments may reduce the noise sensitivity of the authentication system compared to systems based on minutiae detection. Some embodiments may facilitate efficient storage and retrieval of enrollment profiles for authentication.

The details of one or more embodiments of the present invention are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the present invention will be apparent from the description, drawings, and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a diagram of the anatomy of the human eye.

2 is a diagram including an example image showing a portion of the vasculature of the white of the eye.

3 is a diagram of an example image segmented for analysis.

4 is a block diagram of an example security system configured to authenticate an individual based at least in part on one or more images of the white of the eye.

FIG5 is a block diagram of an example online environment.

6 is a flow diagram of an example process for authenticating an individual based on one or more images of the white of the eye.

7 is a flow diagram of an example process for determining a matching score for one or more images of an eye.

8A-8B are flow diagrams of an example process for determining a matching score for one or more images of an eye.

9 shows an example of a computer device and a mobile computer device that can be used to implement the techniques described here.

DETAILED DESCRIPTION

Unique features of the visible vasculature in the whites of an individual's eyes can be used to identify or authenticate the individual. For example, an image of the whites of a user's eyes can be obtained and analyzed to compare the features of the eyes with reference records in order to authenticate the user and grant or deny access to a resource.

The unique structure of an individual's visible vasculature can be reflected in the texture features of an image of the white of the individual's eye. The image can be segmented to identify regions on the white of the eye (e.g., a set of small tiles arranged in a grid to cover the area to the left or right of the iris) for texture analysis. A set of filters can be applied to determine descriptors of the texture features of the individual's vasculature in these small regions. For example, some of the applied filters can be convolution filters, which form the basis of a feature space that can be used to describe the significant texture features of the individual's vasculature. A vector of descriptors derived from the filter outputs can be combined into a descriptor vector.

During a verification or identification operation, a descriptor vector determined by a user may be compared with a corresponding descriptor vector from a reference record of a registered individual. The descriptor vectors may be compared by determining a match score that reflects the likelihood of a match between the user and the registered individual. For example, the match score may be determined as the distance between the two descriptor vectors. In some embodiments, the match score is determined as the output of a trained function approximator (e.g., a neural network) in response to being passed as input in the two descriptor vectors. The match score may be compared to one or more thresholds to determine whether to accept or reject the user.

FIG1 is a diagram of the anatomy of a human eye 100. The diagram is a cross-section of the eye, with a magnified view 102 of the anatomy near the limbal boundary of the eye, which separates the colored iris 110 from the surrounding white of the eye. The white of the eye contains a complex vascular structure that is not only easily visible from the outside of the eye and scannable, but also unique and varies between individuals. Therefore, due primarily to the vasculature of the conjunctiva and episclera, these vascular structures of the white of the eye can be scanned and advantageously used as a biometric feature. This biometric feature can be used to authenticate a specific individual, or to identify an unknown individual.

The white of the eye has several layers. The sclera 120 is the eye's opaque, fibrous protective layer, containing collagen and elastic fibers. The sclera 120 is covered by the episclera 130, which has a significant number of blood vessels and veins running through and on it. The episclera 130 is covered by the bulbar conjunctiva 140, a thin, transparent membrane that interfaces with the eyelid 150 or, when the eyelid is open, with the environment. Blood vessels and veins pass through all of these layers of the white of the eye and can be detected in images of the eye. The eye also contains eyelashes 160, which can sometimes obscure portions of the white of the eye in an image.

FIG2 is a diagram including an example image 200 showing a portion of the vasculature of the white of the eye. This image 200 may be captured using a sensor (e.g., a camera) integrated into a computing device, such as a smartphone, tablet, television, laptop, or personal computer. For example, a user may be prompted, via a display or audio prompt, to look to the left when the image is captured, thereby exposing a larger area of the white of the eye to the right of the iris to the sensor's field of view. Similarly, the user may be prompted to look to the right, up, down, forward, etc., when the image is captured. The example image includes a view of the iris 220, with the pupil 210 at its center. The iris 220 extends to the limbus boundary 225 of the eye. The white of the eye 230 is outside the limbus boundary 225. The gross vasculature 240 of the white of the eye is visible in image 100. This vasculature 240 may be unique to an individual. In some embodiments, unique characteristics of the vasculature 240 may be used as a basis for identifying, verifying, or authenticating an individual user.

FIG3 is a diagram including an example image 300 showing portions of the vasculature of the whites of two eyes, the image being segmented for analysis. Captured image 310 can be obtained in a variety of ways. Captured image 310 can be pre-processed and segmented to isolate a region of interest within the image and enhance the view of the vasculature in the whites of the eyes. For example, the region of interest can be a tiled portion forming a grid covering some or all of the whites of the eyes. Portion 320 corresponding to the white of the right eye, to the left of the iris, can be isolated, for example, by identifying the limbus boundary and the edge of the eyelid. Similarly, portion 322 corresponding to the white of the left eye, to the left of the iris, can be isolated. Pre-processing can be used to enhance the view of the vasculature in this region, for example, by selecting a component color from the image data that maximizes contrast between the vasculature of the white of the eye and the surrounding white portions. In some embodiments, these portions of the image 320, 322 can be further segmented into tiles forming a grid 330, 332 that divides the exposed surface area of the white of the eye into smaller regions for analysis purposes. Features of the vasculature in these regions of interest can be used to identify or verify an individual.

FIG4 is a block diagram of an example security system 400 configured to authenticate an individual based at least in part on one or more images of the white of an eye 410. A user of security system 400 can present their eye 410 to a light sensor 420. In this manner, one or more images of the white of the eye 410 can be captured. Digital cameras, three-dimensional (3D) cameras, and light field sensors are examples of light sensors that can be used. Light sensor 420 can use a variety of technologies, such as a digital charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). In some implementations, the user can be prompted to perform certain gestures to expose portions of the white of the eye 410 and facilitate image acquisition via a message displayed on display 424. For example, the user can be prompted to orient their gaze so that the iris of their eye 410 turns left, right, upward to the left, and upward to the right. In some implementations (not shown), the user can be prompted to perform certain gestures via a message played through a speaker, via an indicator light (e.g., an LED), or not at all.

In some implementations, the sensor 420 can be configured to detect when the eye 410 is properly positioned in the sensor's field of view. Alternatively, software or hardware implemented on the computing device 430 can analyze one or more images generated by the light sensor 420 to determine whether the eye 410 is properly positioned. In some implementations, the user can manually indicate when the eye 410 is properly positioned through a user interface (e.g., a button, keyboard, keypad, touchpad, or touch screen).

The verification module 440 implemented on the computing device 430 can obtain one or more images of the white of the eye via the light sensor 420. In some implementations, the computing device 430 is integrated with or electronically coupled to the light sensor 420. In some implementations, the computing device 430 can communicate with the light sensor 420 through a wireless interface (e.g., an antenna).

Verification module 440 processes the image obtained via light sensor 420 to control access to security device 450. For example, verification module 440 may implement the verification process described with respect to FIG6. In some implementations, security device 450 may include an actuator 460 (e.g., a locking mechanism) that affects access control instructions from verification module 440.

The computing device can be integrated or interfaced with safety device 450 in a variety of ways. For example, safety device 450 can be a car, light sensor 420 can be a camera integrated into the car's steering wheel or dashboard, and computing device 430 can be integrated into the car and electrically connected to the camera and an ignition lock system acting as safety actuator 460. The user can present a view of the whites of their eyes to the camera in order to be authenticated as an authorized driver of the car and start the engine.

In some implementations, the security device 450 may be a real estate key box, the light sensor 420 may be a camera integrated with a user's mobile device (e.g., a smartphone or tablet device), and the processing of the verification module 440 may be performed in part by the user's mobile device and in part by a computing device integrated with the key box that controls the power locking mechanism. The two computing devices may communicate via a wireless interface. For example, a user (e.g., a real estate agent giving a property showing) may use the camera on their mobile device to obtain one or more images and submit data based on the images to the key box in order to be verified as an authorized user and permitted to use the keys stored in the key box.

In some embodiments, the security device 450 is a gate or door that controls access to a property. The light sensor 420 can be integrated into the door or gate or positioned on a wall or fence near the door or gate. The computing device 430 can be positioned near the light sensor 420 and the electric locking mechanism in the door or gate that acts as an actuator 460 and can communicate with the light sensor 420 and the electric locking mechanism via a wireless interface. In some embodiments, the security device 450 can be a rifle, and the light sensor 420 can be integrated with a scope attached to the rifle. The computing device 430 can be integrated into the stock of the rifle and can be electrically connected to the light sensor 420 and the trigger or firing pin locking mechanism that acts as an actuator 460. In some embodiments, the security device 450 can be a piece of rental equipment (e.g., a bicycle).

Computing device 430 may include processing device 432 (e.g., as described with respect to FIG. 9 ) and a machine-readable repository or database 434. In some implementations, the machine-readable repository may include flash memory. Machine-readable repository 434 may be used to store one or more reference records. The reference record may include data derived from one or more images of the whites of the eyes of registered or authorized users of security device 450. In some implementations, the reference record includes the entire reference image. In some implementations, the reference record includes features extracted from the reference image. In some implementations, the reference record includes encrypted features extracted from the reference image. In some implementations, the reference record includes an identification key encrypted from features extracted from the reference image. To establish a reference record for a new user, an enrollment or registration process may be performed. The enrollment process may include capturing one or more reference images of the whites of the eyes of the newly registered user. In some implementations, the enrollment process may be performed using light sensor 420 and processing device 430 of verification system 400.

5 is a block diagram showing an example of a network environment 500 in which the techniques described herein may be implemented. The network environment 500 includes computing devices 502, 504, 506, 508, 510 configured to communicate with a first server system 512 and/or a second server system 514 via a network 511. The computing devices 502, 504, 506, 508, 510 have respective users 522, 524, 526, 528, 530 associated therewith. The first and second server systems 512, 514 each include computing devices 516, 517 and machine-readable repositories or databases 518, 519. The example environment 500 may include thousands of websites, computing devices, and servers, which are not shown.

The network 511 may comprise a large computer network, examples of which include a local area network (LAN), a wide area network (WAN), a cellular network, or a combination thereof, connecting a plurality of mobile computing devices, fixed computing devices, and server systems. The networks included in the network 511 may provide communication in various modes or protocols, examples of which include transmission control protocol/Internet protocol (TCP/IP), global system for mobile communications (GSM) voice calling, short electronic message service (SMS), enhanced messaging service (EMS), or multimedia messaging service (MMS) messaging, Ethernet, code division multiple access (CDMA), time division multiple access (TDMA), personal digital cellular (PDC), wideband code division multiple access (WCDMA), CDMA2000, or general packet radio system (GPRS), among others. Communication may occur via a radio frequency transceiver. In addition, short-range communication may occur, for example, using Bluetooth, WiFi, or other such transceiver systems.

Computing devices 502, 504, 506, 508, 510 enable respective users 522, 524, 526, 528, 530 to access and view documents, such as web pages included in a website. For example, user 522 of computing device 502 may use a web browser to view the web page. The web page may be provided to computing device 502 by server system 512, server system 514, or another server system (not shown).

In example environment 500, computing devices 502, 504, 506 are illustrated as desktop computing devices, computing device 508 is illustrated as a laptop computing device 508, and computing device 510 is illustrated as a mobile computing device. However, it is noted that computing devices 502, 504, 506, 508, 510 may include, for example, a desktop computer, a laptop computer, a handheld computer, a television having one or more processors embedded therein and/or coupled thereto, a tablet computing device, a personal digital assistant (PDA), a cellular telephone, a network appliance, a camera, a smartphone, an Enhanced General Packet Radio Service (EGPRS) mobile phone, a media player, a navigation device, an electronic messaging device, a game console, or a combination of two or more of these data processing devices or other suitable data processing devices. In some implementations, a computing device may be included as part of a motor vehicle (e.g., an automobile, an emergency vehicle (e.g., a fire truck, an ambulance), a bus).

Users interacting with computing devices 502, 504, 506, 508, 510 can interact with a secure transaction service 523, for example, hosted by server system 512, by authenticating themselves and issuing instructions or commands via network 511. Secure transactions may include, for example, e-commerce purchases, financial transactions (e.g., online banking transactions, credit or bank card transactions, loyalty points redemption), or online voting. The secure transaction service may include an authentication module 525 that coordinates authentication of the user from the secure server side of the transaction. In some implementations, the authentication module 525 may receive image data from a user device (e.g., computing device 502, 504, 506, 508, 510) that includes one or more images of the eyes of a user (e.g., user 522, 524, 526, 528, 530). The authentication module may then process the image data to authenticate the user by determining whether the image data matches a reference record of a recognized user identity that was previously established based on image data collected during the enrollment session.

In some embodiments, a user who has submitted a service request may be redirected to a verification module 540 running on a separate server system 514. The verification module 540 may maintain a reference record of registered or enrolled users of the secure transaction service 523 and may also include reference records of users of other secure transaction services. The verification module 540 may use encrypted network communications (e.g., using a public key encryption protocol) to establish a secure session with various secure transaction services (e.g., secure transaction service 523) to indicate to the secure transaction service whether the user is verified as a registered or enrolled user. Much like the verification module 525, the verification module 540 may receive image data from the requesting user's computing device (e.g., computing devices 502, 504, 506, 508, 510) and may process the image data to authenticate the user. In some embodiments, the verification module may determine texture features of an image received from a user and may accept or reject the image based on the texture features.

The verification module 540 may be implemented as software, hardware, or a combination of software and hardware executing on a processing apparatus, such as one or more computing devices (such as the computer system illustrated in FIG. 9 ).

A user device (e.g., computing device 510) may include an authentication application 550. Authentication application 550 may facilitate authentication of the user as a registered or enrolled user identity for accessing secure services (e.g., secure transaction service 523) via network 511. For example, authentication application 550 may be a mobile application or another type of client application for interacting with a server-side authentication module (e.g., authentication module 540). Authentication application 550 may drive a sensor (e.g., a camera connected to or integrated with the user computing device) to capture one or more images of a user (e.g., user 530), including a view of the whites of the user's eyes. Authentication application 550 may prompt the user (e.g., via a display or speaker) to pose for image capture. For example, the user may be prompted to face the sensor and direct their gaze to the left or right, exposing a significant portion of the whites of their eyes to the sensor.

In some implementations, the verification application 550 transmits the captured image data to a verification module (e.g., verification module 525 or 540) on a remote server (e.g., server system 512 or 514) via the network 511. Collecting image data from the user can facilitate registration and establishing a reference record for the user. Collecting image data from the user can also facilitate verifying the user's identity against the reference record.

In some embodiments, additional processing of the image data may be performed by the verification application 550 for verification purposes, and the results of that processing may be transmitted to a verification module (e.g., verification module 525 or 540). In this way, verification functionality may be distributed between client-side and server-side processing in a manner suitable for a particular application. For example, in some embodiments, the verification application 550 determines texture features of captured images and rejects any images. The texture features may be transmitted to a server-side verification module (e.g., verification module 525 or 540) for further analysis.

In some embodiments, the authentication application accesses a reference record of the user's identity and performs a full authentication process, then reports the results (eg, whether the user was accepted or rejected) to the server-side authentication module.

The authentication application 550 may be implemented as software, hardware, or a combination of software and hardware executing on a processing apparatus, such as one or more computing devices (such as the computer system illustrated in FIG. 9 ).

FIG6 is a flow chart of an example process 600 for verifying an individual based on one or more images of the white of the eye. Texture features or descriptors are determined for the obtained image by applying a set of filters to the image. A match score is determined by comparing the determined features with a reference record. The individual is then accepted or rejected based on the match score.

For example, process 600 may be implemented by verification module 440 in computing device 430 of FIG. 4 . In some embodiments, computing device 430 is a data processing device comprising one or more processors configured to perform the actions of process 600. For example, the data processing device may be a computing device (e.g., as illustrated in FIG. 9 ). In some embodiments, process 600 may be implemented in whole or in part by verification application 550, which is executed by a user computing device (e.g., computing device 510). For example, the user computing device may be a mobile computing device (e.g., mobile computing device 950 of FIG. 9 ). In some embodiments, process 600 may be implemented in whole or in part by verification module 540, which is executed by a user server system (e.g., server system 514). In some embodiments, server system 514 is a data processing device comprising one or more processors configured to perform the actions of process 600. For example, the data processing device may be a computing device (e.g., as illustrated in FIG. 9 ). In some implementations, the computer-readable medium may include instructions that, when executed by a computing device (eg, a computer system), cause the device to perform the actions of processor 600 .

One or more images of an eye are obtained 602. The images include a view of a portion of the eye's vasculature outside the limbus boundary of the eye. The obtained images may be monochrome or represented in various color spaces (e.g., RGB, SRGB, HSV, HSL, or YCbCr). In some implementations, the images may be obtained using a light sensor (e.g., a digital camera, a 3D camera, or a light field sensor). The sensor may be sensitive to light in various wavelength ranges. For example, the sensor may be sensitive to the visible spectrum of light. In some implementations, the sensor is paired with a flashlight or a flashlight that can be pulsed to illuminate an object in the sensor's view. The image capture may be synchronized with or time-locked to the pulsation of the flashlight. In some implementations, the sensor captures a sequence of images that can be used to track the movement of an object within the sensor's field of view. The sensor may include one or more settings that control image capture (e.g., focus, flash intensity, exposure, and white balance). The images may collectively include multiple focal lengths. For example, a sequence of images may be captured, each image captured with a different focal length setting of the sensor, and/or some sensors (e.g., light field sensors) may capture images focused at multiple distances from the sensor. In some implementations, the one or more images may be obtained 502 by being received via a network interface (e.g., a network interface of the server system 514).

The one or more images may be segmented 604 to identify a region of interest that includes an optimal view of the vasculature in the white of the eye. In some embodiments, anatomical landmarks (e.g., the iris, its center and limbal border, the canthus of the eye, and the edge of the eyelid) may be identified in the one or more images. The region of interest within the image may be identified and selected based on its position relative to the identified anatomical landmarks. For example, the region of interest may be located to the left, right, above, or below the iris in the white of the eye. In some embodiments, the selected region of interest is tiled to form a grid covering a larger portion of the white of the eye. Tiles may be registered (i.e., their positions are aligned) relative to the iris center and iris border so that the same tile positions can be compared across the images. In some embodiments, the selected region of interest of the image is discontinuous (e.g., adjacent regions may overlap or there may be space between adjacent regions). The selected region of interest may correspond to a region of interest selected from a reference image (on which the data in the reference record is based).

In some embodiments, the canthus is found by fitting curves to the selected portion of the eyelid on the sclera, and then extrapolating and finding the intersection of those curves. If an intersection point (the canthus) cannot be found due to the fact that the sclera is too close (e.g., due to gaze direction), a template from the same canthus region but from a photo of the opposite gaze direction can be derived and applied to the problematic canthus neighborhood in the image at hand, and the maximum correlation position can be marked as the canthus.

In some embodiments, the eyelid is found by an adaptive thresholding method that finds the white of the eye from the image, which borders the eyelid.The scleral mask itself can be corrected by morphological operations (e.g., convex hull) to remove aberrations.

In some embodiments, the limbal boundary is found from the scleral mask, which is where the sclera ends because it ends at the iris limbal boundary.

In some implementations, the iris center is found through a variety of methods. If the eye is bright in color, the center of the pupil can be found as the iris center. If the iris is too dark, the center of an ellipse fitted to the limbus boundary and its center is found, or it is determined as the focus of normal rays (i.e., lines perpendicular to the tangents to the limbus boundary) converging around the iris center, or a combination of these methods.

The image region may be pre-processed 606 to enhance the view of the vasculature within the image. In some embodiments, pre-processing 606 includes color image enhancement and contrast-limited adaptive histogram equalization (CLAHE), which enhances the contrast of the intensity image. CLAHE operates on small regions of the image, called tiles. The contrast of each tile is enhanced so that the output histogram approximately matches a histogram specified by a particular distribution (e.g., uniform, exponential, or Rayleigh). Adjacent tiles are then combined using bilinear interpolation to eliminate artifactual boundaries. In some embodiments, the image may be enhanced by selecting the red, green, or blue component that provides the best contrast between the blood vessels and the background. The green component may be preferred because it provides the best contrast between the blood vessels and the background.

In some embodiments, preprocessing 606 includes applying a multi-scale enhancement filtering scheme to enhance the intensity of the image, thereby facilitating the detection of vascular structures and subsequent feature extraction. The parameters of the filter can be determined empirically to take into account variations in the girth of the blood vessels. The algorithm used can have good curve sensitivity, good curve specificity, and suppress objects of other shapes. The algorithm can be based on the second-order derivative of the image. First, since the second-order derivative is sensitive to noise, the image segment is convolved with a Gaussian function. The parameter σ of the Gaussian function can correspond to the thickness of the blood vessel. Next, for each image data element, a Hessian matrix can be established, and the eigenvalues λ1 and λ2 can be calculated. In each Hessian matrix, the matrix ridge is defined as the point where the image has an extreme value in the curvature direction. The curvature direction is the eigenvector of the second-order derivative of the image, which corresponds to the maximum absolute eigenvalue λ. The sign of the eigenvalue determines whether it is a local minimum λ>0 or a maximum λ>0. The calculated eigenvalues are then used to filter the blood vessel lines using the following equation:

I_line(λ1,λ2)=|λ1|-|λ2| (if λl<0), and I_line(λ1,λ2)=0 (if λ1≥0)

The diameter of the blood vessels varies, but the algorithm assumes that the diameter is in the interval [d0, d1]. A Gaussian smoothing filter with a scale of [d0/4, d1/4] can be used. This filtering can be repeated N times based on the following smoothing scale:

σ1＝d0/4,σ2＝r*σ1,σ2＝r^2*σ1,...σ2＝r^(N-1)*σ1＝d1/4

This final output may be the maximum of the outputs from all individual filters at the N scales.

Descriptors may be determined 608 in part by applying a set of filters corresponding to texture features of those image regions to the image regions. Several distinct filters may be applied to each of the image regions to generate a plurality of respective descriptors for the regions. The distinct filters may include a set of convolution filters, each configured to describe one or more aspects of the eye vasculature. The combined set of convolution filters may describe the visible eye vasculature in feature space. The distinct filters may include filters that extract pixel statistics (e.g., mean, median, standard deviation, and entropy) of the image regions.

In some embodiments, the descriptors may be determined 608 in part by applying a set of complex Gabor filters to the image at various angles. The parameters of the filters may be determined empirically (e.g., using a separate component analysis of the body of the eye vasculature image) to account for variations in the spacing, orientation, and girth of the vessels. The phase of the Gabor-filtered image, when binarized using a threshold, may facilitate detection and reveal sharp visible vasculature.

The phase of a complex Gabor-filtered image reflects vascular patterns at different angles. The phase of a Gabor-filtered image can vary from -π to +π radians. Phase values above 0.25 and below -0.25 radians may correspond to vascular structures. To binarize the phase image using thresholding, all phase values above 0.25 or below -0.25 can be set to 1, and the remaining values set to 0. This can result in sharp (i.e., well-defined or high-contrast) vascular structures in the corresponding phase image. This operation can be performed on images generated by applying multiple Gabor kernels at different angles. All resulting binarized images can be added to reveal detailed and clear vascular structures. In some embodiments, a vector of elements of the binarized phase image can be used as a descriptor vector for comparing an image to a reference recording. In some embodiments, differences in descriptors reflecting texture features between image regions of interest can be used as the descriptor vector. The sum of all 1s in a binarized image region divided by the region of interest can reflect the extent of visible vasculature.

In some implementations, other convolution filters may be applied to the image to determine 608 a descriptor of the texture of a region (e.g., a tile) in the image. For example, a wavelet transform or a Fourier transform may be applied to the region in the image. Other convolution filters are possible. In some implementations, determining 608 the descriptor may include determining the magnitude or phase of a complex filter output for the region of the image.

In some implementations, the visible eye vasculature is described by some of the determined 608 descriptors based on co-occurrence statistics of image data elements in one or more of the image regions. For example, a gray level co-occurrence matrix (GLCM) may be determined for each of one or more of the regions (e.g., tiles) identified by segmentation of the image. The GLCM may be determined for a color component (e.g., green, blue, or luminance) of the region in one of the images. Various statistics (e.g., contrast, correlation, energy, or homogeneity) may be derived from the GLCM.

In some embodiments, a large set of various GLCM statistical descriptors is generated using datasets of eyes from different subjects. For each region of interest, a GLCM matrix is generated at different grayscale levels, orientations, offsets, tile sizes, and image scales. For each combination, the aforementioned four statistics of the GLCM (energy, contrast, correlation, and homogeneity) can be calculated for each tile and at different angles, and the results can be concatenated into a descriptor vector (e.g., four GLCM statistics for each tile at different sets of angles (e.g., 0°, 30°, 60°, 90°, 120°, and 150°) are generated and concatenated). This descriptor vector extraction process can be repeated for different grayscale levels, offsets, tile sizes, and image scales. Descriptor vectors of true matches and non-matches (regions of interest from the same eye versus regions of interest from different eyes) are compared using a similarity metric (e.g., correlation or mean square error). An ROC curve is generated for each combination (descriptor vectors at different image scales, offsets, gray levels, and tile sizes for comparing true and imposter match scores) and the best feature is picked (e.g., the feature with the highest area under the ROC curve). If there are multiple good descriptor sets under different configurations, their resulting match scores can be fused using a weighted sum rule.

In some implementations, the visible eye vasculature is described by some of the determined 608 descriptors that are based on information-theoretic statistics (e.g., entropy, self-similarity, fractal dimension) of image data elements in one or more of the image regions. For example, the entropy of a region (e.g., a tile) of an image can be determined by generating a histogram of the image data element values within the region and calculating the entropy of this distribution as follows:

∑ _i [-i*ln(h(i))]

where i is the index taken at a possible image data element level, h(i) is the count of image data elements in the bin of the histogram for that level i, and ln() is the natural logarithm. In some implementations, a box counting algorithm can be used to determine the fractal dimension of the vasculature in a region. For example, after applying a set of Gabor filters and thresholding the resulting phase image to produce a binarized image of the vasculature in the region, a box counting analysis can be applied to the binarized image to determine the fractal dimension of the vasculature.

In some implementations, the visible eye vasculature is described locally or globally by some of the determined 608 descriptors that are based on one or more non-convolved statistical derivatives of image data elements. The eye vasculature is described locally by determining statistics of data image elements in a small region (e.g., a tile) of the image. The eye vasculature is described globally by determining statistics of data image elements in a larger region of interest (e.g., an area to the left or right of the iris) or the entire image. For example, the mean or variance of the image data element values in a region or image may be determined 608.

In some implementations, the descriptors are determined 608 by applying a non-linear filter.For example, a trained neural network or other non-linear function approximator can be applied to the image or a region (eg, a tile) of the image.

In some implementations, several distinct filters are applied to each of the image regions (e.g., tiles), and the outputs of these filters are combined to produce multiple descriptors of the region. For example, the outputs of multiple Gabor filters applied to a region (e.g., the square of the magnitude of each image data element of the output) can be added together to reveal a clear image of the eye's eye system characteristics, and statistics can be determined for this combined output signal for the region.

In some implementations, the descriptor is determined 608 based on the output of a cascade of filter banks conditionally applied to the obtained image.Some examples of this process are described with respect to Figures 7 and 8A-8B.

For example, the descriptor may be determined 608 by verification module 440 , verification application 550 , verification module 525 , or verification module 540 .

A match score is determined 610 based on the features and corresponding features from the reference record. The reference record may include data based at least in part on one or more reference images captured during a user's enrollment or registration process. In some implementations, the match score may be determined 610 as a distance (e.g., Euclidean distance, correlation coefficient, modified Hausdorff distance, Mahalanobis distance, Bregman divergence, cosine similarity, Kulbeck-Leibler distance, and Jensen-Shannon divergence) between a feature vector extracted from one or more obtained images and a feature vector from the reference record. In some implementations, the match score may be determined 610 by inputting the features extracted from one or more obtained images and the features from the reference record into a trained function approximator.

The function approximator models the mapping from input data (i.e., training image data) to output data (i.e., the resulting matching score) using a set of model parameters. The model parameter values are selected using a training algorithm applied to the training data. For example, the function approximator can be based on linear regression, Volterra series, Wiener series, radial basis kernel function, kernel method, polynomial method, piecewise linear model, neural network, support vector machine, or chaotic function approximator. Other models are possible.

In some implementations, determining 610 a match score can include combining respective descriptors for each of the regions (e.g., tiles) into respective vectors for the regions and comparing the respective vectors for the regions with vectors derived from descriptors for corresponding regions of an image from a reference record to generate similarity scores for the regions. The match score can be determined based in part on the similarity scores for one or more analyzed regions.

In some embodiments, a quality-based fused match score is determined 610 based on the match scores of the multiple images of the same vasculature. In some embodiments, the match scores of the multiple images are combined by adding the match scores together in a weighted linear combination with weights that depend on the quality scores determined for each of the multiple images, respectively. Other examples of techniques that can be used to combine the match scores of the multiple images based on their respective quality scores include hierarchical blending, sum rule, product rule, gated fusion, Dempster-Schafer combination, and stacked generalization, among others.

In some implementations, a match score is determined 610 by a verification module (eg, verification module 440 running on computing device 430).

The match score can be checked 612 to determine whether a match exists between the one or more acquired images and the reference record. For example, the match score can be compared to a threshold. A match can reflect a high likelihood that the user whose eyes are depicted in the one or more acquired images is the same individual associated with the reference record. For example, a robust threshold corresponding to a point in the neighborhood of a three-dimensional receiver operating curve generated from a sensitivity analysis can be used.

If there is no match, the user may be rejected 614. As a result, the user may not be allowed to access the secure device or service (e.g., security device 450 or secure transaction device 523). In some embodiments, the user may be notified of the rejection 614 by a message displayed on a display or played through a speaker. In some embodiments, the rejection may be effected by transmitting a message reflecting the status of the rejected user over a network. For example, verification module 540 may transmit a rejection message to secure transaction server 523 using the network interface of server system 514 immediately after rejecting user 530. Verification module 540 may also send the rejection message to user computing device 510 in this case.

If there is a match, the user may be accepted 616. As a result, the user may be granted access to the secure device or service (e.g., secure device 450 or secure transaction device 523). In some embodiments, the user may be notified of the acceptance 616 by a message displayed on a display or played through a speaker. In some embodiments, acceptance may be effected by transmitting a message reflecting the status of the user being accepted via a network. For example, upon accepting user 530, verification module 540 may transmit an acceptance message to secure transaction server 523 using the network interface of server system 514. Verification module 540 may also send the acceptance message to user computing device 510 in this case.

FIG7 is a flow chart of an example process 700 for determining a match score for one or more images of an eye. Several sets of Gabor filters are applied 704 to one or more images at each of a range of resolution scales, including images of a region of interest on the white of the eye. Descriptors are determined 706 based on the filter outputs, and partial match scores are determined by comparing the descriptors with corresponding descriptors from a reference record. At each resolution level, the match score is checked for a high confidence acceptance or a high confidence rejection to determine whether the partial score will be returned as the final match score or the next set of Gabor filters is applied at the next higher resolution scale. This process is repeated until a high confidence match score occurs or the highest available resolution at which Gabor filters have been applied.

For example, process 700 may be implemented by verification module 440 in computing device 430 of FIG. 4 . In some embodiments, computing device 430 is a data processing device comprising one or more processors configured to perform the actions of process 700. For example, the data processing device may be a computing device (e.g., as illustrated in FIG. 9 ). In some embodiments, process 700 may be implemented in whole or in part by verification application 550, which is executed by a user computing device (e.g., computing device 510). For example, the user computing device may be a mobile computing device (e.g., mobile computing device 950 of FIG. 9 ). In some embodiments, process 700 may be implemented in whole or in part by verification module 540, which is executed by a user server system (e.g., server system 514). In some embodiments, server system 514 is a data processing device comprising one or more processors configured to perform the actions of process 700. For example, the data processing device may be a computing device (e.g., as illustrated in FIG. 9 ). In some implementations, the computer-readable medium may include instructions that, when executed by a computing device (eg, a computer system), cause the device to perform the actions of processor 700 .

A first set of Gabor filters having a low resolution scale is selected 702 for extracting descriptors from one or more views including the white of the eye. The Gabor filters have a complex kernel used to convolve the input image. The real part of the Gabor kernel is given by:

and the imaginary part is given by

where x and y are the image pixel coordinates, λ is the spatial frequency, θ is the kernel orientation, σ is the kernel spread (standard deviation), and γ is the spatial aspect ratio, and

x′＝x*cosθ+y*sinθ

y′＝-x*sinθ+y*cosθ

The resolution scale of the Gabor filter is primarily controlled by the kernel spread (σ) parameter. In some embodiments, a first set of Gabor filters is selected where θ = {0°, 30°, 60°, 90°, 120°, 150°}, σ = 20 pixels, λ = 6, and γ = 1. The Gabor filter parameters can be selected to match the corresponding Gabor filter used to derive the descriptors stored in the reference record.

The choice of frequency can depend on the distance between vessels, which in turn depends on the resolution and the distance between the image acquisition system and the subject. These parameters can be invariant for the image. For example, the nuclear parameters can be derived for an image of the eye captured at a distance of 6 to 12 centimeters from the eye using a specific sensor (e.g., the rear camera on a smartphone), and the segmented scleral region can be resized to a resolution of (e.g., 401×501 pixels) for analysis. The surface vasculature of the eye can be seen to be scattered in all directions across the white of the eye.

A selected set of Gabor filters is applied 704 to each of the regions in the image selected for analysis by segmentation. For example, one or more images may be encoded as a two-, three-, or four-dimensional array of image data elements (e.g., pixels, voxels, rays, or red, green, or blue channel values). As described above, the image can be segmented by identifying regions of interest in the acquired image (e.g., the white of the eye to the left or right of the iris) and further segmenting these regions of interest into smaller regions called tiles. In some implementations, the tiles are analyzed using Gabor filters, and descriptors are derived from the filter output corresponding to each tile. The filters can be applied 704 by convolving each Gabor kernel with the input image or each tile or other selected region from the input image. In some implementations, the convolution operation can be performed by multiplying a frequency-domain representation of the input image with the Gabor kernel and then transforming the result back into the spatial domain. For example, the Gabor filters can be applied 704 by a verification module or application (e.g., verification module 440).

A descriptor is determined 706 from the output of the Gabor filter for each selected region of the image. The output of the Gabor filter may be a complex number. In some embodiments, the mean or median magnitude of the image data elements in the tile of the Gabor filter output is taken as the descriptor. In some embodiments, the mean or median phase of the image data elements in the tile of the Gabor filter output is taken as the descriptor. In some embodiments, the set of descriptors for each tile or other selected region is combined to form a vector of descriptors. The vector of descriptors may describe the visible eye vasculature in feature space. For example, the descriptor may be determined 706 by a verification module or application (e.g., verification module 440).

A partial match score is determined 708 based on a subset of the descriptors at the low-resolution scale (that have been processed to this extent), and the corresponding descriptors form a reference record. In a first iteration, the descriptors from the lowest-resolution scale filter are compared with the corresponding descriptors from the reference record. In each successive iteration, additional descriptors for the next highest-resolution scale filter are added to the analysis along with their corresponding descriptors from the reference file. In some implementations, the number of elements in the descriptor vector is expanded at each iteration, and the descriptor vector is compared with the corresponding descriptor vector from the reference record. Elements of the vector from the reference record can be selected from a potentially larger set of stored descriptors by using descriptors of tiles that are co-registered (i.e., corresponding to the same local area of the white of the eye, with the position being specified relative to the iris or other landmark) with the tiles for the corresponding descriptors in the determined descriptor vector. In some embodiments, a partial match score can be determined 708 as a distance (e.g., Euclidean distance, correlation coefficient, modified Hausdorff distance, Mahalanobis distance, Bregman divergence, cosine similarity, Kulbeck-Leibler distance, and Jensen-Shannon divergence) between the determined vector of descriptors extracted from the image and the vector of descriptors from the reference recording. In some embodiments, the partial match score can be determined 807 by inputting the determined descriptor vector and the descriptor vector from the reference recording into a trained function approximator.

A function approximator can be trained using data corresponding to training images of the eyes of registered individuals associated with a reference record and other unregistered individuals that have been correctly labeled to provide the desired output signal (reflecting the presence or absence of the brand corresponding to the reference record). The function approximator models the mapping from input data (i.e., training image descriptors) to output data (i.e., partial match scores) using a set of model parameters. The model parameter values are selected using a training algorithm applied to the training data. For example, the function approximator can be based on linear regression, Volterra series, Wiener series, radial basis kernel functions, kernel methods, polynomial methods, piecewise linear models, neural networks, support vector machines, or chaotic function approximators.

In some implementations, the determined descriptor vector is compared against multiple alternative descriptor vectors from reference records, and the average or best partial match score is used.

For example, the partial match score can be determined 708 by a verification module or application (eg, verification module 440).

The partial match score can be checked 710 for a high confidence rejection. In some implementations, the partial match score is compared to a threshold value (e.g., 0.3) that corresponds to a high confidence level for rejecting the user or a very low likelihood of a match between the user and the reference record. For example, the threshold value can be set to a value obtained using the operating point on the receiver operating characteristic (ROC) curve of a classifier (generated using training data for the selected descriptor) at the zero false rejection point. If the partial match score is worse than the threshold value, the partial match score is returned 712 (possibly after being converted to a different scale) and can be used by the call process to reject the user.

In some implementations, a higher probability of error can be tolerated, and the threshold can be obtained using different points on the ROC curve corresponding to non-zero but acceptably small error levels.

Otherwise, the partial match score can be checked 714 for a high confidence acceptance. In some implementations, the partial match score is compared to a threshold value (e.g., 0.65) that corresponds to a high confidence level for accepting the user or a very high likelihood of a match between the user and the reference record. For example, the threshold value can be set to a value obtained using the operating point on the ROC curve of a classifier (generated using training data for the selected descriptor) at the zero false acceptance point. If the partial match score is better than the threshold value, the partial match score is returned 712 and can be used by the call process to accept the user.

Otherwise, if a higher-resolution Gabor filter is still to be applied to the image at 716, a set of Gabor filters of a next-higher resolution scale is selected 718. For example, the average resolution scale may correspond to σ = {20, 10, 5, 2.5} pixels. Other parameters of the Gabor filters may also be varied. The next set of selected Gabor filters is then applied 704 in the next iteration of process 700.

If 716 a higher resolution scale Gabor filter is not available (i.e., there is no corresponding descriptor data in the reference record), then the final partial match score is returned 720 as the match score (possibly after conversion to a different scale), and the calling process may use the partial match score to accept or reject the user.

8A and 8B are flow diagrams of an example process 800 for determining a match score for one or more images of an eye. The example process applies a conditionally executed three-layer filter stack to determine a match score based on descriptors derived from the filter outputs. For example, process 800 can be implemented by a verification module or application (e.g., verification module 440).

The green component of one or more images is retrieved 802. The green component may exhibit a particularly high contrast level for the vasculature of the white of the eye. The images may have been previously segmented to identify one or more regions of interest (i.e., the white of the eye area to the left and right of the iris). The green components in these regions of interest are extracted and passed to the first stage filter bank.

A complex Gabor filter bank is applied 804 to the green image in the region of interest. The filters may be applied 804 by convolving each Gabor kernel with the input image. In some implementations, the convolution operation may be performed by multiplying a frequency domain representation of the input image with the Gabor kernel and then transforming the result back into the spatial domain. In some implementations, six Gabor filters are used, each with a different kernel orientation θ. For example, a Gabor filter with θ = {0°, 30°, 60°, 90°, 120°, 150°}, σ = 2.5 pixels, λ = 6, and γ = 1 may be applied 804 to the region of interest in the image. In some implementations, an inverse tangent function is used to determine the phase of the complex filter output for each image data element (e.g., each pixel).

The region of interest 806 in the resulting phase image is tiled into small sub-regions. For example, the region of interest corresponding to the white of the eye to the left of the iris can be tiled 806 into 80 tiles arranged in an 8 by 10 grid, where each tile can be 50 pixels by 50 pixels. In some embodiments, some of these tiles near the edge of the exposed sclera area can be eliminated. For example, tiles having less than 80% of their image data elements within the sclera area can be discarded. In some cases, portions of the tile area can be excluded based on glare or eyelash artifacts, which, if severe enough, can cause the tile to be eliminated.

The phase of the tiled phase image is then thresholded 808 to convert the phase image into a binary image (i.e., an image in which the data elements take on one of two possible values). The phase of the Gabor-filtered image can vary from -π to +π radians. For example, phase values above 0.25 and below -0.25 radians may correspond to vascular structures. To binarize the phase image using thresholding, all phase values above 0.25 or below -0.25 may be set to 1, and the remaining values to 0. This may result in sharp vasculature structures in the corresponding phase image. This operation may be performed for images resulting from the application of all six Gabor kernels at different angles.

A vector of descriptors is determined 810 based on the binarized filtered phase images from each of the Gabor filters. The sum of all 1s for the binarized image region (e.g., tile) divided by the area of the region of interest can reflect the extent of visible vasculature. In some implementations, the mean of the binarized image data element values in each tile can be taken as the descriptor. The set of means for each tile can be combined to form a descriptor vector. For example, an image of a single eye can include two regions of interest (e.g., left and right of the iris), each with six orientation angles (θ), and each with 80 tiles (8x10 grid), resulting in a descriptor vector with 960 elements or descriptors.

A match score is determined 812 by comparing the determined descriptor vector with a reference descriptor vector from a reference recording. Elements of the vector from the reference recording can be selected from a potentially large set of stored descriptors using descriptors of tiles that are co-registered with the tiles for the corresponding descriptors in the determined descriptor vector (i.e., corresponding to the same local area of the white of the eye, the location being specified relative to the iris or other landmarks). In some embodiments, a match score can be determined 812 as the distance (e.g., Euclidean distance, correlation coefficient, modified Hausdorff distance, Mahalanobis distance, Bregman divergence, cosine similarity, Kulbeck-Leibler distance, and Jensen-Shannon divergence) between the determined vector of descriptors extracted from the image and the vector of descriptors from the reference recording. In some embodiments, a match score can be determined 812 by inputting the determined descriptor vector and the descriptor vector from the reference recording into a trained function approximator. In some embodiments, the determined descriptor vector is compared against multiple alternative descriptor vectors from the reference recording, and an average or best match score is used.

This first match score can be checked 814 for a high confidence rejection. In some implementations, the first match score is compared to a threshold value (e.g., 0.3) that corresponds to a high confidence level for rejecting the user or a very low likelihood of a match between the user and the reference record. For example, the threshold value can be set to a value obtained using the operating point on the receiver operating characteristic (ROC) curve of the classifier (generated using the training data for the first layer of the stack) at the zero false rejection point. If the first match score is worse than the threshold value, the first match score is returned 816 (possibly after being converted to a different scale), and the call process can use this partial match score to reject the user.

Otherwise, the first match score can be checked 818 for a high confidence acceptance. In some implementations, the first match score is compared to a threshold value (e.g., 0.65) that corresponds to a high confidence level for accepting the user or a very high likelihood of a match between the user and the reference record. For example, the threshold value can be set to a value obtained using the operating point on the ROC curve of a classifier (generated using the training data for the first layer of the stack) at the zero false acceptance point. If the first match score is better than the threshold value, the first match score is returned 816 (possibly after being converted to a different scale), and the call process can use the partial match score to accept the user. Otherwise, the second layer of the stack, which conditionally executes the filter, is applied.

In this example, the second layer uses the same Gabor filter operating on the same green component input image, and thus can reuse the previously determined filtered phase image.In general, a different filter set can be used at each layer of the stack.

At the second layer of the stack, a second descriptor vector is determined 830 based on the filtered phase image that has not been binarized (i.e., the image data elements take on more than two possible values). A descriptor is determined for each tile of each filter output for each region of interest by determining the mean of the phase values within the tile. The second descriptor vector is formed from the set of descriptors for each of these tiles.

A second match score is determined 832 by comparing the determined descriptor vector with a reference descriptor vector from a reference recording. Elements of the vector from the reference recording can be selected from a potentially larger set of stored descriptors using descriptors of tiles that are co-registered with the tiles for corresponding descriptors in the determined descriptor vector (i.e., corresponding to the same local area of the white of the eye, the location being specified relative to the iris or other landmark). In some embodiments, the second match score can be determined 812 as a distance (e.g., Euclidean distance, correlation coefficient, modified Hausdorff distance, Mahalanobis distance, Bregman divergence, cosine similarity, Kulbeck-Leibler distance, and Jensen-Shannon divergence) between the determined vector of descriptors extracted from the image and the vector of descriptors from the reference recording. In some embodiments, the second match score can be determined 832 by inputting the determined descriptor vector and the descriptor vector from the reference recording into a trained function approximator. In some implementations, the determined descriptor vector is compared against multiple alternative descriptor vectors from reference records, and the average or best match score is used.

The second match score can be checked 834 for a high confidence rejection. In some implementations, the second match score is compared to a threshold value (e.g., 0.45) that corresponds to a high confidence level for rejecting the user or a very low likelihood of a match between the user and the reference record. For example, the threshold value can be set to a value obtained using the operating point on the ROC curve of a classifier (generated using the training data for the second layer of the stack) at the zero false rejection point. If the second match score is worse than the threshold value, the second match score is returned 836 (possibly after being converted to a different scale), and the call process can use the partial match score to reject the user.

Otherwise, the second match score can be checked 838 for a high confidence acceptance. In some implementations, the second match score is compared to a threshold value (e.g., 0.8) that corresponds to a high confidence level for accepting the user or a very high likelihood of a match between the user and the reference record. For example, the threshold value can be set to a value obtained using the operating point on the ROC curve of a classifier (generated using the training data for the second layer of the stack) at the zero false acceptance point. If the second match score is better than the threshold value, the second match score is returned 816 (possibly after being converted to a different scale), and the call process can use the partial match score to accept the user. Otherwise, the third layer of the stack, which conditionally executes the filter, is applied.

Referring now to FIG8B , a CLAHE filter is applied 850 to the green component of the region of interest in the acquired image. The enhanced green image is tiled 852 into small subregions. For example, the region of interest corresponding to the white of the eye to the left of the iris can be tiled 806 into 80 tiles arranged in an 8 by 10 grid, where each tile can be 50 pixels by 50 pixels. In some embodiments, some of these tiles near the edge of the exposed sclera area can be eliminated. For example, tiles having less than 80% of their image data elements within the sclera area can be discarded. In some cases, portions of a tile area can be excluded based on glare or eyelash artifacts, which, if severe enough, can cause the tile to be eliminated.

A descriptor vector is determined 854 based on the enhanced green image. The average intensity of the enhanced green image in a tile may reflect the extent of visible vasculature in the tile. In some implementations, the mean of the image data element values in each tile may be taken as a descriptor. The set of means for each tile may be combined to form a descriptor vector. For example, an image of a single eye may include two regions of interest (e.g., left and right of the iris), each with 80 tiles (8x10 grid), resulting in a descriptor vector with 160 elements or descriptors.

A third match score is determined 856 by comparing the determined descriptor vector with a reference descriptor vector from a reference recording. Elements of the vector from the reference recording can be selected from a potentially large set of stored descriptors using descriptors of tiles that are co-registered with the tiles for the corresponding descriptors in the determined descriptor vector (i.e., corresponding to the same local area of the white of the eye, the location being specified relative to the iris or other landmark). In some embodiments, the third match score can be determined 856 as the distance (e.g., Euclidean distance, correlation coefficient, modified Hausdorff distance, Mahalanobis distance, Bregman divergence, cosine similarity, Kulbeck-Leibler distance, and Jensen-Shannon divergence) between the determined vector of descriptors extracted from the image and the vector of descriptors from the reference recording. In some embodiments, the third match score can be determined 856 by inputting the determined descriptor vector and the descriptor vector from the reference recording into a trained function approximator.

The third match score can be checked 858 for a high confidence rejection. In some implementations, the third match score is compared to a threshold value (e.g., 0.82) that corresponds to a high confidence level for rejecting the user or a very low likelihood of a match between the user and the reference record. For example, the threshold value can be set to a value obtained using the operating point on the ROC curve of a classifier (generated using training data for the third layer of the stack) at the zero false rejection point. If the third match score is worse than the threshold value, a third match score is returned 860 (possibly after being converted to a different scale), and the call process can use the partial match score to reject the user. In some implementations, the determined descriptor vector is compared against multiple alternative descriptor vectors from the reference record, and the average or best match score is used.

Otherwise, the third match score can be checked 862 for a high confidence acceptance. In some implementations, the third match score is compared to a threshold value (e.g., 0.86) that corresponds to a high confidence level for accepting the user or a very high likelihood of a match between the user and the reference record. For example, the threshold value can be set to a value obtained using the operating point on the ROC curve of a classifier (generated using training data for the third layer of the stack) at the zero false acceptance point. If the third match score is better than the threshold value, the third match score is returned 860 (possibly after being converted to a different scale), and the call process can use this partial match score to accept the user.

Otherwise, additional images are requested 864, to which the first layer of the conditionally executed cascade of filters is applied. In some implementations, the additional images may have been previously captured and retrieved from the calling process or a remote device (e.g., user computing device 510). In some implementations, the additional images may be captured by prompting the user (e.g., using display 424) to submit more images via a sensor (e.g., light sensor 420).

In some implementations not shown, the conditional stacking of filters may include additional layers with different filters, where each layer uses a confidence threshold set to 0 false rejects and 0 false accepts on the ROC curve for that stage.

FIG9 shows an example of a general-purpose computer device 900 and a general-purpose mobile computing device 950 that can be used with the techniques described herein. Computing device 900 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Computing device 950 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, and other similar computing devices. The components shown here, their connections and relationships, and their functions are meant to be exemplary only and are not meant to limit the implementation of the present invention described and/or claimed in this document.

Computing device 900 includes a processor 902, memory 904, storage device 906, a high-speed interface 908 connected to memory 904 and a high-speed expansion port 910, and a low-speed interface 912 connected to a low-speed bus 914 and storage device 906. Each of components 902, 904, 906, 908, 910, and 912 is interconnected using various buses and can be mounted on a common motherboard or other suitable means. Processor 902 can process instructions for execution within computing device 900, including instructions stored in memory 904 or on storage device 906 to display graphical information of a GUI on an external input/output device (e.g., display 916 coupled to high-speed interface 908). In other embodiments, multiple processors and/or multiple buses can be used, along with multiple memories and memory types, as appropriate. Furthermore, multiple computing devices 900 can be connected, with each device providing a portion of the necessary operations (e.g., as a server bank, a cluster of blade servers, or a multi-processor system).

The memory 904 stores information within the computing device 900. In one embodiment, the memory 904 is one or more volatile memory units. In another embodiment, the memory 904 is one or more non-volatile memory units. The memory 904 may also be another form of computer-readable media, such as a magnetic or optical disk.

Storage device 906 can provide mass storage for computing device 900. In one embodiment, storage device 906 can be or contain a computer-readable medium, such as a floppy disk drive, a hard disk drive, an optical disk drive, or a magnetic tape drive, a flash memory or other similar solid-state memory device, or an array of devices including devices in a storage area network or other configuration. A computer program product can be tangibly embodied in an information carrier. A computer program product can also contain instructions that, when executed, perform one or more methods, such as those described above. For example, the information carrier is a computer- or machine-readable medium, such as memory 904, storage device 906, or memory on processor 902.

High-speed controller 908 manages bandwidth-intensive operations of computing device 900, while low-speed controller 912 manages less bandwidth-intensive operations. This allocation of functionality is exemplary only. In one embodiment, high-speed controller 908 is coupled to memory 904, display 916 (e.g., via a graphics processor or accelerator), and to high-speed expansion ports 910, which can accept various expansion cards (not shown). In one embodiment, low-speed controller 912 is coupled to storage device 906 and low-speed expansion ports 914. The low-speed expansion ports, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input/output devices, such as a keyboard, pointing device, scanner, or network connection device, such as a switch or router, for example, via a network adapter.

As shown in the figure, computing device 900 can be implemented in a variety of different forms. For example, computing device 900 can be implemented as a standard server 920 or multiple times in a cluster of such servers. Computing device 900 can also be implemented as part of a rack-mounted server system 924. In addition, computing device 900 can be implemented in a personal computer such as laptop computer 922. Alternatively, components from computing device 900 can be combined with other components in a mobile device (not shown), such as device 950. Each of these devices can contain one or more of computing devices 900, 950, and the entire system can be composed of multiple computing devices 900, 950 communicating with each other.

Computing device 950 includes a processor 952, a memory 964, input/output devices such as a display 954, a communication interface 966, and a transceiver 968, among other components. Device 950 may also be provided with a storage device such as a micro drive or other device to provide additional storage. Each of components 950, 952, 964, 954, 966, and 968 are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other suitable manners.

Processor 952 can execute instructions within computing device 950, including instructions stored in memory 964. The processor can be implemented as a chipset including a single or multiple analog and digital processors. For example, the processor can provide coordination of other components of device 950, such as control of a user interface, applications executed by device 950, and wireless communications performed by device 950.

The processor 952 can communicate with the user through a control interface 958 and a display interface 956 coupled to a display 954. For example, the display 954 can be a TFT LCD (thin film transistor liquid crystal display) or an OLED (organic light emitting diode) display or other appropriate display technology. The display interface 956 may include appropriate circuits for driving the display 954 to present graphics and other information to the user. The control interface 958 can receive commands from the user and convert them for submission to the processor 952. In addition, an external interface 962 in communication with the processor 952 can be provided to enable the device 950 to communicate with other devices in the vicinity. For example, the external interface 962 can provide wired communication in some embodiments, or wireless communication in other embodiments, and multiple interfaces can also be used.

Memory 964 stores information within computing device 950. Memory 964 may be implemented as one or more of one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Expansion memory 974 may also be provided and connected to device 950 via expansion interface 972, which may include, for example, a SIMM (Single In-line Memory Module) card interface. This expansion memory 974 may provide additional storage space for device 950 or may also store applications or other information for device 950. Specifically, expansion memory 974 may include instructions for performing or supplementing the processes described above and may also include security information. Thus, for example, expansion memory 974 may be provided as a security module for device 950 and may be programmed with instructions that permit secure use of device 950. Furthermore, secure applications and additional information may be provided via a SIMM card, such as placing identification information on the SIMM card in an uncontrolled manner.

For example, the memory may include flash memory and/or NVRAM memory, as discussed below. In one embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as the memory 964, the expansion memory 974, memory on the processor 952, or a propagated signal that can be received, for example, via the transceiver 968 or the external interface 962.

Device 950 can communicate wirelessly via communication interface 966, which may include digital signal processing circuitry, if necessary. Communication interface 966 can provide communication in various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among other protocols. This communication can occur, for example, via radio frequency transceiver 968. Additionally, short-range communication can occur, for example, using Bluetooth, WiFi, or other such transceivers (not shown). Additionally, a GPS (Global Positioning System) receiver module 970 can provide additional navigational and location-related wireless data to device 950, which can be used, as appropriate, by applications running on device 950.

Device 950 may also communicate auditorily using audio codec 960, which may receive verbal information from a user and convert it into usable digital information. Audio codec 960 may also produce audible sound for the user (e.g., via a speaker in a handset of device 950, for example). This sound may include sound from a voice phone call, recorded sound (e.g., voice messages, music files, etc.), and sound generated by applications operating on device 950.

As shown in the figure, computing device 950 can be implemented in a variety of different forms. For example, computing device 950 can be implemented as a cellular telephone 980. Computing device 950 can also be implemented as part of a smartphone 982, a personal digital assistant, or other similar mobile device.

Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementations in one or more computer programs executable and/or interpretable on a programmable system comprising at least one programmable processor coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device for special or general purposes.

These computer programs (also referred to as programs, software, software applications, or code) contain machine instructions for a programmable processor and can be implemented in high-level procedural and/or object-oriented programming languages and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., a magnetic disk, optical disk, memory, programmable logic device (PLD)) that provides machine instructions and/or data to a programmable processor, including machine-readable media that receives machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal that provides machine instructions and/or data to a programmable processor.

To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user. For example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic input, voice, or tactile input.

The systems and techniques described herein can be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form of digital data communication or any digital data communication medium (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

A computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

Several embodiments have been described, however, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.

Additionally, the logic flows depicted in the figures do not require the particular order or sequential sequence shown to achieve the desired results. Additionally, other steps can be provided, or steps can be eliminated from the described flows, and other components can be added to, or removed from, the described systems. Accordingly, other embodiments are within the scope of the following claims.

Claims (27)

1. A method for biometric verification, comprising: Obtain one or more first image regions of a first image of the eye, wherein each of the first image regions contains a view of a corresponding portion of the vascular system of the eye; The classifier is trained using a training algorithm and descriptors derived from filters applied to the eye image; The confidence threshold of the trained classifier is determined based on the false rejection or false acceptance points of the receiver operating characteristic curve generated from the sensitivity analysis of the classifier; and Until the score exceeds the confidence threshold, the filter resolution is gradually increased, and the score is calculated based on the filter resolution according to the following steps: Multiple filters having the specified filter resolution are applied to the first image region to produce a filter output; Multiple first descriptors are derived from the filter output; Define a plurality of second descriptors derived from a second image region of a second image of the eye, the second image region including regions co-registered with first image regions for corresponding descriptors among the plurality of first descriptors; and The classifier is provided with the plurality of first descriptors and at least a subset of the plurality of second descriptors as input, and the output of the classifier is obtained as the score. 2. The method of claim 1, wherein applying the plurality of filters includes convolving the filters with the first image region to produce the filter output. 3. The method of claim 2, wherein convolving the filter with the first image region comprises multiplying the frequency domain representation of the first image region with the kernel of the filter and then transforming the result to the spatial domain to produce the filter output. 4. The method according to claim 1, wherein the filter is a Gabor filter. 5. The method of claim 4, wherein each of the filters has a different orientation. 6. The method of claim 1, wherein one of the plurality of first descriptors is the average or median value of an image data element that is a portion of the filter output. 7. The method of claim 1, wherein one of the plurality of first descriptors is the average or median phase of an image data element that is a portion of the filter output. 8. The method of claim 1, wherein the score is calculated based on the distance between the plurality of first descriptors and a subset of the at least plurality of second descriptors. 9. The method according to claim 8, wherein the distance is Euclidean distance, correlation coefficient, improved Hausdorff distance, Mahalanobis distance, Bregman divergence, cosine similarity, Körbek-Leibler distance, or Jensen-Shannon divergence. 10. A system for biometric verification, comprising: Data processing devices programmed to perform operations including: Obtain one or more first image regions of a first image of the eye, wherein each of the first image regions contains a view of a corresponding portion of the vascular system of the eye; The classifier is trained using a training algorithm and descriptors derived from filters applied to the eye image; The confidence threshold of the trained classifier is determined based on the false rejection or false acceptance points of the receiver operating characteristic curve generated from the sensitivity analysis of the classifier. Until the score exceeds the confidence threshold, the filter resolution is gradually increased, and the score is calculated based on the filter resolution according to the following steps: Multiple filters having the specified filter resolution are applied to the first image region to produce a filter output; Multiple first descriptors are derived from the filter output; Define a plurality of second descriptors derived from a second image region of a second image of the eye, the second image region including regions co-registered with first image regions for corresponding descriptors among the plurality of first descriptors; and The classifier is provided with the plurality of first descriptors and at least a subset of the plurality of second descriptors as input, and the output of the classifier is obtained as the score. 11. The system of claim 10, wherein applying the plurality of filters includes convolving the filters with the first image region to produce the filter output. 12. The system of claim 11, wherein convolving the filter with the first image region comprises multiplying the frequency domain representation of the first image region with the kernel of the filter and then transforming the result to the spatial domain to produce the filter output. 13. The system of claim 10, wherein the filter is a Gabor filter. 14. The system of claim 13, wherein each of the filters has a different orientation. 15. The system of claim 10, wherein one of the plurality of first descriptors is the average or median value of an image data element that is a portion of the filter output. 16. The system of claim 10, wherein one of the plurality of first descriptors is the average or median phase of an image data element that is a portion of the filter output. 17. The system of claim 10, wherein the score is calculated based on the distance between the plurality of first descriptors and a subset of the plurality of second descriptors. 18. The system of claim 17, wherein the distance is Euclidean distance, correlation coefficient, improved Hausdorff distance, Mahalanobis distance, Bregman divergence, cosine similarity, Körbek-Leibler distance, or Jensen-Shannon divergence. 19. A non-transitory computer-readable medium storing instructions that, when executed by a data processing device, cause the data processing device to perform operations, the operations including: Obtain one or more first image regions of a first image of the eye, wherein each of the first image regions contains a view of a corresponding portion of the vascular system of the eye; The classifier is trained using a training algorithm and descriptors derived from filters applied to the eye image; The confidence threshold of the trained classifier is determined based on the false rejection or false acceptance points of the receiver operating characteristic curve generated from the sensitivity analysis of the classifier. Until the score exceeds the confidence threshold, the filter resolution is gradually increased, and the score is calculated based on the filter resolution according to the following steps: Multiple filters having the specified filter resolution are applied to the first image region to produce a filter output; Multiple first descriptors are derived from the filter output; Define a plurality of second descriptors derived from a second image region of a second image of the eye, the second image region including regions co-registered with first image regions for corresponding descriptors among the plurality of first descriptors; and The classifier is provided with the plurality of first descriptors and at least a subset of the plurality of second descriptors as input, and the output of the classifier is obtained as the score. 20. The non-transitory computer-readable medium of claim 19, wherein applying the plurality of filters includes convolving the filters with the first image region to produce the filter output. 21. The non-transitory computer-readable medium of claim 20, wherein convolving the filter with the first image region comprises multiplying the frequency domain representation of the first image region with the kernel of the filter and then transforming the result to the spatial domain to produce the filter output. 22. The non-transitory computer-readable medium of claim 19, wherein the filter is a Gabor filter. 23. The non-transitory computer-readable medium of claim 22, wherein each of the filters has a different orientation. 24. The non-transitory computer-readable medium of claim 19, wherein one of the plurality of first descriptors is an average or median value of an image data element that is a portion of the filter output. 25. The non-transitory computer-readable medium of claim 19, wherein one of the plurality of first descriptors is the average or median phase of an image data element that is a portion of the filter output. 26. The non-transitory computer-readable medium of claim 19, wherein the score is calculated based on the distance between the plurality of first descriptors and a subset of the at least plurality of second descriptors. 27. The non-transitory computer-readable medium of claim 26, wherein the distance is Euclidean distance, correlation coefficient, improved Hausdorff distance, Mahalanobis distance, Bregman divergence, cosine similarity, Körbek-Leibler distance, or Jensen-Shannon divergence.