JP2019525290A - Integrated light emitting display and sensor for sensing biological properties - Google Patents

Abstract

A biosensor device integrated with a display portion includes a surface for touching by a body portion such as a finger. A light source, such as an array of LEDs, emits light through the surface so that it is partially reflected and absorbed by the body part. An array of photodetectors detects the light reflected back by the body part and generates a signal corresponding to an image of light reflection corresponding to the light absorption pattern at the body part. The light absorption pattern may correlate with a fingerprint, blood vessel pattern, blood movement within the blood vessel, combinations thereof, or other biometric features. A processor receives the signal from the photodetector, analyzes the signal, and determines the characteristics of the body part. The characteristic can be used to authenticate the user of the biosensor device by comparing the detected characteristic with the stored characteristic. [Selection] Figure 18

Description

(Cross-reference to related applications)
This application is a continuation-in-part of US Application No. 15 / 600,480, filed May 19, 2017, and US Provisional Patent Application No. 62, filed June 9, 2016 by Jerome Chandra Bhat. Claiming the priority of these applications under US / 348,096 and US Provisional Patent Application No. 62 / 340,218, filed May 23, 2016 by Jerome Chandra Bhat and Richard Ian Olsen, The application is assigned to this patent assignee and is hereby incorporated by reference.

(Field of Invention)
The present invention relates to the sensing of biological properties such as blood flow, blood components, fingerprints, blood vessel patterns, faces, and more particularly to sensing such properties using light and photodetectors.

(background)
Applying light of a specific wavelength (eg, red or IR) to a human body part (eg, a finger) and measuring the light transmitted through the body part can result in blood flow (eg, pulse), in blood It is known that it can be used to detect components (such as hemoglobin), fat, and other properties. In general, light absorption correlates to a particular property.

  However, such systems are generally limited to medical devices that perform only a single function. Furthermore, since the measurement of light absorption is through the body part, the device must be specifically made to surround the particular body part being tested, such as a finger.

  What is needed is a more flexible and compact biological sensor that can be used for various functions and for medical purposes as well as non-medical purposes such as user authentication. With respect to user authentication, biological sensors should be able to be easily integrated into existing consumer products.

  US Pat. No. 9,570,002 describes in column 31 the addition of an IR LED emitter to the matrix of a full color display screen, along with a photodetector. At this time, the user can touch the screen and the image of the user's fingerprint can be detected by detecting reflections from the ridges of the fingertips touching the screen. The detection is only on the surface of the fingertip and does not measure any light absorption at a certain depth into the finger to detect blood vessels or blood flow or the like. Thus, if the device is used for authentication, a counterfeit / fake fingerprint can deceive the device. In addition, the display screen is a closely packed array of red, green, and blue LEDs, making it difficult to add IR LEDs and photodetectors to an existing array of LEDs without losing resolution. is there. Furthermore, developing a new display screen with integrated display pixels and detection pixels is very expensive.

US Pat. No. 9,570,002

(Overview)
An integrated compact light emitter and sensor device can be described and used to detect the biological properties of a person using the device. The characteristics can be used for medical / diagnostic purposes as well as for person authentication. For example, the device can be placed in a smartphone, computer, or even a gun to authenticate a user.

  In one embodiment, a high resolution pixel formed by a multi-wavelength light source such as an LED or filtered white light provides light that enters a body part such as a finger when applied to the skin. An array of photodetectors is integrated into the light emitter to detect the amount of light reflected back from the body part, and the amount of light reflected back from the body part depends on the absorption of light by the body part. to be influenced. An array of photodetectors can produce a high resolution image of light absorption. Pixels (eg, emitted wavelengths) can be controlled to target specific biological characteristics, and the resolution of the device can be high to detect detailed characteristics such as fingerprints and blood vessel patterns . A video image may further be captured. The optics can be used to detect only absorption at a certain depth into the skin, such as to detect the location of a blood vessel. A processor in the device may be programmed to analyze the signal from the photodetector and generate a result such as user authentication.

  Since it is very difficult to counterfeit both fingerprints and blood vessel patterns, the device has much higher reliability with respect to authentication compared to fingerprint detectors alone.

  In another embodiment, LED light (eg, IR light) is coupled to the edge of the transparent light guide and the light is guided in the light guide by TIR due to very different refractive indices of the light guide and air. . Assuming the user's finger is pushed onto a light guide surface such as glass for fingerprint detection, the refractive indices of the finger and glass are closely matched. The apex (bump) of the fingerprint contacts the glass surface, while the depression provides a void. Thus, light is drawn from the light guide to the finger only in the apex pattern. Thus, the light guide can be transparent. A high resolution photodetector array is positioned behind the light guide. Light reflected from a body part (eg, a finger) that contacts the top surface of the light guide is then transmitted back through the light guide and detected by the photodetector array. The device may also optionally operate as a display with display pixels behind the light guide. The resulting detected image can be used to authenticate the user by fingerprint, blood vessel pattern, or other combination of biological characteristics. In one embodiment, the light guide is also a display glass for the display screen that is laterally spaced from the detector portion, so that the detector and display are integrated. The display pixels can be conventional, such as OLEDs, and only the detector light source uses the display glass as a light guide. By using the display glass for the dual purpose of the light guide and protective layer for the display (or touch screen for the display), there is synergy and integration, TIR is the fingerprint / blood vessel detection In order to diffuse light uniformly. The light source for the detector can be optically coupled proximate any edge of the display glass.

  The detector array can be distributed throughout the display pixels or laterally spaced from the display portion.

  Light is incident at some distance into the finger via the apex of the finger, is scattered within the finger, and is absorbed by blood vessels within a certain depth. This generates a secondary reflection pattern that can be used in conjunction with the fingerprint pattern to detect the positional relationship between the fingerprint and the blood vessel pattern. The fingerprint pattern is high contrast, while the blood vessel pattern is effectively superimposed over the fingerprint pattern.

  In one embodiment, the light source and light sensor portions are spaced laterally from the display pixels, such as along one side of the display portion, so as not to use up any space in the display portion. The display portion emits light through a light-passing panel, such as a thin glass panel, to generate an image. The glass panel may further include a capacitive touch sensor. As previously described, the light source for the biodetector portion is laterally spaced from the display portion, such as adjacent to the photosensor portion, or the light source is optically coupled to the edge of the light passing panel The light passing panel acts as a waveguide for the light source. Detection is therefore through an area separate from the display area. The light source can emit IR light, while the display portion emits red, green, and blue light from the array of pixels. Thus, the light source and detector portions do not affect the display portion, while both the display portion and detector portion use the same light passing panel. Thus, the display and sensor are integrated but spaced laterally.

  The device can also be used as a pulse sensor or as a sensor for specific blood components that absorb target wavelengths.

  Different LEDs emitting different wavelengths can be energized separately and the reflected light is detected by a broad spectrum photodetector to determine different types of biological characteristics of the body part.

  The optics can be used to focus the light and detect only biological features at a certain depth into the body part, such as blood vessels.

  Additional sensors such as gas sensors, electrodes for detecting the pulse can be integrated into the optical sensor.

  The device is compact and inexpensive and may allow its use in portable devices for authentication or analysis of a user's biological properties.

  Various other designs and uses of such devices are also described.

FIG. 1 is a top-down view of a high-resolution light emitter that can emit light of a selected wavelength in a selected area, and the device also absorbs light from a body part of a person in contact with the light emitting surface. An array of photodetectors is included in each multi-wavelength pixel area for sensing.

FIG. 2 is a cross section of the device of FIG. 1 showing a person's finger placed on a light emitting surface to detect fingerprints, blood flow, hemoglobin in blood, or other characteristics for authentication or medical diagnosis. FIG.

FIG. 3 illustrates the correlation of light wavelength versus absorption of hemoglobin and oxyhemoglobin in the blood, which is used in the devices of FIGS. 1 and 2 to determine the concentration of such components in a person's blood. Can be used by other processors.

FIG. 4 is a cross-sectional view of another embodiment of a device that includes a light source and a light sensor to detect biological properties, where light is injected into an edge portion of a transparent or translucent light guide, Emitted from the front, a photodiode array is positioned behind the light guide to detect an image of light absorption by a person's body part.

FIG. 5 illustrates the device of FIG. 4 attached to a transparent layer, such as a protective outer layer of a smartphone, laptop computer, or other consumer device.

FIG. 6 illustrates the device of FIG. 4 augmented with electrodes to sense electrical signals from a person's body part, such as detection of an EKG signal, in combination with detection of reflected light.

FIG. 7 illustrates the device of FIG. 5 further enhanced with a sensitive gas sensing element to detect gas emanating from a person.

FIG. 8 illustrates the device of FIG. 6 further enhanced using a non-contact infrared temperature sensor to determine the temperature of a body part (eg, a finger) that contacts the device.

FIG. 9 shows another embodiment of the present invention where the focusing lens abuts the photosensor array for higher resolution, for higher signal-to-noise ratio, and to form a more compact device. Illustrated.

FIG. 10 is a front view of a single integrated circuit chip having a high resolution array of photodetectors. A lens array is positioned across the detector array, which focuses the incident light and reduces the amount of direct light incident on the photodetector to increase the signal to noise ratio. The light source can have many suitable designs, such as a transparent light guide that emits light only in the direction of the body part of the person in contact with the light guide. The reflected light is transmitted through the light guide back to the photodetector.

FIG. 11 is similar to FIG. 10, but integrates a light source (eg, LED or laser) on the same surface as the photodetector.

FIG. 12 identifies functions that can be implemented using various devices.

FIG. 13 is a flowchart of an example of the basic steps performed by the device to detect a biological characteristic of a user.

FIG. 14 is a top down view of another embodiment of an integrated sensor and display, where the sensor portion is laterally spaced from the display portion, but uses the same cover glass as the display portion.

FIG. 15 is a cross-sectional view of the device of FIG. 14 showing the sensor portion and IR emitter spaced laterally from the display portion so as not to affect the display portion. The sensor and IR emitter are shown side-by-side in FIG. 15 for ease of understanding, as opposed to the more accurate cross section of FIG.

FIG. 16 is a cross-sectional view of an integrated display / sensor showing further details of the sensor portion along the edge of the display portion.

FIG. 17 is a cross-sectional view of an integrated display / sensor showing conductive traces that are electrically coupled to the bottom portion of the detector element.

FIG. 18 illustrates the device of FIG. 15 that emits light reflected and absorbed by the body part and reflected to create a sensed image of the reflected / absorbed light to authenticate the user. Light is detected.

FIG. 19 illustrates another embodiment where the display area is partially transparent and the sensor receives light from the body part through the display area.

FIG. 20 is a cross-sectional view of an example of the embodiment of FIG. 19, wherein the display portion has a dispersed transparent area, and light reflected from the body part through the dispersed transparent area is below the display portion. Pass for sensing by the detector array. Display pixels (eg, red, green, and blue subpixels) can provide a light source for the detector array, or the light source can be a separate light source, such as an IR emitter.

FIG. 21 shows that red, green, and blue OLED sub-pixels are formed on the top substrate, and one or more photodiodes are thin film transistors on the bottom substrate to detect light reflected from the body part. While formed with, thin film transistors are cross-sectional views of another embodiment that controls display pixels.

FIG. 22 shows that an IR emitter, such as one or more IR LEDs, is optically coupled to the edge of the display glass of the display that acts as a light guide, the display glass being reflected from a body part that contacts the glass. FIG. 6 is a cross-sectional view of another embodiment that emits the light. Light can be detected only in areas that are laterally spaced from the display area, or can be detected by photodetectors dispersed within the display area.

FIG. 23 shows that a liquid crystal pixel is formed on a transparent substrate (eg, glass) with a photodiode, and light from the backlight passes through the liquid crystal layer and is reflected from the body part for detection by the photodiode. It is sectional drawing of another embodiment.

  The same or equivalent elements in the various figures are labeled with the same number.

(Detailed explanation)
Various types of biosensor devices are described that emit light of a wavelength and use image capture to detect the absorption of light by a human body part in contact with (or close to) the light emission window. The Other uses are also described.

  FIG. 1 is a top down view of a biosensor device 10 having a suitable size for a body part reflection image to be captured. In one embodiment, device 10 is only large enough to detect a finger touching its surface.

  Device 10 comprises an array of pixels 11 formed by other light sources such as micro LEDs 12 or vertical cavity surface emitting lasers (VCSEL). Such lasers are considered a subset of light emitting diodes. The LEDs 12 can be of different types that emit different peak wavelengths of interest, or the LEDs 12 can be of the same type (eg, UV LEDs) with different phosphors for emitting different wavelengths of interest. In the example shown, the LEDs 12 within a single pixel 11 include red R, blue B, green G, and IR emitting LEDs. Different wavelengths penetrate the skin by different amounts, and shorter wavelengths (blue, green) penetrate more shallowly into the skin than longer wavelengths (red, infrared). In another embodiment, the device 10 is intended only for a specific function, such as detection of blood flow in the finger, and the LED 12 is a single narrowband wavelength for a specific function, such as red or IR. Only release. Each pixel 11 also includes a broadband photodiode 14 or other type of photodetector. The LEDs 12 that emit different wavelengths can be energized separately, so the output of the photodiode 14 can be correlated to the wavelength emitted by the energized LED 12. The resolution of device 10 may be the same as a high resolution display.

  Any one peak wavelength LED in the pixel array is also optionally illuminated continuously and one or more detectors from within the pixel array can be read simultaneously and acquired by digital processing. Crosstalk can be eliminated to allow further enhancement of resolution. For example, such continuous illumination allows enhanced differentiation of the location of absorbing features (eg, blood vessels) within a body part from locations that only scatter light and do not absorb light. In addition, the detectors that are read can only be in proximity to the illuminated LEDs, and thus the sampled body part area is highly localized.

  The LED can be an OLED or an inorganic LED.

  Alternatively, the various color pixels can be formed by a liquid crystal display (LCD). A typical LCD uses a white light backlight having a wide range of wavelengths. Color filters such as a red filter, a green filter, a blue filter, and an IR filter form subpixels and are positioned behind the controllable liquid crystal layer. The liquid crystal layer effectively has an optical shutter for each subpixel location. By controlling the light shutter, different colors within each pixel are controlled. The photodiode 14 can then be formed in a transparent stacked layer over the LCD layer or under the LCD layer.

  In either embodiment, the photodiode 14 and the light emitting pixel are on the same side of the body part of the person to be analyzed, so that the photodiode 14 is for detecting light absorption by the person's skin, for example. Detect the reflected light image. This is in contrast to known devices that detect the amount of light that passes through a person's body part. Since the localized absorption within the body part being detected is basically determined by detecting the difference between the reflected light received by the different photodiodes 14, the image of the reflected light is Corresponds to the image of absorbed light. Such differences can be used to map the absorption pattern and compare it to the stored absorption pattern. Thus, although the absolute magnitude of the reflected light can vary based on the body part being sampled, any ambient light, the current to the LED, etc., the difference in the magnitude of the light reflection is still Corresponds to the absorption pattern in the part.

  An example of an array of pixels 11 for a combined fingerprint detector and vessel location detector is 2 cm × 2 cm, and may be an array that includes 800 pixels 11. The resolution can be as small as 0.25 mm. Such an array size can be incorporated into a smartphone or other handheld device for authentication of users using two different tests.

  The LED controller 18 controls the energization of the LEDs 12 using, for example, row and column addressing, and the photodiode 14 is controlled by the photodetector controller 20. The processor 22 provides overall control of the controllers 18 and 20 and detects the output of various photodiodes 14 for further processing, depending on the desired function. In one embodiment, the processor 22 controls the device to detect fingerprints and blood vessel locations, pulses, and components in the blood within the finger. Multiple images, such as videos, can be acquired to analyze blood flow. The processor 22 may compare the data with information stored in memory to authenticate the user, or may use the data for medical analysis.

  In one embodiment, the LED controller 18 rasters, or otherwise spatially orders, the illumination of the LEDs, while the photodetector controller 20 captures images corresponding to multiple unique illumination conditions. Capture the sequence. The processor 22 may process the sequence of data to compensate for the scattered light of the captured image, thereby increasing the resolution of the combined image formed by the absorbing features in the body part that contacts the device. .

  The overall device 10 including the processor 22 and the controller 18/20 may be formed as a single modular device that only requires power and data leads.

  Different sections of the pixel array. Or, different pixels of the pixels 11 can be energized separately to reduce crosstalk between the photodiodes 14 and the LEDs 12 from the different pixels 11. Ideally, the body part is in direct contact with the device 10 to minimize the scattering of light reflected upon impact to obtain the highest resolution image. Focusing and directional optics can also be used to further improve resolution.

  FIG. 2 is a simplified cross section of the device 10 of FIG. 1 showing a person's finger 23 in direct contact with the protective glass cover 24 of the device. A blood vessel 25 in the finger 23 is also shown. The ridge of the fingerprint (in direct contact with the glass) due to the finger 23 having a refractive index (e.g. 1.5) that is much higher than the refractive index of air but similar to that of the glass cover 24 26 and the fingerprint valley 27 (separated from the glass) cause different amounts of light reflection 28 in those areas. The valley area reflects a relatively high percentage of light from the glass surface due to TIR at the glass / air interface, while the raised area absorbs a relatively high percentage of light. Due to the high resolution of the photodiodes 14, some photodiodes 14 are located below the valleys 27 and others are located below the ridges 26, so the image of the fingerprint is the processor 22 (FIG. 1). ) And can be correlated with stored fingerprints for user authentication.

  The glass cover 24 may include an inorganic glass, a crystalline material such as sapphire, a glass ceramic, or a polymer.

  Methods for disturbing fingerprint detectors are known. As an additional guarantee against fraud, the processor 22 also processes the photodiode 14 data to determine a pattern of low wavelength (eg, red, IR) absorption that is consistent with blood vessels in the finger. The light is incident on the finger a distance through the fingerprint ridge. The shape and location of the blood vessel relative to the fingerprint is unique among individuals and is very difficult to counterfeit. Such data is correlated to data stored for authentication. For example, the user performs initial baseline detection and the combined fingerprint pattern and blood vessel pattern are detected and stored. With respect to authentication, the user is authenticated if the new detection substantially matches the baseline pattern. Other blood data can also be acquired for medical analysis. FIG. 2 shows the light reflection 28 at the skin depth of the blood vessel 25, where the red / IR light absorption at the blood vessel location is higher compared to the area of the finger that does not contain a blood vessel within the target depth. A map of detected blood vessels (in conjunction with or separate from the fingerprint pattern) can then be generated by the processor 22 and compared to a stored map of reference blood vessels.

  The signals of the photodiodes 14 can be subtracted from each other by the processor 22, so the difference correlates with the absorption of the wavelength of interest. The difference in the signal output by the high resolution photodiode 14 rather than its absolute value allows the common mode rejection of the light directly received by the photodiode 14 from the LED, thus greatly increasing the signal to noise ratio. Let

  In general, an array of photodiodes 14 that traverse a large area allows the device 10 to act as an image capture device for items of appropriate contrast in contact with or close to the glass cover 24. To do. Thus, when a body part such as a finger, palm, wrist, or face is in contact with or in proximity to the glass cover 24, the device 10 may provide useful biometric and biometric data such as fingerprints, blood vessels, and the like. It can be used to capture shallow subdermal biometric data such as patterns, facial and other skin tone information, and the presence of skin resident pathogens.

  In addition, the pixel array is simply used to detect the presence and movement of one or more fingers or other body parts that are in contact with or close to the glass cover 24 so that the array is Allowing it to be used to sense various gestures, or as a touch screen or simple “button”, without the need for additional dedicated touch sensing arrays, mechanical buttons, or separate gesture subsystems obtain.

  In addition, by ordering the different colored LEDs 12, body part spectroscopy can be performed, resulting in optically collected biometric data. Subcutaneous biometric data can be acquired if the wavelength penetrates the skin to some extent. For example, both red and green can be used to detect blood flow, blood vessels, and confirm heart rate, and a combination of red and infrared at the appropriate wavelength to confirm blood oxygen supply Can be used.

  Different wavelengths penetrate the skin by different amounts, and shorter wavelengths (blue, green) penetrate more shallowly into the skin than longer wavelengths (red, infrared). Thus, by scanning across the pixel wavelength, images that sample different depths of the dermis can be acquired. Data from different images can then be subtracted from each other to provide additional resolution. For example, a blue image that captures epidermis data may be subtracted from a red image that captures both epidermis data and dermis data so that only the dermis data appears.

  Increasing the range of wavelength response allows more information to be collected. By mixing or adding additional pixels with responsiveness at either higher or lower frequencies, additional functions such as pathogen detection, UV exposure, hydration, and body chemistry are confirmed. Can. In some cases, UV can be used to detect fluorescent components in blood or skin.

  In addition, by capturing a sequence of images (ie, video), blood pulse movements along the blood vessel can be acquired and the heart rate, blood pressure, blood pressure, cardiac output, Additional biometric data such as stroke volume can be provided (either directly or by speculation). The sequence of images also observes blood flow, the body part being verified comprises a truly living body part, a biometric event through the use of a spoof material sample or a separated or post-mortem body part, etc. Can be used to estimate that no attempt is made to forge. Similarly, the use of a combination of LEDs that generate various colors with a photodetector can provide identification of material applied to a human finger, eg, holding a copy of a fingerprint, but at the same time physiological The verification is shown. This “video” capability can then help to prevent fingerprint spoofing by conformal material.

  In the case of capturing and processing an extended sequence of images, compensation may need to be provided for the motion of the object during video capture. In this case, for example, the motion of the skin image captured using blue, green, or other suitable light source can be used to track the motion of the object during video capture, for example red Or it can be used to compensate for motion in a captured subdermal image that is captured using infrared light.

  Taking into account that light that penetrates into the finger is subject to backscatter and reflection fluctuations based on the structure inside the finger, fingerprint detection is performed from light that does not penetrate most into the finger, such as blue light. Is the best. However, fingerprints can be likely to be read using a wide range of visible and near infrared wavelengths.

  Using longer wavelengths, a subdermal image can be acquired. For example, using an illumination wavelength and power that allows penetration of 2-3 mm or more into the dermis allows light to penetrate into the subdermal region where capillaries and veins can be found. . Assuming that blood vessels coincide with fingerprint ridges and valleys, images collected from blood vessels are also likely to be affected by the presence of fingerprint ridges and valleys. Thus, both the fingerprint ridges and the presence of valleys can enhance the specificity of the layers used for recognition or can be removed from the image of the blood vessel. In one example where it is desirable to remove the fingerprint, techniques can be developed that simply subtract the fingerprint image as captured by shorter wavelength light as discussed above. Alternatively, this can be done via image processing, such as by filtering or other techniques with similar effects. The effect of fingerprints is also that the detector array is such that the detector focus of the detector array is essentially exclusively or predominantly under the skin, such as by the inclusion of optics within this system (shown in the figure below). Can be minimized by setting

  Also, by detecting both fingerprint and blood vessel images, the orientation of the finger on the device may be random. This is because the fingerprint image can provide a reference for the blood vessel image. Thus, the images relative to each other can be accurately compared with the stored relative images regardless of the position of the finger on the device.

  As an artifact of this bioauthentication methodology, significant physiological information can be extracted simultaneously. By studying images of blood vessels captured under different illuminations, it is possible to perform spectroscopic analysis of tissue directly under the sensing area, from which blood, interstitial fluid, or tissue chemistry Can be implemented. For example, by studying images under two different wavelengths, such as 680 nm and 850 nm, where hemoglobin and oxyhemoglobin have different extinction coefficient changes (FIG. 3), it is possible to obtain a measure of blood oxygen saturation SpO2. is there. In turn, by studying images captured under the appropriate illumination wavelength, blood glucose, red blood cell count, white blood cell count, blood CO2, blood glucose, and other blood and interstitial fluid solutes can be detected.

  In all of the above configurations, the effects of ambient light may need to be considered. Specifically, when studying thin body parts such as fingers, ambient light can propagate through the fingers to the detector array and interfere with the received signal or image. Hence, the influence of ambient light may need to be considered. Ambient light may comprise an essentially steady state light source such as sunlight or a modulated light source such as incandescent or modulated LED lighting. The portion of the detected optical signal derived from ambient light can be quantified by first sampling the detector array with the array light source in the “off” state. This signal can then be subtracted from the signal that the detector array subsequently captures when the array light source is in the “on” state so as to estimate a signal that is only relevant for illumination by the array light source. In addition, since all photodiodes 14 can detect the same amount of ambient light, subtracting the in-phase signal cancels the ambient light.

  This method of correlated double sampling can be further enhanced by both the photodetector configuration and options for multiple photodetectors within each pixel. This “ambient light removal” can be facilitated by modulating the array light source and array detector sampling times at a frequency much higher than the frequency of any modulation of the ambient light. The frequency of any ambient light modulation can further be detected by the detector array itself.

  The above describes an example utilizing a micro LED display, but the functionally equivalent modality is extended to be integrated into any other display of appropriate resolution, such as an OLED display or an active matrix LCD display. This can be achieved through integration of an array of detectors. Specifically, if the display includes integrated semiconductor elements such as amorphous silicon, polycrystalline silicon, or organic semiconductors, the photodiode array is formed in the display using a substantially similar semiconductor process. Thus, it may involve minimal additional processing costs and higher resolution.

  A functionally equivalent modality is also a separate extended detector, for example in an optical module, a chip on the glass (chip attached directly to the cover glass), or a chip on the display (chip attached directly to the display glass) Can be achieved using an array of Such modules, chips on glass, or chips on the display can be integrated directly with the display, or can comprise a stand-alone array, can have its own illumination source, or an external light source from the display, etc., or any Other suitable existing pixelated, uniform, side, point, or other illumination sources may be arranged to utilize.

  The expanded array of detectors may comprise an integrated sensor array, such as a CMOS image sensor, and may have integrated optics as depicted in FIG. 4, described later. The module cover glass itself can optionally be fully or partially coated with optically reinforced, optically filtered or optically blocking layers, antireflective coatings, and the like.

  Such a module may operate as an integration point into end uses for multiple human interfaces and physiological enhancements. Features such as, but not limited to, gesture recognition for high-level functions such as optically functional on / off or other “buttons”, pinch, zoom, scroll, joystick, trackball, signature, etc. on consumer devices Can be added to existing operations via algorithms or software.

  An integrated or stand-alone sensor module can also be enhanced with a pressure sensor or pressure sensing array to provide additional mechanical button actions. The stacked capacitive sensor array layer can also be integrated, and then the “touch” event and the associated force of the touch event can be determined. The determination of the force of the touch event can also be used as feedback to the consumer of the touch event that is of a size that is not optimized for optimal biometrics or biometric data. For example, pushing the module too hard can limit blood flow to the capillaries, thereby affecting the signals received therefrom.

  The integrated or stand-alone sensor module is further bonded directly to the display or cover glass via an optically clear adhesive such as epoxy, silicone, acrylic, or low temperature melting glass such as frit glass or compound semiconductor glass. obtain. In this case, the bond may comprise an adhesive that extends across the entire interface of the module cover glass and the display glass. Alternatively, the adhesive can be dispensed over a portion of the interface (eg, at the edges), and the majority of the module cover glass and display glass can simply touch or be in close proximity.

  FIG. 4 illustrates another embodiment in which a light source 30, such as an LED, that emits the same or different wavelengths is located only around the outer edge of the light guide 32 that also serves as a protective cover. Light is coupled to the edge portion of the light guide 32, such as by embedding the LED in the light guide 32, or otherwise coupling light to the edge. Light is transported to all areas of the light guide by TIR and exits only through the top surface of the light guide 32.

  If the body part should be in direct contact with the light guide 32, the light is naturally extracted from the light guide at the point of contact by the body part due to skin having a refractive index similar to that of the light guide. The If the body part to be detected can be spaced from the light guide 32, the light guide 32 can have a light extraction feature such as a shaped micro-reflector (prism), which directs the light upward. And mix the light. It is well known to use micro-reflectors so dispersed in the light guide for the backlight. Thus, light from an energized LED can be emitted substantially uniformly upward into a body part that contacts or is close to the light guide surface. There is little or no downward light emitted by the light guide 32. Many of the light guides 32 are transparent so that light reflected back from the body part passes through the light guide 32. Such designs using light guides are relatively inexpensive compared to integrating IR LEDs and photodetectors with display pixel LEDs into the display screen. Furthermore, the resolution of the photodetector array is independent of any display pixel array, and thus a higher resolution can be created.

  The optic 34 may comprise a focusing microlens that focuses light reflected from only a certain distance into the body part onto the photodiode array 38. The optic 34 may also limit incident light from the body part only to a narrow angle perpendicular to the light guide, reduce crosstalk, and better map the characteristics of the body part being analyzed. The photodiode array 38 can be a CMOS image sensor, a CCD image sensor, or any other image sensor.

  A light blocking wall 40 can be used to prevent direct light from impinging on the photodiode array 38. The module may include an opaque thermally conductive enclosure 42 to absorb heat from the LED. The control electronics of FIG. 1 may be attached to the module or separate. The module may have contact pads for soldering to the printed circuit board.

  FIG. 5 illustrates the module of FIG. 4 abutting a protective transparent cover 44 such as a transparent surface layer of a smartphone or other device.

  As will be described in detail later, the IR emitter of FIG. 2 can be removed from within the display pixel array and edge coupled to the display glass. The IR light is then diffused uniformly within the display glass by TIR until the IR light encounters skin tissue (eg, a finger) that is in direct contact with the display glass. At the point of contact, the light is reflected / absorbed by the skin tissue and detected by the dispersed photodiodes 14 in the display pixel array. Thus, the body part can touch any part of the display glass in any orientation for detection.

  As shown in FIG. 6, the integrated or stand-alone sensor module may further comprise one or more electrodes 48 and 50, one or more electrodes 48 and 50 passing through the light guide 32 or the light guide. Extends around 32 or is external to the module but is electrically connected to the module to allow the user to be electrically monitored. For example, such electrodes 49 and 50 can be used to detect a user's electrocardiogram (ECG or EKG) or bioimpedance. The ECG signal can be used in isolation to determine an associated, possible or actual medical condition, as determined by heart rate, or form of ECG signal. Alternatively, ECG can be used in conjunction with an optically derived photoplethysmogram (PPG) to determine blood pressure and other medically important indicators. Bioimpedance can be used to determine hydration, fat content, or other vital signs. Muscle action can also be monitored electrically. All such monitors can be combined in a single module.

  The ECG signal can also be used as a biometric signature.

  Thus, bio-authentication can be performed through a single modality such as fingerprint, angiography, or ECG, or a combination of multiple modalities, ie through a form of multi-factor authentication. Assuming that all biometric events are subject to errors in the form of false positives and false positives, the use of multi-factor biometrics can improve the accuracy of biometrics. For example, a multi-factor bio-authentication scheme may be configured to confirm an authentication event comprising a two-factor positive authentication and a third-factor rejected authentication, thereby (at the expense of reduced security). (Although providing increased security against false positives and spoofing by reducing the probability that a user will be locked out of the system due to false positives) while requiring that at least two factors be authenticated Can do. As an alternative, assuming that different modalities have different authentication times, the multi-factor authentication scheme provides an initial fast low security authentication based on a single fast authentication factor, with additional slower authentication. It may be configured to provide one or more subsequent increasing levels of security authentication over a long period of time required to collect and process the factors.

  As shown in FIG. 7, the module can be further augmented with an integrated gas sensing element 54. The gas sensing element 54 may instead be a separate proximity device. The use of highly specific, highly directional gas sensors, such as suitable electrochemical sensing elements, can be used to detect current or possible medical conditions so that the dog can sniff a person's illness. Such data can be used in isolation or in conjunction with the optical and electrical data outlined above to provide a more complete context in which the data can be interpreted, thereby providing a more (statistically) accurate A medical diagnosis or a comprehensive health / wellness diagnosis can be provided.

  The use of complementary gas sensing elements 54 can also be used in bioauthentication applications. The fingerprint spoof sample can comprise a fingerprint reproduction in or on organic materials such as wood, glue, putty, acetate sheets, and the like. Such materials release most volatile organic compounds, especially in a short time after formation or curing. In addition, all humans emit volatile organic compounds (VOC) through the skin and sweat. Thus, a fingerprint or other biometric event in contact or close proximity, where the gaseous environment at the point of contact is sampled substantially simultaneously with the biometric event, provides an opportunity to “sniff” the presence of spoof material or direct skin. Have. Furthermore, the specific ratio of VOCs (its “odor”) emitted by any one individual is sufficiently unique, which is an additional bioauthentication factor if sufficient resolution exists in the gas sensing element. Or it can be used as a modality. Such a sensing scheme is depicted in FIG. 7 by a photodiode array 38 that senses light absorption by fingerprints and / or blood vessel locations in conjunction with a gas sensing element 54 that detects the presence of an actual human finger 23.

  The integrated or stand-alone sensor module can be arranged as a compact line scanner (single or thin line of pixels) and a finger can be physically scanned or swipe across the line scanner. This may provide the advantage that it is possible to scan an enlarged part of a finger or other body part using a reduced sensor footprint. In the case of a stand-alone sensor module, the reduced footprint can facilitate design-in to space constrained platforms such as mobile phones, watches, and other wearables, which can facilitate reduced module costs. . The finger swipe action can allow a small form factor sensor to investigate a wide range of body tissues. Being able to sense over a wide range of body tissues offers the advantage that certain biometrics and biometric markers can be sampled within specific tissue areas where the markers are the strongest or most highly defined . For example, when trying to acquire a traditional fingerprint, scanning tissue across the distal phalanx is most useful because the distal phalanx is an area rich in epidermal features (ridges and valleys) Provide information. On the other hand, capturing unique subdermal information such as finger vein identification data can be seen to be most successful across the middle phalange, where the vein is larger and therefore can be detected. It is easier and is less affected by the application of pressure to the finger by contact with the sensor module during the biometric event. Thus, the optical biometric line scanner can be configured to optimally capture fingerprint data from the distal phalange and finger vein identification data from the middle phalanx as the finger is scanned over it. The module further senses the passage of the finger across the module, such as by changing the illumination source or focal length as the sensor passes under one of the interphalangeal joints, and alters the module's operation during finger scanning. obtain.

  A line array sensor is further formed on or behind the flat or curved cover glass, and the curve of the glass may substantially match that of the finger or wrist. Such a sensor can then be incorporated into a wearable device such as a ring or watch that can be used as a wearable device to provide user bio-authentication when networked. Such a device can conduct a one-time bioauthentication event. This can then continuously or periodically poll the sensor array to confirm that the wearable device has not been removed and the user is still alive. The wearable device can then verify that the user remains bio-authenticated without having to perform any subsequent bio-authentication events. Such devices can then be networked wirelessly with, for example, telephones, credit card payment systems, ATMs, cars, doors, data bolts, and the like, to facilitate fast user authentication. Such a modality is particularly useful when the biometric event comprises a redundant event, such as when the biometric event comprises an analysis of an ECG signal that may require several heartbeats to be captured.

  As shown in FIG. 8, signals from integrated or stand-alone sensor modules can be further augmented with data from integrated or individual proximity contact or non-contact temperature sensors 58. A low wavelength (red or IR) filter 60 may cover the sensor 58. The temperature data can be used as a stand-alone biometric or can be used to provide further context in which other biometrics are interpreted.

  FIG. 9 shows an embodiment in which the control circuit and photodiode are integrated into the sensor chip 62 and the focusing optics 64 is formed directly on the photodiode array for more precise focusing. Focusing may be within a distance into the body part that contacts the light guide 32. An edge coupled LED light source 30 is also shown. The resulting module is very thin and is easily incorporated into various applications.

  FIG. 10 is a top view of the sensor chip 62 and the optical unit in FIG. 9 as viewed from above. Each plenoptic lens 68 can overlap an array of photodiodes 14, and a microlens 66 can cover each photodiode 14. The pitch between the photodiodes 14 can be less than 0.25 mm to form a compact array. In this scheme, the array of photodiodes 14 (detector pixels) under each plenoptic lens 68 results in one macropixel having a resolution of 0.25 mm. Each microlens 66 across each photodiode 14 may improve light capture and provide directivity. Each plenoptic lens 68 in this embodiment overlaps approximately 16 photodiodes 14. The plenoptic lens 68 may be directional to focus at different depths and collect more information about the light reflected from the body part. The various lenses can be formed by molding a transparent sheet and laminating the sheet over the photodiode array, or can be formed by molding directly over the photodiode array. Although a hemispherical lens 68 is shown for simplicity, the lens can be any shape suitable for focusing. The chip can be attached to the rear of any light guide glass. In another embodiment, the optic is spaced from the photodiode array using a spacer. A reference photodiode 69 may be located outside the plenoptic lens 68 to detect ambient light. System logic 70 for processing the detected signal is formed on the same chip as the photodiode array for compactness and improved signal to noise ratio. The logic may include an analog / digital converter and digital processing circuitry.

  FIG. 11 illustrates the module of FIG. 10 augmented with an LED light source 72. The light source 72 can inject light into the upper light guide and diffuse the light across the photodiode array, as described above. Light reflected from the body part passes through the light guide back to the photodiode. In another embodiment, the module can be small (eg, below 2 × 2 cm) and the light guide is scattered because the light from the LED light source 72 is scattered within the body part and reflected onto the photodiode array. Not needed.

  As LED performance degrades over time, the photodiode 14 can also be used to compare LED light output against the baseline, provide feedback to the LED energization circuit, and achieve baseline performance.

  The above embodiments comprise spectroscopic analysis performed by an array of broadband detectors in conjunction with narrowband emitters, but their functionality also includes optical filters in conjunction with broadband emitters such as phosphor-converted white LEDs. It can be implemented on or through the use of wavelength specific detectors such as detectors associated with the optical path.

  FIG. 12 identifies various possible uses of the modules described herein.

  FIG. 13 is a flowchart identifying basic steps performed by various modules, according to one embodiment of the invention.

  In step 80, an LED display panel, light guide panel, or other light source is arranged with respect to the photodetector array to form a biosensor.

  In step 82, the output window of the biosensor is touched by the body part for analysis. A touch sensor or light shadow sensor may be incorporated into the device to detect when a body part is on the output window to initiate the image detection process.

  In step 84, the LED is energized, applies the desired wavelength to the body part, and detects localized light absorption by the body part for fingerprint detection, blood vessel detection, pulse, etc. The LEDs can be energized simultaneously or sequentially. If the LED is energized continuously, the photodetector can be selectively read to better associate the absorption of light with a location in the body part proximate to the energized LED. Continuous illumination can be any pattern.

  In step 86, the signal from the photodetector array is scanned, such as by column and row, to detect the relative magnitude of the detected light across the array and effectively produce a detailed image of light absorption by the body part. To get to.

  In the case where the LEDs are energized continuously, the method can then loop back to step 84 after each energization to provide additional spatial absorption data.

  In step 88, the raw data is processed by the module's processor to obtain results such as user authentication, medical analysis, and the like.

  Various other embodiments are disclosed below, including embodiments that are located outside the display area so that the photodetector and light source do not affect the display portion itself. Thus, the display portion can generally be conventional. The photodetector is covered by the same glass that covers the display screen. The light emitted for the biosensor function can be from the edge portion of the display or from a light source separate from the display portion.

  Embodiments are also described in which the display portion includes a dispersed transparent portion through which light reflected by the body portion passes and is detected by the underlying photodetector array.

  14-18 illustrate an integrated display and optical sensor that uses the same display glass or cover glass, although the sensor is laterally spaced from the display portion. Display area 90 includes an array of red, green, and blue pixels that are addressed to display any image. The device can be a smartphone display. Display area 90 is covered by a thin display glass 92 (or other type of light transmissive panel), which may include a touch screen sensor layer.

  The active area of the display area 90 is surrounded by black ink 94 for aesthetic purposes. The ink is typically black, but can be any color.

  Adjacent to the display area 90 is a sensor area 96 that uses the same glass 92. If the black ink 94 is impermeable to IR, the sensor area 96 is either not coated with ink or is coated with IR ink 98 that passes IR. IR ink 98 may appear black.

  One or more optical detector elements 100, such as a photodiode die, are mounted under the IR ink 98, so that they can detect light passing through the IR ink, as shown in FIG. An IR emitter 102, such as an IR LED, is energized to apply light to a body part that contacts the glass 92 above the sensor area 96. Other wavelength LEDs such as red may also be in the sensor area 96 depending on the biological property to be detected. Assuming that the body part is a finger, IR light or red light is partially reflected from the surface of the finger and is incident on the finger, particularly where the fingerprint apex contacts the glass 92. As explained earlier, some of the light is absorbed by blood and blood vessels, or by some component in the blood, and the pattern of absorption is detected.

  In another embodiment, the light emitted by the sensor area 96 may be white light, and the detector element 100 reflects white light from any body part or printed object in front of the sensor area 96. Detect the image.

  By laterally separating the detector element 100 and the IR emitter 102 from the display area 90, the display area 90 is not affected by the sensor area 96. Accordingly, the display area 90 can have an array of very high resolution pixels.

  An optic can be incorporated into the optical path of the detector. For example, the optic can be molded over or mounted on the detector die. Alternatively, the optics can be molded onto the display glass 92 or IR ink 98, or otherwise mounted on the display glass 92 or IR ink 98. The optics can also optionally be etched into the display glass 92. Optical filters such as organic dyes, dielectric filters, or metal dielectric filters can also be placed directly on the photodetector die, on the display glass 92, or on any other suitable element in the optical path. The detector may comprise an array of pixels or a focal plane array.

  FIG. 16 illustrates the detector element 100 in more detail. The reflected light 104 passes through the IR ink 98 and is detected by the detector element 100, which is a single detector element or an array of detector elements that reflects / reflects light by a body part such as a finger. Absorption high resolution images can be generated.

  In another embodiment, the LEDs in the display area 90 provide light reflected from the body part because there is some scattering of light in the body part that reaches the detector element 100.

  An advantage of the devices of FIGS. 14-18 is that the sensor area 96 does not affect the structure or operation of the display area 90, which may be a smartphone display.

  Sensor area 96 may be used to detect a user's fingerprint and / or blood vessel pattern for authentication.

  As shown in FIG. 16, the electrical interconnections to detector element 100 include conductive traces 106 and 108 deposited on IR ink 98 and appropriate interconnections 110 and 112 (eg, conductive adhesive, Via solder etc.). Conductive traces 106/108 may alternatively be deposited directly on glass 92 and patterned IR ink 98 may be deposited on top of them. The conductive traces 106/108 can comprise, for example, a conductive oxide, a conductive organic material, a semiconductor, a metal, or a mixture thereof. Interconnect 110/112 may comprise, for example, solder, metal, conductive paste, conductive film, or anisotropic conductive elements. The detector element 100 may comprise a stand-alone photoreceptor or a two-dimensional or linear array thereof, or one or more photoreceptor elements, such as photodiodes, with analog electronics and optional analog / digital converters, digital control logic. , Digital / analog circuitry, and elements that integrate with processing circuitry. The detector element 100 may comprise a CMOS image sensor or similar device.

  The rear cover 113 is also shown in FIG.

  As shown in FIG. 17, the electrical connection to the detector element 100 can also be, for example, a flexible PCB (flex circuit), PCB, ribbon connector, wire assembly, or any other connected to the back side of the detector element 100. It can be made independent of the glass 92 by other suitable elements. A metal trace 114 is shown in this example. Electrical connections to the back side of detector element 100 can also be made via spring connectors or other similar interconnection schemes. In such a scheme, detector element 100 may comprise technologies such as those used in backside illumination (BSI) CMOS image sensors.

  One or more light emitting devices such as LEDs or VCSELs may also be mounted on the glass 92. The light emitting device and detector element 100 may be mounted on the glass 92 in a similar manner. An optic may be incorporated into the light path of the light emitting device. For example, the optic can be molded over or mounted on the emitter. Alternatively, the optics can be molded onto the display glass 92 or IR ink 98, or otherwise mounted on the display glass 92 or IR ink 98. The optics can also optionally be etched into the display glass 92. Optical filters such as organic dyes, dielectric filters, or metal dielectric filters can also be placed directly on the photodetector die, on the display glass, or on any other suitable element in the optical path. In this case, light is emitted from the light emitting device through IR ink 98 and onto a partially reflective body (such as the user's face, hand, or finger) above or in contact with the display glass 92. Incident. As shown in FIG. 18, the light 117 emitted from the IR emitter 102 is reflected back from the body part 116 (light ray 118) and detected by the detector element 100. Alternatively, the display itself can be used as an emitter, and the detector element 100 can simply detect light reflected from the display.

  Although the example given above comprises a detector element 100 positioned behind the IR ink 98, the detector element 100 may be positioned on a glass 92 that is free of any ink.

  In the above embodiment where emitted light is required as part of the sensing scheme, double correlation sampling at the time of turning the light emitter on and off distinguishes between the reflected signal and background light. Can help you.

  19 and 20 show a partially transparent display 119, such as a partially transparent OLED display, in which one or more optical detector elements 100 are mounted on the back side of the display 119, such as on the back glass plate 120. Illustrated. The display 119 may comprise an array of intentionally transparent areas 121 (“transparent pixels”) shown in FIG. 20, the purpose of which is to provide an optical path through the display 119, thereby making it partially transparent. It is to be. In such an embodiment, the detector element 100, which may be an array of photodiodes or other photodetectors, mounted on the back of the partially transparent display 119, is optionally included in the pixel array of the partially transparent display 119. The photosensitive elements in the detector array can therefore be aligned and therefore are located directly below the transparent area 121 in the partially transparent display 119. The light reflected from the body part 16 can be from the display pixel 124 or from the dedicated emitter 102. In FIG. 19, light ray 125 is shown as emitted by dedicated emitter 102, and in FIG. 20, light ray 126 is shown as emitted by red pixel 124. The reflected light 127 is shown to be detected by the detector element 100.

  An optic can be incorporated into the optical path of the detector element 100. For example, the optic can be molded over or mounted on the detector die. Alternatively, the optic is molded on the back glass plate 120 or otherwise mounted on the back glass plate 120 or etched into the back glass plate 120, or alternatively the back glass plate 120. Can be patterned into Optical filters such as organic dyes, dielectric filters, or metal dielectric filters can also be placed directly on the photodetector die, on the display glass, or on any other suitable element in the optical path. Detector element 100 may comprise an array of pixels or a focal plane array.

Any of the display elements can be used as a light source, and the light reflected from the body part can be detected by the detector element 100. A red, green, and blue LED 124 (labeled RGB) is shown in each display pixel area proximate to the transparent area 121. The display pixel array itself, typically comprising red, green, and blue array-like pixels, is optionally augmented to include infrared emitting pixels, ultraviolet emitting pixels, or any other suitable wavelength emitting pixels. Can be done. In such schemes, OLED materials that emit in the infrared, ultraviolet, or other suitable wavelength range may be used. In the case of micro LED displays, for example, infrared and ultraviolet LEDs made from Al _x In _y Ga _z P and Al _x Ga _y N, respectively, can be incorporated as part of a red, green, and blue LED pixel array. Other suitable wavelengths may be produced by these same materials or other compound semiconductor alloys.

  Alternatively, one or more separate optical emitters 102 can be mounted behind the partially transparent display 119. Light from these emitters 102 can be transmitted through the partially transparent display 119, reflected from the external body part 116, and detected by the detector element 100. An optic can be incorporated into the optical path of the emitter 102. For example, the optic may be molded over or mounted on the emitter die. Alternatively, the optic is molded on the back glass panel 120 or otherwise mounted on the back glass panel 120, or etched into the back glass panel 120, or alternatively the back glass panel 120. Can be patterned into Optical filters such as organic dyes, dielectric filters, or metal dielectric filters can also be placed directly on the photodetector die, on the display glass, or on any other suitable element in the optical path.

  Similar to the design of FIG. 14, separate emitters can alternatively be mounted along the side of the display. These separate emitters can comprise an infrared emitter, an ultraviolet emitter, or any suitable wavelength emitter, for example, an LED or a phosphor-converted LED. Alternatively, a separate emitter in the final product, such as a camera flash, indicator LED, light source incorporated in other optical communication links, or any other light source may be used as an illumination source, thereby proximate A signal reflected from the body can be detected. In addition, as described above, the light source can be edge coupled to the display glass, which diffuses the light internally by TIR until the light escapes at the contact points of the body part and the display glass.

  FIG. 21 illustrates a single pixel in a display 130, such as an AMOLED display, which is formed in low temperature polycrystalline silicon (LPTS), amorphous silicon (a-Si), or crystalline silicon. A semiconductor backplane 132 comprising a thin film transistor (TFT) array 134 is further provided. In such embodiments, one or more photosensitive detector elements 100 are formed in the semiconductor backplane 132 prior to deposition of the surface organic electroluminescent layer. The detector element 100 may comprise a photodiode, such as a p-i-n photodiode, for example. OLED material can be grown or deposited across the TFT array 134 to form red, green, and blue subpixels 135. Infrared subpixels can also be so formed as part of the array. The optic 136 can optionally be formed across the detector element 100 or otherwise arranged. Such an optic may comprise, for example, a lens formed by, for example, a photoregistry flow process or a printing process. One or more detector elements 100 may be used as an optical detector or detector array that is integrated directly into the display. Ray 138 is also shown.

  Optical filters such as organic dyes, dielectric filters, or metal dielectric filters can optionally be incorporated across the detector element 100 to provide pixels that are sensitive to specific wavelengths. For example, red, green, blue, cyan, magenta, yellow, IR, and UV-absorbing dyes optionally across different detector elements to provide a multi-pixel detector capable of color sensing or spectroscopy. Can be placed. The OLED material itself is optionally placed over the photodiode, and the red, blue, and blue OLED materials all have different absorption spectra and can therefore act as intrinsic optical filters.

  FIG. 22 illustrates another embodiment of an integrated display and detector, and FIG. 23 illustrates a single pixel area within the display / detector of FIG.

  Full color display 140 includes a backplane 142 such as a TFT-LCD with a thin film transistor (TFT) array formed in low temperature polycrystalline silicon (LPTS), amorphous silicon (a-Si), or crystalline silicon. The transistor may be patterned and doped amorphous silicon deposited on a transparent glass substrate 146 (FIG. 23), and patterned conductors such as ITO may be transparent. An array of thin film transistors 147 is formed on the glass substrate 146 along with the LCD layer 148, and the transistor 144 is turned on by a conventional column and row decoder to control each LCD pixel, one of which is shown in FIG. .

  Color filters 150, such as blue, red, and green filters, are formed above or below the LCD layer 148 to form blue, red, and green subpixels. Infrared subpixels can also be so formed as part of the pixel. A black mask 152 is printed on the upper display glass 154 to better optically isolate the subpixels.

  The display portion of the backplane 142 can be conventional.

  A white backlight 156 is positioned below the backplane 142 and the LCD sub-pixel acts like a controllable shutter, emitting a selected amount of blue, red, and green light per pixel, wide Generate a color gamut.

  Photolithography is commonly used to form the backplane 142, and very high resolution can be obtained.

  In order to enhance such a display using a detector, one or more photosensitive elements 158, such as amorphous silicon photodiodes, are also deposited and patterned on the backplane 142 by photolithography. The photodiode can be a variation of the silicon transistor 147 formed in the TFT array and formed using an additional masking step. The TFT array and the photodiode can be formed in the same plane on the glass substrate 146. A suitable conductor on the substrate 146 connects the photodiode to the detector circuit. The photodiodes can be distributed outside the display area or throughout the display area. The photosensitive element 158 may comprise a photodiode, such as, for example, a pin photodiode.

  As shown in FIG. 22, an IR emitter 159 or other peak wavelength LED is optically coupled to the edge of the display glass 154, which is used as a waveguide to diffuse IR light 164 and back IR light 164 is emitted only above the photodetector element 158 away from the plane 142. As explained earlier, light is extracted from the display glass 154 only in areas where the body part 116 contacts the display glass 154. If the body part does not need to contact the display glass, the surface of the display glass 154 can be roughened over the photodetector elements 158 to extract the guided light. The display glass can be clear over the display portion of the pixel.

  The IR filter 162 can also be printed on the bottom surface of the display glass 154 and can be planar with the RGB color filter 150.

  FIG. 23 shows the backlight 156, which provides light 168 for the detector, but the light for the detector is coupled to the edge of the display glass 154, as shown in FIG. Can be done.

  The optic may optionally be formed over the photosensitive element 158 or otherwise arranged. Such an optic may comprise, for example, a lens formed by, for example, a photoregistry flow process or a printing process. The one or more photosensitive elements can be used as an optical detector or detector array that is integrated directly into the display. In such an illustration, a light blocking layer can be incorporated between the growth substrate and the photosensitive element to block direct illumination of the photosensitive element by the backlight.

  Optical filters, such as organic dyes, dielectric filters, or metal dielectric filters, can optionally be incorporated across the detector pixels to provide pixels that are sensitive to specific wavelengths. For example, red, green, blue, cyan, magenta, yellow, IR, and UV absorbing dyes optionally across different detector pixels to provide a multi-pixel detector capable of color sensing or spectroscopy. Can be placed. The standard LCD color filter material itself can optionally be placed over the photodiode.

  In some embodiments, the detector pixels can also be placed directly under the LCD element, and thus the LCD matrix can be used to shutter-close the detector pixels to facilitate image capture by the array.

  In FIG. 22, the display portion typically operates to display any image or instruction, while the IR LED 159 of FIG. 22 causes the display glass 154 to emit IR light into the area of the photodetector element 158. So that it is energized. The emitted light is reflected and absorbed by the body part 116 above or in contact with the display glass 154 proximate to the photodetector element 158, and the signal from the photodetector element 158 is detected by the detector circuit. To identify characteristics of the body part 116 such as fingerprints, blood vessels, face recognition, combinations thereof, and the like. Photodetector element 158 may comprise any number of pixels depending on the desired resolution. Thus, all emission for detection is generated by a side-emitting source, such as IR LED 159, and therefore no display pixel area is occupied by the IR emitter.

  At the point where the glass is in contact with human tissue, the close refractive index of the glass 154 and human tissue allows light to be transmitted into the tissue. The light is then attenuated and backscattered from the tissue onto the detector array, and the light can be detected from the detector array. A bioauthentication signature or biometric data can be verified from the detected light.

  Wavelengths other than IR can be used to detect other biological properties, particularly with respect to facial recognition.

  In one embodiment, the display portion may be high resolution to display a conventional image, while the detector portion uses low resolution visible light pixels to identify the location of the detection portion under the display glass. May be included. For example, light emitting pixels (eg, RGB or white) in the detection portion may provide a contour of the detector area or identify simple instructions to the user. In such embodiments, the pixels in the display portion operate independently of the light emitting pixels in the detection area. The light emitting pixels in the detector area can also serve as illumination pixels for the body part to be detected.

  In all embodiments, the detection and control circuits of FIG. 1 can be integrated into the device.

  While particular embodiments of the present invention have been shown and described, changes and modifications can be made in its broader aspects without departing from the invention, and thus the appended claims are intended to It will be apparent to those skilled in the art that all such changes and modifications within the true spirit and scope of the invention should be included within those scope.

Various other designs and uses of such devices are also described.
For example, the present invention provides the following.
(Item 1)
A biosensor device,
A display portion having a light transmissive panel, wherein the display portion emits light at a pixel to generate an image, the pixel generating light having a first set of peak wavelengths; ,
A detector portion configured to receive light through a portion of the light passing panel, wherein the detector portion is laterally spaced from the pixel in the display portion that emits the light. Part,
A light source configured to emit light through the light transmissive panel to be partially reflected and absorbed by a human body part, wherein the light source is used to generate the image A light source other than the pixels in the display portion;
One or more photodetectors in the detector portion spaced laterally from the pixels in the display portion and arranged to detect light reflected back by at least the body portion; The photodetector generates a first signal corresponding to the reflected light back; and
A detection circuit coupled to the one or more photodetectors and configured to analyze the first signal to determine a characteristic of the body part;
A device comprising:
(Item 2)
Item 2. The device of item 1, wherein the pixels generate red, green, and blue light and the light source generates infrared light.
(Item 3)
The light source is optically coupled in proximity to an edge of the light passage panel, and the light passage panel acts as a waveguide to guide the light from the light source and to emit the light from the light source. The device according to item 1, wherein the device emits light through a light emitting surface of the light transmission panel.
(Item 4)
Item 4. The light passing panel is configured such that light is emitted substantially only through the light emitting surface of the light passing panel, wherein the body part is in direct contact with the light emitting surface. Devices.
(Item 5)
The device of claim 1, wherein the light source is laterally spaced from the pixels in the display portion.
(Item 6)
The device of item 1, wherein the light source comprises one or more light emitting diodes.
(Item 7)
Item 2. The device of item 1, wherein the light source comprises a light emitting diode that emits light of different peak wavelengths.
(Item 8)
Item 2. The device of item 1, wherein the light source emits light of a wavelength that is absorbed by a blood vessel.
(Item 9)
The light source comprises a light guide, light from one or more light emitting diodes is optically coupled to the light guide, and the light guide is configured to emit light through its surface; Item 2. The device according to Item 1.
(Item 10)
Item 10. The device according to Item 9, wherein the light passage panel is the light guide.
(Item 11)
The one or more photodetectors are located behind the light guide such that light reflected from the body part passes through the light guide for detection by the one or more photodetectors. 10. The device according to item 9, wherein the device is located.
(Item 12)
The device of item 1, further comprising a lens spanning the one or more photodetectors to limit the angular range of light reflected from the body part.
(Item 13)
Item 1 further comprising a lens across the photodetector to focus light reflected from the body part onto the one or more photodetectors from only a particular distance into the body part. Device described in.
(Item 14)
The device of item 1, further comprising a non-optical sensor integrated with the device.
(Item 15)
15. A device according to item 14, wherein the non-optical sensor comprises an electrode for touching by the body part.
(Item 16)
The device of item 1, wherein the body part comprises a finger and the device is configured to detect a fingerprint.
(Item 17)
The device of claim 1, wherein the body part comprises a finger and the device is configured to detect a blood vessel in the finger.
(Item 18)
The device of item 1, wherein the body part comprises a finger and the device is configured to detect both fingerprints and blood vessels in the finger.
(Item 19)
The device of claim 1, further comprising a processor integrated with the device, wherein the processor controls energization of the light source and detection of the reflected light received by the one or more photodetectors.
(Item 20)
The processor of claim 1, further comprising a processor configured to process data from the photodetector, compare the data with stored data, and indicate authentication of a user of the device. Devices.
(Item 21)
And further comprising a processor, wherein the light source comprises a plurality of light emitting diodes, the processor continuously while signals from one or more of the photodetectors are read and processed by the processor. The device of claim 1, wherein the device is configured to control the light emitting diode to be illuminated.
(Item 22)
The device of claim 1, further comprising a processor, wherein the processor is configured to detect a light reflection image of the body part and to determine an area of absorption of the light within the body part.
(Item 23)
The device of claim 1, further comprising a processor configured to process data from the one or more photodetectors, wherein the processor detects movement of the body part to detect a gesture. .
(Item 24)
A biosensor device,
A display portion having a light transmissive panel, wherein the display portion emits light at a pixel to generate an image, the pixel generating light having a first set of peak wavelengths; ,
A light source optically coupled proximate to an edge of the light passage panel, the light passage panel acting as a light guide to guide the light from the light source and from the light source A light source that emits light through a light emitting surface of the light passing panel, wherein the light source is other than a pixel in the display portion used to generate the image;
One or more photodetectors arranged to detect light reflected back by the body part contacting the light passing panel, wherein the light from the light source is transmitted by the body part to the light; A photodetector that produces a first signal corresponding to the reflected light that is extracted from the light passage panel in an area that contacts the passage panel;
A detection circuit coupled to the one or more photodetectors and configured to analyze the first signal to determine a characteristic of the body part;
A device comprising:
(Item 25)
25. The device of item 24, wherein the one or more photodetectors are laterally spaced from the display portion.
(Item 26)
25. The device of item 24, wherein the one or more photodetectors are distributed within the display portion.
(Item 27)
25. A detector according to item 24, wherein the detection circuit detects a fingerprint, and a vertex of the fingerprint contacts the light passage panel and extracts light from the light passage panel.
(Item 28)
28. The detector according to item 27, wherein the detection circuit also detects a blood vessel pattern in the finger.
(Item 29)
29. A detector according to item 28, wherein the detection circuit detects a combined pattern of fingerprints and blood vessels on the finger.

Claims (29)

A biosensor device,
A display portion having a light transmissive panel, wherein the display portion emits light at a pixel to generate an image, the pixel generating light having a first set of peak wavelengths; ,
A detector portion configured to receive light through a portion of the light passing panel, wherein the detector portion is laterally spaced from the pixel in the display portion that emits the light. Part,
A light source configured to emit light through the light transmissive panel to be partially reflected and absorbed by a human body part, wherein the light source is used to generate the image A light source other than the pixels in the display portion;
One or more photodetectors in the detector portion spaced laterally from the pixels in the display portion and arranged to detect light reflected back by at least the body portion; The photodetector generates a first signal corresponding to the reflected light back; and
A detection circuit coupled to the one or more photodetectors and configured to analyze the first signal to determine a characteristic of the body part.   The device of claim 1, wherein the pixels generate red, green, and blue light and the light source generates infrared light.   The light source is optically coupled in proximity to an edge of the light passage panel, and the light passage panel acts as a waveguide to guide the light from the light source and to emit the light from the light source. The device of claim 1, wherein the device emits light through a light emitting surface of the light passing panel.   The light passing panel is configured to emit light only through the light emitting surface of the light passing panel, wherein the body part is substantially in direct contact with the light emitting surface. The device described.   The device of claim 1, wherein the light source is laterally spaced from the pixels in the display portion.   The device of claim 1, wherein the light source comprises one or more light emitting diodes.   The device of claim 1, wherein the light source comprises a light emitting diode that emits light of different peak wavelengths.   The device of claim 1, wherein the light source emits light of a wavelength that is absorbed by a blood vessel.   The light source comprises a light guide, light from one or more light emitting diodes is optically coupled to the light guide, and the light guide is configured to emit light through its surface; The device of claim 1.   The device of claim 9, wherein the light passage panel is the light guide.   The one or more photodetectors are located behind the light guide such that light reflected from the body part passes through the light guide for detection by the one or more photodetectors. The device of claim 9, which is located.   The device of claim 1, further comprising a lens across the one or more photodetectors to limit an angular range of light reflected from the body part.   The lens further comprising a lens across the photodetector to focus light reflected from the body part only on the one or more photodetectors from a particular distance into the body part. The device according to 1.   The device of claim 1, further comprising a non-optical sensor integrated with the device.   The device of claim 14, wherein the non-optical sensor comprises an electrode for touching by the body part.   The device of claim 1, wherein the body part comprises a finger and the device is configured to detect a fingerprint.   The device of claim 1, wherein the body part comprises a finger and the device is configured to detect blood vessels in the finger.   The device of claim 1, wherein the body part comprises a finger and the device is configured to detect both fingerprints and blood vessels in the finger.   The device of claim 1, further comprising a processor integrated with the device, wherein the processor controls energization of the light source and detection of the reflected light received by the one or more photodetectors. .   The processor of claim 1, further comprising a processor configured to process data from the photodetector, compare the data with stored data, and indicate authentication of a user of the device. The device described.   And further comprising a processor, wherein the light source comprises a plurality of light emitting diodes, the processor continuously while signals from one or more of the photodetectors are read and processed by the processor. The device of claim 1, wherein the device is configured to control the light emitting diode to be illuminated.   The device of claim 1, further comprising a processor, wherein the processor is configured to detect a light reflection image of the body part and determine an area of absorption of the light within the body part.   The processor of claim 1, further comprising a processor configured to process data from the one or more photodetectors, wherein the processor detects movement of the body part to detect a gesture. device. A biosensor device,
A display portion having a light transmissive panel, wherein the display portion emits light at a pixel to generate an image, the pixel generating light having a first set of peak wavelengths; ,
A light source optically coupled proximate to an edge of the light passage panel, the light passage panel acting as a light guide to guide the light from the light source and from the light source A light source that emits light through a light emitting surface of the light passing panel, wherein the light source is other than a pixel in the display portion used to generate the image;
One or more photodetectors arranged to detect light reflected back by the body part contacting the light passing panel, wherein the light from the light source is transmitted by the body part to the light; A photodetector that produces a first signal corresponding to the reflected light that is extracted from the light passage panel in an area that contacts the passage panel;
A detection circuit coupled to the one or more photodetectors and configured to analyze the first signal to determine a characteristic of the body part.   25. The device of claim 24, wherein the one or more photodetectors are laterally spaced from the display portion.   25. The device of claim 24, wherein the one or more photodetectors are distributed within the display portion.   25. The detector of claim 24, wherein the detection circuit detects a fingerprint, and a vertex of the fingerprint contacts the light passage panel and extracts light from the light passage panel.   28. The detector of claim 27, wherein the detection circuit also detects a blood vessel pattern in the finger.   29. The detector of claim 28, wherein the detection circuit detects a combined pattern of fingerprints and blood vessels on the finger.

Images