CN1774692A - Touch Screen Signal Processing - Google Patents

Abstract

A touch screen (1) uses light sources (4) at one or more edges of the screen (1) that direct light across the surface of the screen (1), and at least two cameras (6) with electronic outputs, located at the periphery of the screen (1), that receive light from the light sources (4). A processor receiving the camera (6) output and using triangulation techniques to determine the position of objects in proximity to the screen (1). The detection of the presence of the object includes detecting the presence or absence of direct light due to the object at the camera (6), using the screen surface as a mirror surface and detecting the presence or absence of reflected light due to the object at the camera (6). The light source (4) may be modulated to provide a frequency band in the output of the camera (6).

Description

Touch Screen Signal Processing

field of invention

The present invention relates to touch sensitive screens and more particularly to the optical detection of the presence of objects through the use of signal processing.

technical background

Prior art touch screens come in five main forms. The five forms of touch screen input devices include resistive, capacitive, surface acoustic wave (SAW), infrared (IR), and optical. Each touch screen type has its own characteristics, advantages and disadvantages.

Resistive is the most commonly used type in touch screen technology. It is a low-cost solution found in many touch-screen applications, including handheld computers, PDAs, consumer electronics, and point-of-sale applications. Resistive touch screens use specific coated glass overlays on the controller and display sides to form the touch connection. The main types of resistive overlays are 4-wire, 5-wire, and 8-wire. 5- and 8-wire technologies are more expensive to manufacture and calibrate, while 4-wire offers lower image definition. Usually two options are given: polished or anti-glare. Polishing provides sharp images but often introduces glare. Anti-glare minimizes glare, but will also spread light further, reducing clarity. One of the benefits of using a resistive display is that it can be used with fingers (gloved or not), pens, stylus, or hard objects. However, resistive displays are less effective in public environments due to the loss of image clarity caused by the resistive film layer, and their susceptibility to scratches. Regardless of the trade-off, because of relatively low price (smaller screen size), and the ability to utilize a range of input devices (finger, glove, hard and soft stylus), resistive screens is the most popular technique.

Capacitive touch screens are all glass and are designed for use in ATMs and similar kiosk-type applications. Circuitry sits in the corners of the screen, and a small charge flows through the screen, measuring the capacitance of a person touching the overlay. Touching the screen interrupts the current flow and activates the software that runs the kiosk. Because the glass and the glass frame that mounts it to the monitor can be sealed, the touchscreen is durable and resistant to water, dirt and dust. This makes it commonly used in harsher environments like gaming, vending retail displays, public kiosks, and industrial applications. However, capacitive touch screens can only be activated by the touch of human fingers and gloved fingers, pens, stylus or hard objects will not work. Therefore, it is not suitable for use in many applications, including medical and food preparation.

Surface Acoustic Wave (SAW)-style technology offers better image clarity because it uses an all-glass construction. SAW touchscreens use a glass display overlay. Sound waves propagate across the display surface. Each sound wave propagates through the screen by reflection from an array of reflectors along the edges of the cladding. Two receivers detect sound waves. When the user touches the glass surface, the user's finger absorbs sound waves of a certain energy, and the controller circuit measures the contact position. SAW touchscreen technology is used in ATMs, amusement parks, banking and financial applications, and kiosks. This technique does not secure the seal and is therefore unsuitable for many industrial and commercial applications. Comparing resistive and capacitive technologies, it offers higher image clarity, resolution, and higher light transmission.

Infrared technology relies on the interruption of a grid of infrared light in front of the display. The touch frame or light matrix frame contains an array of infrared LEDs and phototransistors; each mounted on two opposing faces to generate an invisible grid of infrared light. The frame assembly includes a printed circuit board on which optoelectronic devices are assembled, and the frame assembly is hidden behind an infrared transparent glass frame. A glass frame protects the optoelectronic device from the operating environment while allowing the infrared beam to pass through. An infrared controller continuously pulses the LEDs to produce a grid of infrared beams. When a stylus, such as a finger, enters the grid, it blocks the light beam. One or more phototransistors detect the absence of light and emit signals identifying the x and y coordinates. Infrared touchscreens are often used in manufacturing and medical applications because they can be completely sealed and operated with any number of hard or soft objects. The main problem with infrared is that the touch frame "sit" slightly on top of the screen. Therefore, it is susceptible to "premature activation" before a finger or stylus actually touches the screen. The cost of manufacturing the infrared glass frame is also quite high.

Optical imaging of touchscreens uses a combination of line-scan cameras, digital signal processing, forward or rearward lighting, and algorithms to determine touch points. By scanning along the display surface, the imaging lens images the user's finger, stylus or object. This touch screen type is susceptible to false readings due to moving shadows and bright lights, while requiring the screen to be touched before a reading can be taken. Attempts have been made to overcome these disadvantages. Touch screens using optical imaging techniques are disclosed in the following publications.

A touch screen using digital ambient light sampling is disclosed in US4943806. Specifically, this patent discloses a touch input device that can continuously sample and store ambient light reading information and compare it with previous readings. This minimizes the effects of highlights and shadows.

A touch screen for use with a computer system is disclosed in US5914709. In particular, a touch-sensitive user input device using a threshold adjustment process is disclosed. The light intensity value is read and an "on" threshold is established, and this threshold measurement and adjustment is performed frequently and periodically.

US Patent No. 5317140 discloses a method for optically determining the position and orientation of an object on a touch screen display. Specifically, a diffuser is placed above the light source to generate an average light intensity on the touch screen.

US Patent No. 5698845 discloses a touch screen display that uses a light detection device to modulate the on/off frequency of a light emitter at twice the frequency of a commercial AC line light source. The receiver determines the light present and compares it to the actual signal transmitted.

US Patent No. 4,782,328 discloses a touch screen using a light sensor unit, placed at a predetermined height above the touch screen, and when a pointing object approaches the touch screen, its reflected or blocked ambient light beam makes it sensed.

US Patent No. 4,868,551 discloses a touch screen that can detect pointers approaching the display surface by detecting light reflected back from the pointer (reflection or scattering).

Contents of the invention

It is an object of the present invention to provide a touch-sensitive screen for somehow overcoming said disadvantages, or at least providing the public with a beneficial choice.

According to the first aspect of the present invention, the present invention can be broadly regarded as a touch display, wherein the touch display includes:

screens on or in which users touch and view images;

a light source at one or more edges of the screen, the light source directing light across the surface of the screen;

at least two cameras having output signals, said cameras each located at the periphery of said screen, imaging the space in front of said screen, said output signals comprising scanned images;

means for processing said output signal to detect a level of light comprising:

direct light from said light source, and/or

light reflected from said light source;

a processor receiving the camera's processed output signal, the processor using triangulation techniques and the processed output signal to determine whether the processed output signal indicates the presence of an object proximate to the screen, and if so If yes, the location of the object is determined.

The processed output signal preferably indicates the relative orientation of the assumed object position with respect to the camera.

The processed output signal preferably indicates the relative orientation of the assumed object position with respect to the lens center of the camera.

Preferably the processor determines the position of the object in planar screen coordinates.

Preferably said light source is arranged behind said screen for projecting light through said screen, and said display comprises, with a light source at each edge, a light reflector in front of said screen, directing said light source The emitted light passes through the surface of the screen.

Preferably said camera is a line scan camera, said camera output signal comprises information on a scan line, and said processor uses said information to determine the position of said object.

Preferably, the touch display includes:

means for modulating said light emitted by said light source to provide a frequency band within the imageable range of said camera;

Means for excluding image data outside said frequency band.

Preferably said means for processing said output signal comprises said means for excluding image data outside said frequency band, and said means for excluding image data outside said frequency band comprises a filtering function.

Preferably said filtering function comprises applying a filter selected from the set comprising:

comb filter;

high pass filter;

notch filter; and

bandpass filter.

Preferably, the touch display includes:

means for controlling said light source; and

means for obtaining and processing images, wherein the images are obtained in a non-illuminated ambient light state as well as in an illuminated state;

Wherein said means for processing said output signal subtracts the ambient light state from the light emission state prior to detecting the light level.

Preferably, the light source is an LED, and the touch display includes means for controlling the operation of a part of the light source, wherein this part of the light source is independent of other parts of the light source.

Preferably the means for controlling the operation of a portion of said light source comprises means for independently controlling the effective intensity of said light source.

Preferably, said means for controlling part of said light source comprises wiring said part in reverse and driven by a bridge driver.

Preferably the means for controlling part of said light source comprises the use of a diagonal bridge driver.

Preferably the means for controlling parts of said light source comprises the use of shift registers for each part being controlled.

It is preferred that said means for obtaining and processing an image comprises controlling part of said light source and said each camera, and said means for processing said output signal comprises processing information to determine whether said part is lit illuminated or unlit of.

It is preferred that some parts are illuminated while others are not illuminated when receiving an image.

According to a second aspect of the present invention, the present invention can be broadly considered to lie in a touch display, wherein the touch display comprises:

screens on or in which users touch and view images;

a light source at one or more edges of the screen, the light source directing light across the surface of the screen;

at least two cameras located at the periphery of the screen with output signals positioned so that they do not receive direct light from the light source, each camera imaging the space in front of the screen, the output signals including scanned images;

means for processing said output signal to detect the level of reflected light; and

a processor receiving the camera's processed output signal, the processor using triangulation techniques and the processed output signal to determine whether the processed output signal indicates the presence of an object proximate to the screen, and if so If affirmative, indicate the location of said object.

The processed output signal preferably indicates the relative orientation of the assumed object position with respect to the camera.

The processed output signal preferably indicates the relative orientation of the assumed object position with respect to the lens center of the camera.

Preferably the processor determines the position of the object in planar screen coordinates.

Preferably, the touch display includes:

means for modulating said light emitted by said light source to provide a frequency band within the imageable range of said camera;

Means for excluding image data outside said frequency band.

Preferably said means for processing said output signal comprises said means for excluding image data outside said frequency band, and said means for excluding image data outside said frequency band comprises a filtering function.

Preferably said filtering function comprises applying a filter selected from the set comprising:

comb filter;

high pass filter;

notch filter; and

bandpass filter.

Preferably, the touch display includes:

means for controlling said light source; and

means for obtaining and processing images, wherein the images are obtained in a non-illuminated ambient light state as well as in an illuminated state;

Wherein said means for processing said output signal subtracts the ambient light state from the light emission state prior to detecting the light level.

Preferably, the light source is an LED, and the touch display includes a device for controlling part of the light source, wherein this part is independent of other parts of the light source.

Preferably the means for controlling the operation of a portion of said light source comprises means for independently controlling the effective intensity of said light source.

Preferably the means for controlling part of said light source comprises wiring said part in inverse phase and driven using a bridge driver.

Preferably the means for controlling said light source section comprises the use of a diagonal bridge driver.

Preferably the means for controlling part of said light source comprises the use of a diagonal bridge driver.

Preferably the means for controlling parts of said light source comprises the use of shift registers for each part being controlled.

Preferably said means for obtaining and processing images comprises controlling said light source portion and said each camera, and said means for processing said output signal comprises processing information to determine whether said portion is illuminated or not illuminated.

It is preferred that when an image is obtained some parts are illuminated while others are not.

Preferably said screen is reflective, said camera further images said screen and said means for processing output signals detects light levels from the mirror image.

The processed output signal preferably indicates an assumed relative orientation of the object with respect to the camera and a distance between the object and the screen.

According to a third aspect of the invention, the invention may be broadly regarded as residing in a method of receiving a user input signal about an image, wherein the method comprises the steps of:

provide a screen on or in which the user touches and views the image;

providing a light source at one or more edges of the screen, the light source directing light through the surface of the screen;

providing at least two cameras having an output signal, said camera each located at the periphery of said screen, imaging the space in front of said screen, said output signal comprising a scanned image;

The output signal is processed to detect levels of light comprising:

direct light from said light source, and/or

reflected light from the light source;

The processed output signal of the camera is processed to obtain the position of the object using a triangulation technique.

The processed output signal preferably indicates the relative orientation of the assumed object position with respect to the camera.

The processed output signal preferably indicates the relative orientation of the assumed object position with respect to the lens center of the camera.

Preferably said position is a planar screen coordinate.

Preferably said light source is arranged behind said screen for projecting light through said screen, and said display comprises, with a light source at each edge, a light reflector in front of said screen, directing said light source The emitted light traverses the surface of the screen.

Preferably said camera is a line scan camera, said camera output signal comprises information on a scan line, and said processor uses said information to determine the position of said object.

Preferably the method comprises the steps of:

means for modulating said light emitted by said light source to provide a frequency band within the imageable range of said camera;

Means for excluding image data outside said frequency band.

Preferably, the step of processing said output signal comprises the step of excluding image data outside said frequency band, and said step of excluding image data outside said frequency band comprises filtering.

Preferably said filtering comprises the step of applying a filter selected from the set comprising:

comb filter;

high pass filter;

notch filter; and

bandpass filter.

Preferably the method comprises the steps of:

controlling the light source, and

obtaining and processing an image, wherein the image is obtained in a non-illuminated ambient light state as well as an illuminated state;

Wherein said step of processing said output signal subtracts ambient light status from lighting status prior to detecting light level.

Preferably, the light source is an LED, and the touch display includes a device for controlling part of the light source, wherein this part is independent of other parts of the light source.

Preferably the step of controlling the operation of said light source comprises independently controlling the effective intensity of said light source.

Preferably, the step of controlling the portion of the light source includes wiring the portion in reverse phase and driving it with a bridge driver.

Preferably the step of controlling part of said light source comprises using a diagonal bridge driver.

Preferably the step of controlling portions of said light source comprises using a shift register for each portion being controlled.

Preferably said step of receiving and processing images includes controlling portions of said light source and said each camera, and said step of processing said output signals includes processing information to determine whether said portions are illuminated or non-illuminated.

It is preferred that when an image is obtained some parts are illuminated while others are not.

According to a fourth aspect of the invention, the invention may be broadly regarded as residing in a method of receiving user input about an image, wherein the method comprises the steps of:

provide a screen on or in which the user touches and views the image;

providing a light source at one or more edges of the screen, the light source directing light through the surface of the screen;

providing at least two cameras located at the periphery of the screen with an output signal, the cameras positioned so that they do not receive direct light from the light source, imaging the space in front of the screen, the output signal comprising a scanned image;

means for processing said output signal to detect the level of reflected light; and

processing the camera processed output signal, using triangulation techniques and the processed output signal, determining whether the processed output signal indicates the presence of an object proximate to the screen, and if so, indicates the object s position.

The processed output signal preferably indicates the relative orientation of the assumed object position with respect to the camera.

The processed output signal preferably indicates the relative orientation of the assumed object position with respect to the lens center of the camera.

Preferably the processor determines the position of the object in planar screen coordinates.

Preferably the method comprises:

means for modulating said light emitted by said light source to provide a frequency band within the imageable range of said camera;

Means for excluding image data outside said frequency band.

Preferably said means for processing said output signal comprises said means for excluding image data outside said frequency band, and said means for excluding image data outside said frequency band comprises filtering.

Preferably said filtering comprises applying a filter selected from the set comprising:

comb filter;

high pass filter;

notch filter; and

bandpass filter.

Preferably the method comprises:

means for controlling said light source; and

means for obtaining and processing images, wherein the images are obtained in a non-illuminated ambient light state as well as in an illuminated state;

Wherein said means for processing said output signal subtracts the ambient light state from the light emission state prior to detecting the light level.

Preferably, the light source is an LED, and the touch display includes a device for controlling part of the light source, wherein this part is independent of other parts of the light source.

Preferably the means for controlling the operation of a portion of said light source comprises means for independently controlling the effective intensity of said light source.

Preferably the means for controlling part of said light source comprises wiring said part in inverse phase and driven using a bridge driver.

Preferably the means for controlling part of said light source comprises the use of a diagonal bridge driver.

Preferably the means for controlling parts of said light source comprises the use of shift registers for each part being controlled.

Preferably said means for obtaining and processing images includes controlling said light source portion and said each camera, and said means for processing said output signal includes processing information to determine whether said portion is illuminated or not illuminated.

It is preferred that when an image is obtained some parts are illuminated while others are not.

Preferably said screen is reflective, said camera further images said screen and said means for processing output signals detects light levels from the mirror image.

The processed output signal preferably indicates an assumed relative orientation of the object with respect to the camera and a distance between the object and the screen.

According to a fifth aspect of the invention, the invention may be broadly regarded as residing in a method of receiving user input about an image:

providing at least one light source at or near the periphery of the image, the light source directing light through the image;

detecting light levels at at least two locations on or near the periphery of said image and providing said levels as output signals;

The output signal is processed using triangulation techniques to determine whether the output signal indicates the presence of an object proximate to the image, and if so, the location of the object.

Preferably said positions are substantially non-relative such that when an object is present said output signal is substantially indicative of reflected light from said object.

According to the sixth aspect of the present invention, the present invention can be broadly regarded as an image-related user input device for locating objects, including:

at least one light source located at or near the periphery of the image, the light source directing light through the image;

at a detector having an output located at or near the image, imaging the space in front of the screen, the output signal indicating the level of light; receiving the output signal and using triangulation techniques and the processor, and the output signal determines the presence of the object and, if present, indicates the location of the object.

Brief description of the drawings

A preferred form of the invention will now be described with reference to the accompanying drawings;

Figure 1 is a graphical illustration of a front view of a preferred embodiment of the touch screen of the present invention,

Figure 1a is an illustration of a cross-sectional view taken along line X-X in Figure 1,

Figure 1b is a diagram of the forward lighting of the preferred embodiment of the touch screen of the present invention,

Fig. 2 is the illustration of the mirror image effect of the preferred embodiment of the touch screen of the present invention,

Figure 2a is a block diagram of a filtering implementation of a preferred embodiment of the touch screen of the present invention,

Figure 2b is a graphical illustration of pixels captured by an area camera and transmitted to a processing module in a preferred embodiment of the invention,

Fig. 3 is the system block diagram of the preferred embodiment of touch screen of the present invention,

Fig. 4 is a side view of determining the position of an object using the image signal in a preferred embodiment of the touch screen of the present invention,

Fig. 4a is a top view of using the image signal to determine the position of an object in a preferred embodiment of the touch screen of the present invention,

FIG. 5 is a diagram of a calibration method in a preferred embodiment of the touch screen of the present invention,

Fig. 6 is a graph showing the output signal of the imager in the processing module in the frequency domain in a preferred embodiment of the touch screen of the present invention,

Fig. 6a is a graph representing the output signal of the filter response imager in the frequency domain in a preferred embodiment of the touch screen of the present invention,

Fig. 6b is a graph representing the separation of an object and a background in the frequency domain after two types of filtering in a preferred embodiment of the touch screen of the present invention,

Figure 7 is an illustration of a front view of an alternative embodiment of the touch screen of the present invention,

Figure 7a is an illustration of a cross-sectional view taken along line X-X in an alternative embodiment of the touch screen of the present invention,

Figure 7b is an illustration of the backlighting of an alternative embodiment of the touch screen of the present invention,

Figure 7c is an illustration of the backlighting controlling the sensing height of an alternative embodiment of the present invention,

Figure 7d is a graphical illustration of pixels captured by a line scan camera and transmitted to a processing module in an alternative embodiment of the invention,

Fig. 8 is a graph representing a simple separation of an object and a background in an optional embodiment of the present invention,

Figure 9 shows different drive arrangements for partial backlighting of the present invention,

Figure 9a shows the two-segment backlight driven by two lines in the present invention,

Figure 9b shows a twelve-segment backlight driven by four lines in the present invention, and

Figure 9c shows an example of a distributed shift register backlight in the present invention.

Detailed description of the invention

The present invention relates to improvements in signal processing in the field of optical imaging touch screens. In a preferred embodiment, the optical touch screen uses forward lighting and consists of a screen, a series of light sources, and at least two area scan cameras, where the cameras are placed in the same plane and at the edge of the screen. In another embodiment, an optical touch screen uses backlighting; the screen is surrounded by an array of light sources placed behind the touch panel, where the light sources are redirected across the surface of the screen. Use at least two line scan cameras in the same plane as the touch screen panel. The resulting improvement in signal processing from these implementations is that objects can be sensed when they are in close proximity to the touchscreen surface, calibration is simple, and perception of objects is not affected by changing ambient light conditions, such as shifting light or shadows.

A block diagram of a conventional touch screen system 1 is shown in FIG. 3 . Information flows from the camera 6 to a video processing unit and computer, collectively referred to as processing module 10 . Processing module 10 performs many types of calculations, including filtering, data sampling, and triangulation, and controls the modulation of illumination source 4 .

forward-lit touchscreen

A preferred embodiment of the touch screen of the present invention is shown in FIG. 1 . The touch screen system 1 consists of a monitor 2 , a touch screen panel 3 , at least two light sources 4 , a processing module (not shown) and at least two area scan cameras 6 . Behind the touch screen panel 3 is a monitor 2 that displays information to the user. Below the touch screen panel 3 and the monitor 2 are an area scan camera 6 and a light source 4 . The light source 4 is preferably a light emitting diode (LED), but could be another type of light source, eg a fluorescent tube. Since they can be modulated as desired, LEDs can ideally be used, which have no intrinsic switching frequency. Camera 6 and LED 4 are located on the same plane as touch panel 3.

Referring to FIG. 1a, the field of view 6a of the area scan camera 6 and the radiation path 4a of the LED 4 are located in the same plane and parallel to the touch panel 3. When an object 7, shown as a finger, enters the radiation path 4a, it is irradiated. This is typically considered front panel lighting or object lighting. In Fig. 1b, this principle is clarified again. As soon as the finger 7 enters the radiation field 4 a, the signal is reflected back to the camera 6 . This indicates that the finger 7 approaches or touches the touch panel 3 . In order to determine whether the finger 7 actually touches the touch panel 3, the position of the touch panel 3 must be determined. Use another signal, the mirror signal, to perform this step.

image signal

A mirror image signal is generated when an object 7 approaches the touch panel 3 . Preferably, the touch panel 3 is made of reflective glass. As shown in FIG. 2 , the finger 7 is located above the touch panel 3 at a distance 8 and its mirror image in the touch panel 3 is 7a. A camera 6 (only shown as a camera lens) images the finger 7 and its reflection 7a. The image of finger 7 is reflected in panel 3 as 7a; this can be observed by field lines 6b, 6c and virtual field line 6d. This allows the camera 6 to image a reflected image 7a of the finger 7 . The data produced by the camera 6 corresponds to the position at which the field lines 6e, 6b enter the camera 6 . This data then enters the processing module 10 for analysis.

A portion of the processing module 10 is shown in Figure 2a. Within the processing module 10 are a series of scanning imagers 13 implemented in software, as well as digital filters 11 , and comparators 12 . There is a fixed number of pixels on the touch panel, for example 30,000 pixels. These can be broken down into 100 columns of 300 pixels. The number of pixels can be more or less than used, this number is for example only. In this case, there are 30,000 digital filters 11 and comparators 12 divided into 300 pixels in 100 columns, which form a matrix similar to that formed by the pixels on the monitor 2 . Figure 2a shows the representation of a column, realized by means of an image scanner 13 and three groups 14a, 14b, 14c of digital filters 11 and comparators 12, which allow the information in three pixels to be read. A more illustrative example of this matrix is shown in Figure 2b. Eight pixels 3a-3h are connected in clusters of columns to an image scanner 13, which in turn is connected to a filter 11 and a comparator 12 (as part of the processing module 10). The number used in Figure 2b is for illustration only; the exact number of pixels may be larger or smaller than this number. The pixels shown in the illustration may not form the shape in the panel 3, their shape will be dictated by the location and type of camera 6 used.

Referring back to Figure 2, finger 7 and mirrored finger 7a activate at least two pixels; two pixels are used for simplicity. This is shown by the field lines 6e and 6b entering the processing module 10 . This activates the software so that the two signals pass through the digital filter 11 and comparator 12 and result in the formation of digital signal outputs 12a-12e. Comparator 12 compares the output signal of filter 11 with a predetermined threshold. If the finger 7 in question is detected at the pixel, the output signal will be high, otherwise it will be low.

The mirror signal also provides information on the position of the finger 7 with respect to the camera 6 . The height 8 of the finger 7 above the panel 3 as well as the angular position of the finger 7 can be determined. The information gathered from the mirrored signal is sufficient to determine where the finger 7 is positioned with respect to the panel 3 without the finger 7 needing to touch the panel 3 .

Figures 4 and 4a illustrate the position information that can be obtained from image signal processing. Position information is given in polar coordinates. The position information relates to the height of the finger 7 and the position of the finger 7 is above the panel 3 .

Referring again to FIG. 2 , the height above the panel 3 at which the finger 7 is located can be seen in the distance between the outputs 12a-12e. In this example, with finger 7 at height 8 above the panel, outputs 12b and 12e generate a high signal. The other outputs 12a, 12d output low signals. It can be found that the distance 9 between the high outputs 12b, 12e is twice the height 8 of the finger above the panel 3 .

modulation

The processing module 10 modulates and calibrates the LEDs 4 and sets the sampling rate. The LED 4 is modulated, in the simplest embodiment the LED 4 is switched on and off at a predetermined frequency. Other types of modulation are possible, such as sine wave modulation. Modulating the LED 4 at a high frequency results in a reading frequency (when sensing the finger 7) that is significantly greater than any other frequency produced by changing light and shade. The modulation frequency is greater than 500Hz but not exceeding 10kHz.

sampling

The camera 6 continuously generates an output signal which is periodically sampled by the processing module 10 due to data and time constraints. In a preferred embodiment, the sampling rate is at least twice the modulation frequency; this is to avoid aliasing. The modulation of the LED and the sampling frequency do not need to be synchronized.

filtering

In the frequency domain, the output signal of the scanning imager 13 is shown in FIG. 6 . In Fig. 6, there are two typical curves, one shown as 21 when no object is sensed, and one shown as 20 when a finger is sensed. In both curves, at about 5 to 20 Hz there is a moving region of shadow 22 and at about 50 to 60 Hz there is an AC mains frequency region 23 .

In a preferred embodiment, when there is no object in the field of view, no signal is transmitted to the area camera, so there are no other peaks in the output signal. When an object is in the field of view, there is a signal 24 corresponding to the LED modulation frequency, eg 500 Hz. Lower unwanted frequencies 22, 23 can be removed by different forms of filters. Filter types may include comb, high pass, notch, and bandpass filters.

In FIG. 6 a the output signal of the image scanner is shown together with a pair of different filter responses 26 , 27 applied to the signal 20 . In a simple implementation, a 500 Hz comb filter 26 could be implemented (if a modulation frequency of 500 Hz is used). This removes only the lowest frequencies. More advanced implementations involve the use of bandpass27 or notch filters. In this case, all data is removed except for the region where the desired frequency is expected to be obtained. A 500 Hz narrowband filter 27 applied to a signal 20 with a modulation frequency of 500 Hz is shown in Fig. 6a. The output signals 30, 31 of the filters 26, 27 are further shown in Fig. 6b. The upper graph shows the output signal 30 using the comb filter 26 , while the lower graph shows the output signal 31 using the bandpass filter 27 . A bandpass filter 27 removes any unwanted signals away from the area of interest.

Once the signal is filtered and those within the area of interest are identified, the resulting signal is passed through a comparator, converted to a digital signal, and triangulation is performed to determine the actual location of the object. Triangulation is known in the prior art and disclosed in US5534917 and US4782328, which are hereby incorporated by reference.

calibration

The preferred embodiment of the touchscreen of the present invention is very quick to use and easy to calibrate, allowing the touchscreen to be used in any situation and allowing it to be moved to a new location, such as making the touchscreen into a laptop. Calibration involves the touch panel 3 in three different positions 31a, 31b, 31c, as shown in Figure 5; this defines the touch plane of the touch panel 3. These three contact points 31a, 31b, 31c provide sufficient information to a processing module (not shown) to calculate the position and size of the touch plane with respect to the touch panel 3 . As before, each contact point 31a, 31b, 31c uses mirrored and direct signals to generate the required data. These contact points 31a, 31b, 31c may vary around the panel 3, they need not be the actual positions shown.

backlit touch screen

Figure 7 shows an alternative embodiment of the touch screen of the present invention. As in the preferred embodiment, the monitor 40 is located behind the touch panel 41 and the light source array 42 surrounds the sides and lower edge of the panel 41 . These rays are directed outward, towards the user, and are redirected across the panel 41 by the diffuser plate 43 . Light source array 42 consists of a number of light emitting diodes (LEDs). The diffuser plate 43 is used to change the direction of the light emitted by the LED 42 and diffuse it across the panel 41. At least two line scan cameras 44 are located at two upper corners of the panel 3 and are capable of imaging objects. Camera 44 may optionally be located anywhere on the outer perimeter of panel 41 . Surrounding the periphery of the touch panel 41 is a bezel 45 or housing. The bezel 45 serves as a frame to prevent the transmitted light from being transmitted to the external environment. The glass frame 45 reflects the light beam into the camera 44 so that when no object is close to the touch panel 41, the light signal can always be read into the camera 44.

Alternatively, the light source array 42 can be replaced by cold cathode tubes. When a cold cathode tube is used, it is not necessary to use the diffuser plate 43 as the outer tube of the cathode tube for diffusing light. The cold cathode tubes are arranged along the entire length of one side of the panel 41 . This provides a substantially uniform light intensity over the surface of the panel 41 . The use of cold cathode tubes is not preferred because it is difficult and expensive to modify them to fit the specific length of each side of panel 41 . The use of LEDs allows greater flexibility in the size and shape of the panel 41 .

When the light source array 42 is composed of many LEDs, the diffusion plate 43 is used. The diffuser plate 43 is used to diffuse the light emitted by the LEDs and change its direction onto the surface of the panel 41 . As shown in Figure 7a, the light 47 emitted by the LED 42 starts its path at a right angle to the panel 41. Once it hits the diffuser plate 43, it changes direction, parallel to the panel 41. Light 47 propagates slightly above the surface of panel 41 to irradiate panel 41 . Light 47 is collimated and modulated by a processing module (not shown) as previously described.

Referring to Figure 7a, the width 46 of the bezel 45 can be increased or decreased. Increasing the width 46 of the bezel 45 can increase the distance at which objects can be perceived. Similarly, the reverse is used to reduce the width 10 of the bezel 45 .

The line scan camera 44 is composed of a CCD element, a lens, and a drive control circuit. When the camera 44 captures an image, a corresponding output signal is generated.

Referring to Figures 7b and 7c, when the touch screen is not in use, that is, when there is no user interaction or input, all light emitted by the light source array 42 is transmitted to the line scan camera 44 . When there is user input, that is, when the user selects something on the screen by touching the screen with their finger; a portion of the light transmitted to the camera 44 is interrupted. The activation location can be determined by computing the output data from the camera 44 using a triangulation algorithm.

The line scan camera 44 can read two light variables, namely the direct light transmitted by the LED 42 and the reflected light. The method of sensing and reading direct and mirrored light is similar to that described previously, but simpler as a line-scan camera can only read one column of the panel at a time; when using an area-scan camera, the panel is not broken down into a matrix. Figure 7d shows the situation where the panel 41 is broken down into parts 14a-14d (which the line scan camera can capture). The rest of the processing has been described previously. The pixels shown in this illustration may not form the shape in the panel 41, their shape will be dictated by the location and type of camera 44 used.

In an alternative embodiment, since the glass frame surrounds the touch panel, the line scan camera will continuously read the modulated light transmitted by the LED. This will cause the modulation frequency to appear in the output signal whenever there is no object interrupting the light path. When an object interrupts the light path, the modulation frequency in the output signal will not appear. This indicates that an object is approaching or touching the touch panel. The frequency heights present in the output signal are twice as high (twice the amplitude) as in the preferred embodiment. This is due to two sets of signals (direct and mirrored) appearing at once.

In another alternative embodiment, shown in FIG. 8 , the output signal of the camera is sampled when the LED is modulated on and off. This provides reading 50 of ambient light plus backlighting, as well as reading 51 of ambient light alone. When an object interrupts the light emitted by the LED, there is a dip 52 in the output signal 50 . Smaller dips 52 are difficult to observe when the ambient light varies greatly. For this reason, the ambient light reading 51 is subtracted from the ambient light and backlight reading 50 . This results in an output signal 54 in which dips 52 can be observed, so that dips 52 can be identified using a simple threshold.

The calibration of the alternative embodiment is performed in the same way as previously described, but the contact points 31a, 31b, 31c (cf. FIG. 5) cannot be in a line, they must be spread over the surface of the panel 3.

In Figure 7, the backlighting is broken down into a number of separate parts, 42a to 42f. A section or a subset of sections is active at any time. A subset of pixels of image sensor 44 images each portion. Backlight emitters operate at higher currents for shorter periods of time compared to systems utilizing a single backlight control. Because the average power of the emitter is limited, the peak brightness is increased. Increased peak brightness improves ambient light performance.

The switching of the backlighting can advantageously be arranged so that when one part is illuminated, the ambient light level of the other part can be measured by the signal processor. The speed of a single backlight system is improved by simultaneously measuring the ambient light and backlight components.

Adaptively adjust backlight brightness by controlling LED current or pulse width, activating each section to maintain a constant ratio of signal to noise plus ambient light for pixels viewing that section while using minimal average power.

Control multiple parts with a minimum number of control lines from one of multiple methods.

In a first implementation of a two-part backlight, two sets of diodes 44a, 44b may be wired in antiphase and driven by a bridge driver.

In a second implementation with more than two sections, a diagonal bridge drive is used. In Figure 9b, 4 wires can select one of 12 segments, 5 wires can drive 20 segments, and 6 wires can drive 30 segments.

In a third implementation 9c, for a large number of parts, the shift registers 60 are physically distributed around the backlight and only two control lines are required.

X-Y multiplex arrangements are well known in the art. For example, 8+4 lines are used to control the 4-digit display of 32 LEDs. Figure 9b shows a 4-wire diagonal multiplex arrangement of 12 LEDs. The control lines A, B, C, D are driven by tri-state output signals such as are commonly available in pins of microprocessors such as the Microchip PIC family. Each tri-state output signal has two electronic switches, usually FETs. One switch can be on or both switches can be off. To activate LED L1a, only switches A1 and B0 can be used. To start L1B, operate A0 and B1. To activate L2a, use switches A1 and D0, etc. This arrangement can be used with any number of control lines, but is particularly advantageous for 4, 5, 6 control lines where 12, 20, 30 LEDs can be controlled while keeping the printed circuit board traces relatively simple. Where higher control quantities are used, it is advantageous to use a degenerated form, which omits some possible LEDs, easing the difficulty of the actual interconnection.

A diagonal multiplex system has the following properties:

— Favors situations where 4 or more control lines exist

- Requires three-state push-pull drive for each control line

- Better than an x-y arrangement using LED control lines at intersections, the arrangement manifests itself as a loop of control lines and a pair of inverting LEDs disposed on each set of diagonals between the control lines. Each LED can be selected uniquely or in specific combinations.

— Use the smallest number of lines possible

—When emc filtering needs to be used on the line, the components are obviously saved

For those skilled in the art of this invention, there are also many changes in structure, different embodiments and applications in the present invention without departing from the scope of the invention defined in the appended claims. The disclosures and descriptions herein are purely illustrative and not intended to limit understanding in any way.

Claims (71)

1. A touch display, comprising: screens on or through which users touch and view images; a light source at one or more edges of the screen, the light source directing light across the surface of the screen; at least two cameras having an output, each said camera positioned at the periphery of said screen imaging the space in front of said screen, said output comprising a scanned image; means for processing said output to detect levels of light comprising: direct light from said light source, and/or reflected light from said light source; a processor receiving processed output from said camera, said processor utilizing triangulation techniques with said processed output to determine whether the processed output indicates the presence of an object proximate to said screen, and if so If affirmative, indicates the position of the object. 2. The touch display of claim 1, wherein the processed output represents a relative orientation of an assumed object position with respect to the camera. 3. A touch display according to claim 1 or 2, wherein the processed output represents the relative orientation of an assumed object position with respect to the camera's lens center. 4. A touch display according to any one of claims 1 to 3, wherein the processor determines the position of the object in planar screen coordinates. 5. A touch display according to any one of claims 1 to 4, wherein the light source is arranged behind the screen to project light through the screen, and the display comprises, at each edge, a light source , a light reflector in front of the screen directing the light emitted by the light source across the surface of the screen. 6. A touch display according to any one of claims 1 to 5, wherein the camera is a line scan camera, the camera output includes information on the scan line, and the processor uses the information to determine the object s position. 7. A touch display according to any one of claims 1 to 6, comprising: means for modulating said light emitted by said light source to provide a frequency band within the imageable range of said camera; Means for excluding image data outside said frequency band. 8. The touch display according to claim 7 , wherein said means for processing said output comprises said means for excluding image data outside said frequency band, and said excluding image data outside said frequency band The means include filtering. 9. The touch display of claim 8, wherein filtering comprises applying a filter selected from the group consisting of: comb filter; high pass filter; notch filter; and bandpass filter. 10. A touch display according to any one of claims 1 to 9, comprising: means for controlling said light source; and means for obtaining and processing images, wherein the images are obtained in a non-illuminated ambient light state as well as in an illuminated state; wherein said means for processing said output subtracts the ambient state from the lighting state prior to detecting said light level. 11. The touch display according to any one of claims 1 to 10, wherein the light source is an LED, and the touch display includes means for controlling a part of the light source, wherein the part controlled is independent of the Other parts of the light source. 12. The touch display of claim 11, wherein the means for controlling partial operation of the light source comprises means for independently controlling the effective intensity of the light source. 13. A touch display according to claim 11 or 12, wherein the means for controlling the light source portion comprises wiring the portion in reverse and driving it using a bridge driver. 14. A touch display as claimed in claim 11 or claim 12, wherein the means for controlling the light source portion comprises using a diagonal bridge driver. 15. A touch display as claimed in claim 11 or claim 12, wherein the means for controlling the light source sections comprises using a shift register for each section being controlled. 16. A touch display according to any one of claims 11 to 15, wherein said means for obtaining and processing an image comprises controlling said light source portion and said each camera, and said means for processing said output comprises processing Information about whether the part is illuminated or not. 17. The touch display of claim 16, wherein certain parts are illuminated while others are not illuminated when an image is acquired. 18. A touch display comprising: screens on or through which users touch and view images; a light source at one or more edges of the screen, the light source directing light across the surface of the screen; at least two cameras located at the periphery of the screen with outputs arranged so that they do not receive direct light from the light source, each of the cameras imaging the space in front of the screen, the outputs comprising scanned images; means for processing said output to detect the level of reflected light; and a processor receiving the camera's processed output, the processor using triangulation techniques with the processed output to determine whether the processed output indicates the presence of an object proximate to the screen, and if so The words represent the position of the object. 19. The touch display of claim 18, wherein the processed output represents a relative orientation of an assumed object location with respect to the camera. 20. A touch display as claimed in claim 18 or 19, wherein the processed output represents the relative orientation of an assumed object position with respect to the camera's lens center. 21. A touch display according to any one of claims 18 to 20, wherein the processor determines the position of the object in planar screen coordinates. 22. A touch display according to any one of claims 18 to 21 comprising: means for modulating said light emitted by said light source to provide a frequency band within the imageable range of said camera; Means for excluding image data outside said frequency band. 23. The touch display of claim 22 , wherein said means for processing said output comprises said means for excluding image data outside said frequency band, and said excluding image data outside said frequency band The means include filtering. 24. The touch display of claim 23, wherein said filtering comprises applying a filter selected from the group consisting of: comb filter; high pass filter; notch filter; and bandpass filter. 25. A touch display according to any one of claims 18 to 24, comprising: means for controlling said light source; and means for obtaining and processing images, wherein the images are obtained in a non-illuminated ambient light state as well as in an illuminated state; wherein said means for processing said output subtracts the ambient light state from the lighting state prior to detecting the light level. 26. A touch display according to any one of claims 18 to 25, wherein said light source is an LED, and said touch display includes means for controlling the operation of a portion of said light source, wherein said portion being controlled is independent of said Other parts of the light source. 27. The touch display of claim 26, wherein the means for controlling partial operation of the light source comprises means for independently controlling the effective intensity of the light source. 28. A touch display as claimed in claim 26 or 27, wherein the means for controlling part of the light source comprises wiring the part in reverse and driving it using a bridge driver. 29. A touch display as claimed in claim 26 or 27, wherein the means for controlling part of the light source comprises the use of a diagonal bridge driver. 30. A touch display as claimed in claim 26 or 27, wherein the means for controlling portions of the light source comprises using a shift register for each portion being controlled. 31. A touch display according to any one of claims 26 to 30, wherein said means for obtaining and processing an image comprises part of controlling said light source and said each camera, and said means for processing said output comprises Information about whether the part is illuminated or not illuminated is processed. 32. The touch display of claim 31 , wherein when an image is acquired, certain parts are illuminated and others are not. 33. A touch display according to any one of claims 18 to 31, wherein said screen is reflective, said camera further images said screen, and said means for processing output detects this light from the mirror image grade. 34. The touch display of claim 33, wherein the processed output represents an assumed relative orientation of an object with respect to the camera and a distance between the object and the screen. 35. A method of receiving user input regarding an image comprising the steps of: provide a screen on or through which the user can touch and view the image; providing a light source at one or more edges of the screen, the light source directing light across the surface of the screen; providing at least two cameras having an output, said camera each located at the periphery of said screen, imaging the space in front of said screen, said output comprising a scanned image; The output is processed to detect levels of light including: direct light from said light source, and/or reflected light from said light source; The processed output of the camera is processed to obtain the position of the object using triangulation techniques. 36. A method of receiving user input with respect to an image as claimed in claim 35, wherein said processed output represents a relative orientation of an assumed object location with respect to said camera. 37. A method of receiving user input with respect to an image as claimed in claim 35 or claim 36, wherein the processed output represents the relative orientation of an assumed object position with respect to the camera's lens center. 38. A method of receiving user input with respect to an image as claimed in any one of claims 35 to 37, wherein said location is a planar screen coordinate. 39. A method of receiving user input with respect to an image as claimed in any one of claims 35 to 38, wherein said light source is behind said screen, arranged to project light through said screen, and said display comprises, With a light source at each edge, a light reflector in front of the screen directs the light emitted by the light source across the surface of the screen. 40. A method of receiving user input with respect to an image as claimed in any one of claims 35 to 39, wherein said camera is a line scan camera, said camera output comprises information on a scan line, and said processor uses said The information is used to determine the position of the object. 41. A method of receiving user input with respect to an image as claimed in any one of claims 35 to 40, comprising the steps of: modulating the light emitted by the light source to provide a frequency band within the imageable range of the camera; Image data outside the frequency band is excluded. 42. A method of receiving user input regarding an image as claimed in claim 41 , wherein the step of processing said output includes the step of excluding image data outside said frequency band, and said excluding image data outside said frequency band The steps include filtering. 43. A method of receiving user input with respect to an image as claimed in claim 42, wherein filtering comprises the step of applying a filter selected from the group consisting of: comb filter; high pass filter; notch filter; and bandpass filter. 44. A method of receiving user input with respect to an image as claimed in any one of claims 35 to 43, comprising the steps of: controlling the light source; and obtaining and processing an image, wherein the image is obtained in a non-illuminated ambient light state as well as an illuminated state; wherein said step of processing said output subtracts the ambient state from the lighting state prior to detecting the light level. 45. The method of receiving user input about an image according to any one of claims 35 to 44, wherein the light source is an LED, and the touch display includes means for controlling part of the light source, wherein the controlled This part is independent of other parts of the light source. 46. A method of receiving user input regarding an image as recited in claim 45, wherein the step of controlling partial operation of said light source includes independently controlling the effective intensity of said light source. 47. A method of receiving user input with respect to an image as claimed in claim 45 or 46, wherein said step of controlling part of said light source comprises rewiring said part and driving it using a bridge driver. 48. A method of receiving user input with respect to an image as claimed in claim 45 or 46, wherein the step of controlling part of the light source comprises using a diagonal bridge driver. 49. A method of receiving user input with respect to an image as claimed in claim 45 or 46, wherein the step of controlling portions of the light source comprises using a shift register for each portion being controlled. 50. A method of receiving user input with respect to an image as claimed in any one of claims 45 to 49, wherein the step of obtaining and processing an image comprises controlling portions of said light source and said each camera, and said processing said The step of outputting includes processing the information on whether said portion is illuminated or not. 51. A method of receiving user input with respect to an image as claimed in claim 50, wherein when the image is obtained certain parts are illuminated and others are not illuminated. 52. A method of receiving user input regarding an image comprising the steps of: provide a screen on or through which the user can touch and view the image; providing a light source at one or more edges of the screen, the light source directing light across the surface of the screen; providing at least two cameras located at the periphery of the screen with outputs arranged such that they do not receive direct light from the light source, each of the cameras imaging the space in front of the screen, the output Include scanned images; means for processing said output to detect the level of reflected light; and processing the processed output of the camera, using triangulation techniques in conjunction with the processed output, to determine whether the processed output indicates the presence of an object proximate to the screen, and if so, the object s position. 53. A method of receiving user input with respect to an image as claimed in claim 52, wherein said processed output represents a relative orientation of an assumed object location with respect to said camera. 54. A method of receiving user input with respect to an image as claimed in claim 52 or claim 53, wherein the processed output represents the relative orientation of an assumed object position with respect to the lens center of the camera. 55. A method of receiving user input with respect to an image as claimed in any one of claims 52 to 54, wherein said processor determines the position of said object in planar screen coordinates. 56. A method of receiving user input with respect to an image as claimed in any one of claims 52 to 55 comprising: means for modulating said light emitted by said light source to provide a frequency band within the imageable range of said camera; Means for excluding image data outside said frequency band. 57. A method of receiving user input regarding an image as claimed in claim 56, wherein said means for processing said output comprises said means for excluding image data outside said frequency band, and said excluding said Means for out-of-band image data include filtering. 58. The method of receiving user input regarding an image of claim 57, wherein filtering comprises applying a filter selected from the group consisting of: comb filter; high pass filter; notch filter; and bandpass filter. 59. A method of receiving user input with respect to an image as claimed in any one of claims 52 to 58 comprising: means for controlling said light source; and means for obtaining and processing images, wherein the images are obtained in a non-illuminated ambient light state as well as in an illuminated state; wherein said means for processing said output subtracts the ambient state from the lighting state prior to detecting the light level. 60. The method of receiving user input regarding an image according to any one of claims 52 to 59, wherein the light source is an LED, and the touch display includes means for controlling part of the operation of the light source, wherein the controlled This part is independent of other parts of the light source. 61. The method of receiving user input regarding an image of claim 60, wherein the means for controlling partial operation of the light source comprises means for independently controlling the effective intensity of the light source. 62. A method of receiving user input with respect to an image as claimed in claim 60 or claim 61 wherein the means for controlling part of the light source comprises reverse wiring the part and driving it using a bridge driver. 63. A method of receiving user input with respect to an image as claimed in claim 60 or claim 61 wherein the means for controlling part of the light source comprises the use of a diagonal bridge driver. 64. A method of receiving user input with respect to an image as claimed in claim 60 or claim 61 wherein said means for controlling portions of the light source comprises using a shift register for each portion being controlled. 65. A method of receiving user input with respect to an image as claimed in any one of claims 60 to 64, wherein said means for obtaining and processing an image comprises part of controlling said light source and said each camera, and said processing Said means of outputting comprises processing information as to whether said part is illuminated or not illuminated. 66. A method of receiving user input regarding an image as claimed in claim 60, wherein when the image is acquired certain parts are illuminated and others are not illuminated. 67. A method of receiving user input with respect to an image as claimed in any one of claims 52 to 65, wherein said screen is reflective, said camera further images said screen and said means for processing output is obtained from The level at which this light is detected in the mirror. 68. The method of receiving user input regarding an image of claim 67, wherein the processed output represents an assumed relative orientation of an object with respect to the camera and a distance between the object and the screen. 69. A method of receiving user input about an image: providing at least one light source at or near the periphery of the image, the light source directing light across the image; detecting the light level at at least two locations on or near the periphery of said image and providing said level as an output; The output is processed using triangulation techniques to determine whether the output indicates the presence of an object proximate to the image, and if so, the location of the object. 70. The method of receiving user input with respect to an image of claim 69, wherein the positions are substantially non-relative such that when an object is present, the output represents substantially reflected light from the object. 71. A user input device for locating an object with respect to an image comprising: at least one light source located at or near the periphery of the image, the light source directing light across the image; at a detector having an output at or near said image to image the space in front of said screen, said output being indicative of a light level; A processor that receives the output and uses triangulation techniques, and the output determines the presence of the object and, if present, the location of the object.

Images