GB2440583A

GB2440583A - A portable route planning and object identification device for the visually impaired

Info

Publication number: GB2440583A
Application number: GB0615559A
Authority: GB
Inventors: Tom Pey
Original assignee: Guide Dogs for Blind Association
Current assignee: Guide Dogs for Blind Association
Priority date: 2006-08-04
Filing date: 2006-08-04
Publication date: 2008-02-06
Also published as: WO2008015375A1; GB0615559D0

Abstract

A voice-activated portable device to assist visually impaired people comprises scanning means for scanning an object selected by a user and a memory for storing information regarding a scanned object. A sensor and identification means locate an object previously scanned. The scanning means may be able to read electronic tags attached to objects and the portable device may store a voice signal together with the identity of the scanned object. Additionally, means for planning a route to a destination selected by the user are provided, which includes identifying the correct transport for travel to the selected destination. The sensor identification and journey means are able to communicate with a wireless system through a communicating means. A monitoring means may also be provided for monitoring and storing medical information related to the user.

Description

ASSISTANCE DEVICE FOR BLIND AND PARTIALLY SIGHTED

PEOPLE

The present invention relates to a voice activated device suitable for assisting blind and partially sighted people.

In physical space, visually impaired users still depend mainly on basic low technology aids such as canes, guide dogs or spoken directions to reach only known destinations along familiar learned routes. Tactile maps exist, but are hard to use and, as a result are often ineffective, and so are not widely used. In general, low technology aids are not intuitive enough for travel to new or unknown destinations and access to unfamiliar social environments such as a hospital or shopping mall is restricted.

Visually impaired people have great difficulties participating in necessary daily activities such as shopping, going to hospital, a bank or just visiting a friend.

More recently, there have been attempts to use technology to detect nearby objects. For example, some high-tech devices detect potential hazards at head level. Several more advanced navigation systems such as GPS-Talk utilise Global Positioning System (GPS) and offer potential to develop outdoor way-finding solutions. However, to realise this potential, devices must be integrated with real time sensory information about users' dynamic environment. Such a device is the front-end of a complex technologies' integration, and must be affordable, wearable and intuitive.

In virtual space, access to the Web is very difficult -often impossible.

Sensor based technology applications; such as smart homes and telecare offer users some limited alternatives. However, in order to fully address the needs of blind and partially sighted people, Internet mediated services have to be dedicated to users' needs and living environments.

The global incidence of blindness has increased in the last twelve years but the incidence of blindness in the population over 50 years of age has increased even more. Therefore, there is the need, not only to prevent blindness and to find effective strategies to combat its progression, but also there is an urgent requirement to find appropriate, effective rehabilitative and assistance strategies in order to promote independent functioning in the growing visually impaired population.

In the United Kingdom, services to people with significant sight loss are well established and comprise, in the main, mobility and rehabilitation.

However, these services, which tend to be fragmented and/or geographically dependent, are limited in their potential and are not intuitive in function.

People with sight loss experience significant difficulties in undertaking every day tasks and achieving daily goals. Various aids, ranging from tactile maps to touch pad orientation systems exist. However the inherent limitations of such systems prevent their widespread use by people with sight loss.

Considerable advances in both science and technology have, however, resulted in a variety of low and high tech mobility devices ranging from sonic canes, guides and pathfinders to sophisticated sensory substitutions such as echo-locators, and to more complex navigation, orientation and location aware systems.

For example, there are products from Sonar Canes, Sonic Guide and Sonic Pathfinders, which detect obstacles and hazards. VOICe, on the other hand translates video images from a regular PC camera into sounds so that users can "see with their ears". GPS-talks, together with MoBIC and Visuaide Trekers are navigation systems that utilise GPS (Global Positioning Systems) and wireless communication networks for outdoor way-finding. Orientation and navigation are achieved by the use of sophisticated systems such as Talking Sign technology; the system consists of short audio signals sent by invisible infrared light beams from permanently installed transmitters, which are placed at sign locations or on key environmental features, to a hand-held receiver that decodes the signal and delivers the voice message through its speaker or headset.

Other systems such as indoor location systems are all encompassing and are, in the main, sensor-based technologies providing location information, space identifiers, and position coordinates. Cricket, for example, piggybacks onto applications running on handheld computers, laptops, and sensors and is designed for use indoors and in urban areas where outdoor systems such as Global Positioning Systems (GPS) cannot be used.

Other relevant technologies to the present invention are RFID (Radio Frequency Identification) tags. The usage of RFID tags is increasing strongly and they are already being used for the following applications: * cards for transport * access controllcorporate ID * libraries/archives * raw material inventory in industry There are also some original applications such as RFID tags embedded in plates at sushi bars to calculate the customer's bill. New applications are expected in baggage delivery and in inventory management, as well as medical supervision. RFID technology is also being studied to develop a system to help visually impaired people navigate.

Card shaped products for personal usage, such as telephone cards or commuters passes, have traditionally accounted for the majority of RFID usage, but in future growth is especially expected in retail and logistics related applications. The international standardization for RFID at high-frequency band (ISO/LEC 14443 proximity application, ISO/IEC 15693 vicinity application) is expected to facilitate wider usage of RFID.

It is evident that there is an abundance of assistive technologies, however they exist in isolation and are, as a consequence, of limited use to the visually impaired person. Echo locators for example, offer users little or no information about an object, only its location. Access to transport and to unfamiliar destinations remains problematic, time consuming and challenging and individual preferences and personal objectives when travelling are frequently only realised with difficulty or, indeed, not at all.

As mentioned above, there are a number of relevant technologies that provide information for blind and partially sighted people. They are generally however, one service providers and thus are severely limited in use. A device, which suits a person who is partially sighted, can be completely useless for a totally blind person.

There is thus an urgent requirement to address the diverse and changing needs of the blind and partially sighted population by integrating existing technologies such as sensor and data networks, mobile and wearable smart devices and remote healthcare monitoring into one simple device i.e. a multi-modal integrated technology aid. Such technology is not available at the present time.

SUMMARY OF THE INVENTION

The objective of the present invention is to solve the above problems and to pull together diverse technologies, which would otherwise evolve independently.

In general terms the invention provides a device comprising a laser scanning technology, mobile service facility, voice recognition facilities, sensory identification, memory, data acquisition and storage.

A first specific object of the present invention is to use the device in "smart homes" by virtual travel communities, and emergency response agencies. Moreover, the device could be used in shopping and supermarket for automated check-out.

To attain the above objective in the invention's primary mode, it is configured as a voice-activated, portable device to assist visually impaired people comprising: communicating means for transmitting and receiving data to and/or from a network; scanning means for scanning an object selected by a user; memory means for storing information regarding a scanned object; sensor and identification means for locating an object previously scanned; journey means for planning a route to a destination selected by the user and for identifying the correct transport for travel to a selected destination; wherein the sensor identification and journey means are able to communicate with the network through the communicating means.

In its secondary mode, the invention is configured with a wireless system for assisting visually impaired people comprising: a number of portable devices according to the first aspect of the invention and connected to a sensor network; receiving means adapted to receive data from the portable devices and to transmit said data to a server which is adapted to store the data; transmitting means adapted to transmit data from the server to the portable devices; selecting means adapted to select the data transmitted by the transmitting means according to each portable device; wherein each portable device is adapted to receive and store the selected data, and to update its existing data.

In addressing a third aspect of its use, the present invention is configured as a method for assisting visually impaired people using a wireless system comprising a number of portable devices according to its primary mode and connected to a sensor network comprising: receiving data from the portable devices and transmitting said data to a server for storage; transmitting data from the server to the portable devices; selecting the data to be transmitted by the transmitting means according to each portable device. In this way, each portable device receives and stores the selected data, and updates its own existing data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a device according to the invention; and FIG. 2 is a block diagram of functionality according to an embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Below, preferred configurations of the present invention are explained by way of example with reference to the accompanying drawings.

Figure 1 is a schematic view showing the first configuration of the present invention. Figure 2 is a table setting out a range of desired functionality for the device and uses and advantages of the different functionalities.

The device of the present invention comprises laser scanning means, voice recognition means, sensory identification means and storage means.

The device is portable, suitable to be hand or pocket held, wireless and lightweight, weighing less than 250 grams. It can be upgraded on demand, in a similar way to the upgrading of a computer and can be configured for either home base services, i.e. indoor use only, using a form of "wall-mounted control board" or for indoor and outdoor use. If the latter, subscription to the following TalkingGadget service will usually be preferred.

The device is able to provide and support three main functionalities: talkingobj ect, talkingtravel and talkingcare.

The selection of the mode of operation of the device is by push buttons generally indicated in Figure 1. Textured or tactile buttons of various shapes will select specific functions and can be easily identified by touch by a visually impaired user.

For example, a square button can be provided to select the talkingobject function, a round button to select the talkingtravel function and a triangular button to select the ta1kingcare function.

The user is provided with a supply of thin electronic tags that the device can read using the scanning facility. Each tag is attached to an object selected by the user if the object is not already provided with a tag or tag equivalent, which can be read by the scanning facility already inbuilt.

The user can select the talking(äjobject mode using the square button.

Then, the user can uses the device in the talkingobject mode, also referred to as "objectFinder" mode, to scan the object once and the user assigns a preferred name for the object via the device, which is recorded and associated with the tag identity.

Subsequently, when the user wants to know the location of the tagged object, he or she speaks the assigned name so the device can recognize it using the scanning means and tell the user where the object is.

Similarly, if a user wishes to identify an object, the device in the ta1kingobject mode can scan the object and use the scanned tag identity to inform the user what the object is.

Further information about the tagged object can also be stored in the device or an associated storage device able to communicate with the device, such as a home computer so that, on scanning the tag associated with the object, the device will inform the user of this further information.

For example, in addition to what the object is, its colour or other attributes can be described.

This further information allows the device in the ta1kingobject or objectFinder mode to be used as a "mirror". The device reads the tagged clothes and gives a description of the clothes, e.g. colour to the user.

The talkingtravel or "travelFinder" mode can be selected using the round button. In this mode, before travelling, the destination is fed into the device, preferably using a voice recognition system. The device then either plans the route or updates previous information and tells the individual how to get to the desired destination and how long it will take.

The device will identify the correct bus, for example, and announce the stop at which the user has to alight.

The talkingcare or "careFinder" mode is selected using the triangular button. In this mode, the device, when attached to a specific add-on medical sensor, will wirelessly collect and transport data, such as blood pressure, weight, and blood sugar levels to a home computer to be analysed. Alternatively, the device could either transmit the data to a doctor's surgery immediately or stores it temporarily for later transmission or periodic retrieval.

In Fig 2, the benefits to the user achieved by the careFinder are listed in

a table.

The laser scanning means is based on a principle similar to bar code recognition and thus will be multi-purpose, allowing for identification of tagged objects of all kinds. Foodstuffs, for example, will usually be bar coded, at source, i.e. by the manufacturer. The bar code would ideally identify the product, its cost and contents. It may also be possible to extend the bar code so that cooking instructions and recipes can also be included on the packet, tin or bag. At the present time, some clothing manufacturers include "Braille information" on clothing tags. However, there are relatively few who provide this facility and indeed relatively few visually impaired people who use Braille as a communication method. Consequently, bar coding clothing to include cost, size, colour, material and washing instructions would be, without doubt, hugely helpful to the visually impaired person.

Sensor identification means will allow the visually impaired person to locate specific objects. Each object when bought may have an identification bar code or alternatively could be "tagged" ideally, by the manufacturer andlor by the visually impaired person. The object information, when scanned, would then be stored either in the device itself or sent to a home-based computer and/or a central sever. The latter could be provided by an Internet agency, phone company or an independent provider. The device would behave like a personal server and data distributor (data hub). Thus data storage and management would be possible both locally and remotely. If stored locally, i.e. in the device or on a home computer, vital information could still also be sent to the central server as a back-up facility.

Another aspect of the present invention relates to a system for public transport comprising a service provider, a computer and detecting means.

Bus and train companies could subscribe to a service provider and each mode of transport has inbuilt sensors capable of communicating with a computer into which is programmed route information, timetables and travel times, in much the same way as GPS and navigation systems currently operate. Even if bus numbers were altered mid route, this would be recognised by the computer and the visually impaired person would be informed accordingly. The system would also be capable of identifying the correct bus and where it was in relation to the individual and/or bus stop.

Another functional aspect of the invention is provided by a med-alert, which is an added health and safety feature and can be tailored to each user. The user monitors various aspects of body function by attaching body sensors. The information is picked up by the device and then sent to a remote server, which is then accessed by the medical practice responsible for the individual's care. The practice would require, however, subscribing to such a service. Thus significant changes in a visually impaired person's health status would be recorded. Immediate action by the health agencies involved would then be possible.

Preferably, the device will have a face recognition and fingerprint security facility requiring biometric verification of user identity to enable use of the device.

A further extension of the invention is to use the device to support the TalkingGadget service which tailors the device to the specific needs of each user, for example, personalising voice commands, unique privacy protection, and the creation of home based services as alternative mobility solutions are all possible.

The service uses the voice commanded device to enable users to identify, find and manage tagged objects and object information indoors and/or in and out of doors. The device, as previously indicated, can be designed to access a wireless sensor network which will then send the information to either a home based server or a central (remote) service centre. The device will act like a personal service gateway once services are created, subscribed and activated.

The service may have particular appeal and could ultimately be extended to other ad hoc mobile services, and provide a platform on which various services could be developed.

The embodiments described above are exemplary and not exhaustive, the skilled user will be able to envisage other alternatives within the scope of the invention, as set out in the claims.

Claims

WHAT IS CLAIMED IS: A voice-activated portable device to assist

visually impaired people comprising: communicating means for transmitting and receiving data to and/or from a network; scanning means for scanning an object selected by a user; memory means for storing information regarding a scanned object; sensor and identification means for locating an object previously scanned; journey means for planning a route to a destination selected by the user and for identifying the correct transport for travel to a selected destination, wherein the sensor identification and journey means are able to communicate with the network through the communicating means.

2. The voice-activated portable device according to claim 1, further comprising monitoring means for monitoring medical information related to the user and storing said information in the memory means.

3. The voice activated portable device according to claim 1 or claim 2, further comprising textured and shaped tactile control buttons.

4. The voice activated portable device according to any preceding claim, further comprising biometric means adapted to identify the user and enable operation of the device.

5. The voice activated portable device according to claim 4, in which the biometric means is adapted to identify the user's face or fingerprints.

6. The voice activated portable device according to any preceding claim, wherein the communication to the network is wireless and the device is adapted to be used both indoors and outdoors.

7. The voice activated portable device according to any preceding claim, in which the scanning means is adapted to read electronic tags attached to an object selected by the user when scanning the object.

8. The voice activated portable device according to claim 7 and adapted to store a voice signal together with the identity of said object after scanning the object and to subsequently recognise said stored voice signal.

9. The voice activated portable device according to claim 8, adapted to respond to a voice signal recognised as said stored voice signal by informing the user of the location of said object by using the sensor and identification means.

10. The voice activated portable device according to any preceding claim and adapted to be upgradeable on demand by the user.

11. The voice activated portable device according to any preceding claim and having a weight of less than 250 grams.

12. A wireless system for assisting visually impaired people comprising: a plurality of portable devices as defined in any preceding claim connected to a sensor network; receiving means adapted to receive data from the portable devices and to transmit said data to a server which is adapted to store the data; transmitting means adapted to transmit data from the server to the portable devices; selecting means adapted to select the data transmitted by the transmitting means according to each portable device; wherein each portable device is adapted to receive and store the selected data, and to update its existing data.

13. A wireless system according to claim 12, wherein the transmitting means is adapted to transmit medical data to a service centre to be analysed.

14. A method for assisting visually impaired people using a wireless system comprising a plurality of portable devices as defined in any one of claims 1 to 13 connected to a sensor network; and comprising the steps of: receiving data from the portable devices and transmitting said data to a server which stores the data; transmitting data from the server to the portable devices; selecting the data to be transmitted by the transmitting means according to each portable device; whereby each portable device receives and stores the selected data, and updates its existing data.

15. A device substantially as shown in or as described with reference to Figure 1 of the accompanying drawings.