WO2004029865A1

WO2004029865A1 - Capturing a text string

Info

Publication number: WO2004029865A1
Application number: PCT/IB2003/003932
Authority: WO
Inventors: Duncan F. Mcpherson
Original assignee: Koninklijke Philips Electronics NV
Current assignee: Koninklijke Philips NV
Priority date: 2002-09-25
Filing date: 2003-09-10
Publication date: 2004-04-08
Anticipated expiration: 2005-03-25
Also published as: GB0222209D0; AU2003256027A1

Abstract

A mobile telephone or PDA 10 is provided with a camera 12, and with image processing means 13 for processing image data provided by the camera. The image processor means is arranged to search for the text string www, and to use any occurrence found to identify a uniform resource locator (URL) which includes that text string. The URL is tested to determine whether or not it is valid, and may be stored and/or sent by e-mail or SMS to a remote location.

Description

DESCRIPTION

CAPTURING A TEXT STRING

This invention relates to a device for and to a method of capturing a text string. The invention has application in, but is not limited to, capturing a text string using a mobile telecommunications device.

It is known to provide mobile telephone handsets with integrated digital cameras. It is known also to provide mobile telephone handsets with ports for allowing connection of a digital camera, usually designed especially for the particular design of handset. It is expected that the numbers of telephone handsets provided with integrated digital cameras will increase as time progresses.

In accordance with the first aspect of the present invention, there is provided a device for capturing a text string, the device including a receiver for receiving data representing an image, an image processor arranged to process the image data such as to detect an occurrence of a predetermined text string, and to use the detected occurrence to identify a second text string which includes and is longer than the predetermined text string, and a provider for providing non-image data representing the second text string.

The provider may be software, such as optical character recognition software, which determines the second text string and gives an output, to a display or to some other software for example, in a format suitable for use thereby. The output may be in ASCII or in any other suitable format. Preferably, the device is a mobile telecommunications device.

The device may comprise a determiner for determining if the second text string represents a valid network address, or a valid internet address. The determiner may take any suitable form. It may be a software module which causes a document containing a list of valid addresses to be accessed, using a web browser for example, and which initiates a search for a match. Preferably, though, the determiner is a module which causes a network connection to be made and addresses the address represented by the text string. If the address is valid, this can be detected by detection of the supply of data from the address which would be supplied only if the address were a valid one. In accordance with the second aspect of the invention, there is provided a method of capturing text string, the method comprising receiving data representing an image processing the image data such as to detect an occurrence of a predetermined text string, using the detected occurrence to identify a second text string which includes and is longer than the predetermined text string, and providing non-image data representing the second text string.

The invention can allow the capture of a text string from data representing an image without requiring user intervention, or only a limited amount of intervention. The invention removes the possibility of transcription error, which can so easily occur when entering text strings into devices, especially mobile telephones. Where the text string comprises alphabetic characters in particular, the invention can allow quick and simple data entry, avoiding the process of manual data entry, which can often be lengthy and tedious, especially when using mobile telephones and other devices which do not have full alphanumeric keyboards.

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings in which: Figure 1 is a schematic diagram of a mobile telephone handset according to one aspect of the invention;

Figure 2 is a flow chart illustrating operation of the mobile telephone handset of Figure 1 ;

Figure 3 is a schematic diagram of one alternative form of mobile telephone handset according to one aspect of the invention; and

Figure 4 is a schematic diagram illustrating a further embodiment of a mobile telephone handset according to one aspect of the invention. Referring firstly to Figure 1, a mobile telephone handset 10 is shown comprising generally a CPU 11, an integrated digital camera 12, an image processor 13, a keypad 14 and a display 15. Generally speaking, the CPU 11 is operable in response to an input command received via the keypad 14 or via some other input device (not shown) to control the camera 12 to capture an image. Data representing this image is passed to the image processor 13, which is operated under control of the CPU 11. Operation of the mobile telephone handset 10 will now be described in more detail with reference to Figure 2.

Referring to Figure 2, operation begins at step 20, after which it is determined at step 21 whether the scanning of a text string is required. Step 21 is typically performed by the CPU 11 by determining whether or not an input by a user requires the scanning of some text from a printed image in front of the camera 12. Step 21 is repeated until a positive determination is made, whereafter progression is made to step 22, where the CPU 11 instigates operation of the camera 12, and controls it to provide image data to the image processor 13 at step 23. At step 24, the image processor 13 scans the image data for occurrences of the letter W. This may be performed in any convenient manner, and preferably takes account of variations in the font, colour and size of letters in the image data. At step 25, a decision is made as to whether or not the letter W has been found. If the letter W has not been found, the operation returns to step 24. Only when the letter W has been found in the image data does the operation proceed to step 26, at which step it is determined whether or not the W forms part of a string of WWW. If a negative determination is given, the operation returns to step 24; otherwise, it proceeds to step 27, at which orientation of the image is made in line with the WWW text string. The ends of the entire text string including the WWW are then identified at step 28, where the entire text string is isolated. This step may involve detecting a border around the text string, or detecting spaces (i.e. no text) at the ends of the text string. At step 29, optical character recognition (OCR) is performed on the text string identified at step 28, and this is then provided at step 30 in ASCII format to the CPU 11.

At step 31 , the CPU 11 determines, from examination of internal flags (not shown) whether or not the user requires display of the text string for approval. If at step 31 a determination is made that no display is required, the operation proceeds to step 34. If a positive determination is made, the text string is displayed on the display 15 at step 32. A user may then approve or disapprove the text string displayed, with disapproval resulting in a return to step 24. If the user approves the text string, the CPU 11 determines at step 34 whether or not a user requires the text string to be a valid network address. If a negative determination is made at step 34, the operation proceeds to step 37. If a positive determination is made, the CPU 11 at step 35 determines whether or not the text string relates to a valid uniform resource locator (URL), in a known manner. This step preferably comprises attempting to access the webpage identified by the URL, and identifying from the returned data whether a webpage exists. If the URL is determined to be valid, the operation moves on to step 37; otherwise the operation returns to step 24. At step 37, the CPU 11 determines from investigation of internal flags (not shown) whether or not the user requires storage of the valid URL. If a positive determination is made, the operation proceeds to step 38, where the URL is stored within any convenient storage device of the mobile telephone handset 10. After storage, or if a negative determination is made at step 37, the operation proceeds to step 39. Here, the CPU 11 determines whether or not a user requires the transmission of the URL to a remote location, such as for example a personal computer belonging to the user. If a positive determination is made, the URL is transmitted at step 40 in any convenient manner, such as for example by e- mail or by SMS. After step 40, or if a negative determination is made at step 39, the operation proceeds to end at step 41.

Instructions for implementing the steps of the operation of the Figure 2 flowchart are included in software associated with the CPU 11.

Instructions are also included for scanning the image data in a logical way such that step 24 is not performed in respect of the same area and in respect of the letter W having the same size as has occurred for a previous scan of that area. Furthermore, the step 24 may include determining if the entire image has been scanned, in which case it is assumed that there is no readable URL, and the operation proceeds to step 41. As a result of controlling the image processor 13 to search the image data only for occurrences of the letter W, and from that proceeding to search for WWW and then the whole text string of the URL, the mobile telephone handset 10 need not be provided with processing resources of the order required for text recognition in conventional personal computer applications. This is advantageous since it can result in significant savings in battery power, and/or in the time taken to recognise a URL text string from an image captured by the camera 12 and/or a reduction in complexity.

An alternative mobile telephone handset 50 will now be described with reference to Figure 3. Referring to Figure 3, the handset 50 includes generally a camera module 51 and a mobile telephone module 52. The camera module 51 includes camera optics 53 and a processor 54, which are connected to each other. The telephone module 52 includes some image processing hardware 55, a keypad 56 and a display 57, all of which are operated under control of a CPU 58. The image processing hardware 55 is connected to the processor 54 by a data bus 59.

Operation is similar to the operation of the mobile telephone handset of Figure 1. However, the processor 54 of the camera module 51 is arranged to modify the brightness and/or contrast characteristics of the image represented by the data provided by the camera optics 53 in such a way as to make the image data more susceptible to processing by the image processing hardware 55. The processed data is transferred to the image processing hardware 55 from the processor 54 via the data bus 59. The image processing hardware 55 operates under control of the CPU 58 to perform the steps 24 to 41 of the flowchart of Figure 2. The embodiment of Figure 3 may confer advantages over the embodiment of Figure 1 in that it requires less processing resources than the combination of the image processor 13 and the CPU 11 of Figure 1. In an alternative embodiment (not shown), a camera module like the module 51 of Figure 3 is arranged to perform all of the processing required, and supplies a URL (text string) to the CPU of a telephone module without requiring any substantial processing by the telephone module. In a further alternative embodiment (not shown), a camera module like the module 51 of Figure 3 is arranged to perform the steps 24 to 27 of the Figure 2 flow chart. In this case, the camera module provides to a telephone module information identifying the location, size and orientation of the text string WWW. Reference will now be made to Figure 4, which shows an alternative embodiment for a mobile telephone handset 60 according to the invention. Referring to Figure 4, the mobile telephone handset 60 is shown including a port 61 , to which an external digital camera device 62 may be connected. Connection of the camera 62 to the mobile telephone handset 60 allows the transfer of image data from the camera for processing within the mobile telephone handset, and also allows a CPU (not shown) of the mobile telephone handset to control the camera 62. Operation of the apparatus of Figure 4 is substantially the same as that described above with reference to Figures 1 , 2 and 3. In alternative embodiments (not shown), the text string which is searched for by an image processor is not WWW as in the above-described embodiments. The text string may be one which commonly occurs as a prefix in a URL, such as 'wap' or 'http://', or it may be one which commonly appears as a suffix, e.g. '.com', '.net', or '.org' etc. Where a URL does not necessarily contain the string that is searched for, the image processor preferably searches for each of the different possibilities (e.g. different possible suffixes) until an appropriate string is found. Preferably, the mobile telephone handset 10 is provided with a menu system, allowing functions to be selected by a user from a menu-based option list, which type of interface is now very familiar to mobile telephone users. A menu page is included which allows a user to select a text string from a list of displayed text strings. The list may contain those suffixes and prefixes listed above. Preferably, the menu page also contains an option which allows a user to enter a text string, which is subsequently searched for in the manner described above. Any text string manually entered by a user is then made available in the menu list for selection subsequently. In a further embodiment, the mobile telephone handset 10 is arranged to allow the point-and-click capture of telephone numbers. In this embodiment, the predetermined text string is the whole or a part of STD code, which for London, UK is currently '020'. What predetermined text string the handset 10 is arranged to search for depends on the allocation of telephone numbers in the area or country in which the handset is to be used primarily. Using the UK as an example, the majority of numbers used in connection with business are expected to be freephone numbers (having STD code 0800 or 0500), local- rate numbers (having STD code 08457 or 08459 for example), or numbers having the same STD code as the area in which the number is displayed. As such, the mobile telephone handset 10 is arranged, on being controlled to capture a telephone number from an image, to determine the STD code for the area in which the handset is currently located, and to search the image for this code. If no occurrence of the STD code is found, the handset 10 is then controlled to search for the freephone and the local-rate codes. If there is still no success, the handset 10 is controlled to search for STD code prefixes, such as '01' and '02'. Obviously, the predetermined text string and the order for searching the possible strings or prefixes may depend on the country in which the handset 10 is sold or used. A list of STD codes, or similar, may appear in a menu page of the handset 10, allowing a user to select the appropriate number string, after which the handset will identify the number in the image which begins with or otherwise includes that number string. When a telephone number has been captured from an image, it is made available for immediate dialling, included into a phone book in the handset 10, or alternatively saved into a special location for subsequent retrieval by a user. To allow more easy capture of text strings, particularly URLs, from printed materials, it may be desired to use a font which is read easily by OCR software. Furthermore, a text string could be black on a standard colour, e.g. white, background, and may include a line border in the form of a rectangular box. Where a box can be expected by the image processor, the step of detecting a box is preferably a precursor and provides assistance to the step of detecting the predetermined text string. The term 'text string' will be understood to mean a string of alphabetic characters and/or numerals, which string may or may not include marks of punctuation, e.g. '.', '&' or '@'. A text string will not normally include one or more spaces, but this is not precluded by the embodiments. Preferably the text string includes at least one alphabetic character. From reading the present disclosure, other variations and modifications will be apparent to persons skilled in the art. Such variations and modifications may involve equivalent and other features which are already known in the design and manufacture of electronic devices and in their use. Although Claims have been formulated in this Application to particular combinations of features, it should be understood that the scope of the disclosure of the present invention also includes any novel features or any novel combination of features disclosed herein either explicitly or implicitly or any generalisation thereof, whether or not it relates to the same invention as presently claimed in any Claim and whether or not it mitigates any or all of the same technical problems as does the present invention. The Applicants hereby give notice that new Claims may be formulated to such features and/or combinations of such features during the prosecution of the present Application or of any further Application derived therefrom.

Claims

1. A device (10) for capturing a text string, the device including: a receiver (13) for receiving data representing an image; an image processor (13) arranged to process the image data such as to detect an occurrence of a predetermined text string, and to use the detected occurrence to identify a second text string which includes and is longer than the predetermined text string; and a provider (11) for providing non-image data representing the second text string.

2. A device as claimed in claim 1 , further comprising a determiner for determining if the second text string represents a valid network address.

3. A device as claimed in claim 1 , further comprising a determiner for determining if the second text string represents a valid internet address.

4. A device as claimed in any preceding claim, which includes an integral digital camera (12).

5. A device as claimed in claim 4, in which the camera (12) includes a processor (54) for changing attributes of the image data.

6. A device as claimed in claim 5, in which the image processor includes dedicated image analysis hardware.

7. A device as claimed in any of claims 4 to 6, in which at least part of the image processor (54) forms part of the camera.

8. A device as claimed in any of claims 1 to 3, including a port (61) for connecting to an external digital camera.

9. A device as claimed in any preceding claim constituted as a mobile communications terminal.

10. A method of capturing text string, the method comprising: receiving data representing an image: processing the image data such as to detect an occurrence of a predetermined text string; using the detected occurrence to identify a second text string which includes and is longer than the predetermined text string; and providing non-image data representing the second text string.

11. A method as claimed in claim 10, further comprising determining if the second text string represents a valid network address.

12. A method as claimed in claim 10, further comprising determining if the second text string represents a valid internet address,