US20160275904A1

US20160275904A1 - Tiled map display on a wireless device

Info

Publication number: US20160275904A1
Application number: US15/167,574
Authority: US
Inventors: Bob Barcklay; Ritesh Bansal
Original assignee: TeleCommunication Systems Inc
Current assignee: TeleCommunication Systems Inc
Priority date: 2005-07-14
Filing date: 2016-05-27
Publication date: 2016-09-22
Also published as: US20150227555A1; WO2007011616A3; US20070013708A1; EP1907953A2; WO2007011616A2; US9041744B2; US9367566B2; EP1907953A4

Abstract

A tiled-map display control with a predictive caching technique that minimizes user wait time and provides at least the illusion of continuous panning, even while map tile images are being loaded. Important components of the tiled 5 map display are its definition and cached use of map tiles, as well as the way that the map tiles are put together on a small screen. Easy, seamless, wait-free and convenient viewing of a map for a user of a wireless device provides information, e.g., mapped traffic conditions. The disclosed embodiments are techniques that have been reduced to practice in both a BREW platform, and then in a J2ME platform, and deployed for operation in major carrier wireless networks. The invention has particular applicability for use in wireless devices with typically smaller display screens requiring the need for panning, and limited bandwidth capabilities of the supporting wireless network.

Description

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No. 11/251,766, filed 18 Oct. 2005; which claims priority of U.S. Provisional Application No. 60/698,922, filed 14 Jul. 2005, the entirety of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention
This invention relates generally to wireless telecommunication. More particularly, it relates to display techniques in wireless applications that enable viewing by a user of a wireless device.
2. Background of the Related Art
Year by year, wireless devices such as cell phones have grown smaller and lighter. Along with this trend of miniaturization is the implementation of a display screen that is correspondingly small. The size of the display screen, and amount of memory within the wireless device, are often limiting factors in many applications. This is particularly true for applications running in a wireless device that attempt to display a map for a user.
When a map is displayed in a wireless device using a conventional application, typically a map somewhat larger than the size of the display screen is downloaded and displayed centered on the display. However, existing map display applications are a bit cumbersome to use on a wireless device, as they typically do not allow the user to fluidly pan around a map that is being displayed at the time.
For instance, FIG. 6 shows a conventional wireless device (e.g., a cell phone) 600 having an LED display 610 and an image 660 loaded for display thereon. Existing applications load a display image 660 that is at best just a little larger in memory than the displayable portion of the display image 650.
In such conventional applications, the loaded image 660 is centered on the display 610, with undisplayed, or cropped portions of the loaded image forming an overlapping border image area, e.g., a 30 to 50 pixel border area just outside the displayable portion of the image 650.
In many displaying applications, particularly those that display a map, it is desirable to allow panning by the user. Most often, direction or arrow keys on a keypad of the wireless device are operated to cause the relevant application to move the displayed image in a desired direction. When panning, the user can move the displayed image only up until the point at which the edge of the loaded image meets the relevant edge of the display 610. At this point, the panning operation pauses, and a new image is requested by the wireless device—all while the user waits for the download of a new image centered at the current point of the curser.
Conventional panning operations on wireless devices provide a pop up such as a progress bar that visually depicts the state of the downloading operation for the new image centered at the current location. At this point, until the new image is fully downloaded, conventional applications disallow most user operations while the new image is being downloaded to maximize efforts (and reduce time) in fetching the new image, which severely interrupts the user's experience of panning through the displayed image.
When the wireless device is finally ready to continue (which in a wireless society seconds can seem like forever), the previously loaded image is deleted from memory, and replaced with a new image centered at the current point of the cursor. No attempt is made to reuse the old, deleted image.
Thus, most existing applications load a single map image, somewhat larger in dimensions than the screen, and allow the user to pan to one of the edges. Up until this point the panning operation may be smooth, but when the user reaches the edge of the map the application will appear to freeze. Many times a pop-up with a progress bar and/or an hourglass is usually displayed indicating to the user that the wireless device is working, and the user waits for the next map image to be retrieved across the wireless network. The inventors of this application appreciated that this conventional approach results in a poor user experience with significant wait periods each time a loaded map edge is reached, and a new map display image requested to be viewed is loaded. If the map images are not cached, the user often has to wait again for display of the same map if they pan back to a location that they have already viewed.
There is a need for both more seamless panning as perceived by the user of a wireless device such as a cell phone, as well as to minimize the down time during which the user is not permitted to continue on with their mapping control while the image is being fetched.

SUMMARY OF THE INVENTION

In accordance with the principles of the present invention, a display module in a wireless device comprises an image tile loader module. An image tile cache memory (i.e., “cache manager”) is adapted to contain a plurality of image tiles. At least one of the plurality of image tiles is not in a viewable section of an overall image when downloaded to the wireless device. A predictive selection module running a suitable tile selection process causes the image tile loader module (i.e., “data manager”) to fetch a next image tile before the next image tile is required for display on the wireless device.
In accordance with another aspect of the invention, a method of fetching a panable image for display on a wireless device comprises definining the pan-able image into a plurality of image tiles. A direction of panning is determined by a user viewing a displayed portion of the pan-able image on the wireless device. A next one of the plurality of image tiles is fetched based on the determined direction of panning.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an image loaded in blocks, or tiles, providing a 3×3 matrix of tiles about an image currently being displayed, in accordance with the principles of the present invention.

FIG. 2A shows a loaded image comprised of 3×3, or nine total, tile images, loaded in a predictive order, in accordance with the principles of the present invention.

FIG. 2B shows a loaded image comprised of 3×3, or nine total, tile images, loaded in a different predictive order than as shown in FIG. 2A, in accordance with the principles of the present invention.

FIG. 3 shows a predictive loading of additional tiles as the displayed image is panned to the RIGHT under control of a user operating a keypad of a wireless device, in accordance with the principles of the present invention.

FIG. 4 shows a predictive loading of additional tiles as the displayed image is panned UP under control of a user operating a keypad of a wireless device, in accordance with the principles of the present invention.

FIG. 5 shows a predictive loading of additional tiles as the displayed image is panned UP and to the RIGHT under control of a user operating a keypad of a wireless device, in accordance with the principles of the present invention.

FIG. 6 shows a conventional wireless device (a cell phone) having an LED display and an image loaded for display thereon.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The present invention provides a tiled-map display control with a predictive caching technique that minimizes user wait time and allows for the illusion of continuous panning, even while map tile images are being loaded. Important components of the tiled map display are its definition and cached use of map tiles, as well as the way that the map tiles are put together on a small screen.
The present invention enables easy, seamless, wait-free and convenient viewing of a map or other pan-able image by a user of a wireless device. An image implemented in the embodiments herein is a map showing traffic conditions, though the invention has applicability to maps in general.
The disclosed embodiments are techniques that may be implemented in a suitable software platform, e.g., BREW or J2ME, and deployed for operation in major carrier wireless networks. The present invention has particular applicability for use in wireless devices because of the typically smaller 5 display screens requiring the need for panning, and also because of the smaller bandwidth capabilities of the supporting wireless network. FIG. 1 shows an image loaded in blocks, or tiles, providing a 3×3 matrix of tiles about an image currently being displayed, in accordance with the principles of the present invention.
In particular, as shown in FIG. 1, a plurality of tiles 120-127, 150 are loaded into memory of a wireless device such as a cell phone. Upon initial startup of the relevant application, the central tile 150 is loaded first and displayed. The goal of the application map control (i.e., “display module”) is to load and cache all surrounding tiles 120-127 for ready-display to provide a seamless panning experience for the user.
Presuming the user is not yet panning on the wireless device, the displayed tile 150 is preferably downloaded or fetched first, followed by tiles 120-127 in a predetermined order.
When the user first brings up an image tile on the device, the application fetches the first image tile (Tile #1) 150 with dimensions that correspond exactly to the screen dimensions of the device. Upon loading the first tile 150 and storing it in a cache on the wireless device, the technique immediately begins fetching the surrounding image tiles 120-127 in anticipation of user requests to pan around on the image (e.g., map). The user may pan (go left, right, up, down) at anytime. If the image tile hasn't yet been loaded, the user sees the outline of the image tile and a % loaded indicator. The order in which image tiles are requested over the network is a predictive technique based on the current direction of pan and the currently displayed image tile. The sequence of image tiles to be fetched is re-evaluated after a single image tile has been loaded and stored in the cache. The display component of the application combine's these image tiles into what looks like a continuously pan-able image with no edges.
FIG. 2A shows a loaded image comprised of 3×3, or nine total, tile images, loaded in a predictive order, in accordance with the principles of the present invention.
In particular, as shown in FIG. 2A, a displayed tile 150 is fetched and cached first, followed by tiles 121, 123, 126, 124 adjacent the edges of the displayed tile 150. If at that point the user still hasn't yet panned in any direction, the remaining tiles 120,125,127, 122 off the corners of the displayed tile 150 are then fetched and cached in persistent storage on the device such as image tile cache memory (i.e., “cache manager”).
The ordering of tiles fetched when the application first starts up may be other than that shown in FIG. 2A. For instance, FIG. 2B shows a loaded image comprised of 3×3, or nine total, tile images, loaded in a different predictive order than as shown in FIG. 2A, in accordance with the principles of the present invention. In FIG. 28, the displayed tile 150 is fetched first, followed in order by tiles 123, 121, 124, 126, 125, 120, 122 and 127.
Most wireless devices provide a user interface allowing the user to provide directional control. For instance, in the given application on a cell phone, arrow keys 110-113 provide keyed input of direction instructions in the right, down, left and right directions, respectively.
Importantly, in accordance with the principles of the present invention, predictive prioritization of tile fetching is provided as a result of the user panning in any given direction. Unfetched tiles adjacent to the displayed portion of the tiles in that particular direction are prioritized and fetched before other tiles in directions not being panned to at the time.
Preferably, the closer an unfetched tile is to having any portion required for display, the higher a priority it is given in a fetching order. Moreover, should memory requirements limit the number of tiles capable of being stored, tiles are preferably retained longer in a direction of a current or last panning operation, with tiles farthest away in a direction opposite the direction of panning being first to be deleted or overwritten.
Thus, as the user pans the map, using arrows keys 110-113 on the device, new tiles become visible. The preferred technique for fetching tiles is adaptive in the sense that upon the completion of fetching one tile, the technique determines the currently displayed map tile(s). If this has changed, the fetching technique will re-prioritize the list of tiles to be fetched.
FIG. 3 shows a predictive loading of additional tiles as the displayed image is panned to the RIGHT under control of a user operating a keypad of a wireless device, in accordance with the principles of the present invention.
In particular, as shown in FIG. 3, the user is panning to the right using right arrow keypad key 110. In the given example, the wireless device user starts out viewing tile # 1 150, and pans right to view tile # 2 121. Tile # 2 121 will become the new Tile # 1 and the list of tiles to be fetched are reprioritized based on the direction of panning to the right.
Tiles already fetched preferably remain in the cache until they either expire based on a time to live setting, or must be removed to make room for a tile that is closer to the user's currently displayed location in the latest direction of pan.
Hysteresis may be implemented wherein small pans in different directions may not influence the predictive nature of the tile fetching priority as much as large movements. For instance, a threshold distance may be established beyond which the priorities of tile fetching becomes reprioritized.
FIG. 4 shows a predictive loading of additional tiles as the displayed image is panned UP under control of a user operating a keypad of a wireless device, in accordance with the principles of the present invention.
In particular, as shown in FIG. 4, the user is panning UP using UP arrow keypad key 113. In the given example, the wireless device user starts out viewing tile # 1 150, and pans UP to view tile 123. Tile 123 then becomes the new Tile # 1 and the list of tiles to be fetched are reprioritized based on the direction of panning in the UP direction.
For the sake of exhaustiveness, FIG. 5 shows a predictive loading of additional tiles as the displayed image is panned UP and to the RIGHT under control of a user operating a keypad of a wireless device, in accordance with the principles of the present invention. In particular, as shown in FIG. 5, the user is panning UP and to the RIGHT using keypad keys 110, 113 (or a single keypad key if an 8-direction keypad is provided). In the given example, the wireless device user starts out viewing tile # 1 150, and pans UP and to the RIGHT to view tiles 121, 120 and 123. At the point at which tile 120 becomes the tile with the largest amount of DISPLAYED area, tile 121 becomes the new Tile # 1, and the list of tiles to be fetched will be reprioritized based on the direction of panning UP and to the RIGHT.
Preferably, as the user changes direction of panning, the priority given to the tile order being fetched may be dynamically updated.
In further embodiments, tiles are split into sub-tiles to support wireless devices with less available memory. For instance, the tiles 150, 120-127 shown in FIGS. 1-5 may be each broken into 2×2 images. In such a scenario, a single tile fetch returns four (4) sub-tiles. The reason for this is to limit the amount of display memory that is required. The use of the four sub-tiles minimizes use of display memory and/or random access memory (RAM) on the wireless device. In the disclosed embodiments, this is separate from the persistent storage where map tiles are cached by the image tile loader (i.e., “data manager”).
For example, if the user pans so that the corner of a tile is in the center of the display screen, we would initially have to render 4 large tiles in memory even though only a quarter of each tile is actually visible on screen. With the sub-tile enhancement, we are able to render in memory just what is visible, reducing the memory footprint of the application, allowing operation on a wider variety of wireless devices.
Thus, according to the invention, tiled images such as maps are adaptively and predictively fetched, and stitched together on a wireless device such as a cell phone, to create the illusion of an infinitely large image which is seamless to the user. The invention pre-fetches image tiles before they are required for display so that the user can pan around past image tiles fetched initially. When the user finally stops panning, any image tiles not yet fetched are prioritized and quickly fetched and displayed. Accordingly, the user can pan over the edge of the image tile, and that missing image tile is shown as a blank area.
Ideally a progress bar or percentage is shown for each image tile being fetched. In disclosed embodiments, a ‘throbber’ is shown for each image tile or cell on the image grid that's not yet displayed. The ‘throbber’ is a pulsing ball with a percentage complete next to it. In certain instances, there could be as many as 4 ‘throbbers’ representing 4 different image tiles displayed simultaneously.
Thus, some image tiles will be fetched before others, and the progress will complete quicker for those image tiles than for others. Eventually, any missing or blank areas of the image are filled in.
As technology progresses and bandwidth improves for wireless devices such as cell phones, the image tiles will show up as fast as the user can pan.
In the disclosed embodiments, the complete high resolution image tiles are displayed when fully loaded-the low resolution image isn't shown first as it becomes available as it is not deemed worth expending processing time in displaying the low resolution temporary image. However, temporary use of a low resolution image of an image tile being downloaded is within the scope of the invention.
In disclosed embodiments, a 3×3 image tile grid is maintained in cache memory, centered about the image tile most predominant in the display at the time. As the user pans, if it is a quick pan and bandwidth to the wireless device is slow, in a worst case the user may have for display only one of the 3×3 image tile grid, but when the panning operation stops the processor can catch 30 up and fill in the 3×3 in a prioritized order.
A benefit of the invention is that, on average, image tiles load more quickly by taking advantage of the cache. Since the user is never forced to wait for something to happen, the user experience is much smoother and generally better.
The invention has particular application with wireless application developers that want to display large detailed images on a small display, e.g., the display of maps on a wireless device such as a cell phone or PDA. The invention also has applicability with vendors offer mapping functionality for portable, wireless devices in particular.
While the invention has been described with reference to the exemplary embodiments thereof, those skilled in the art will be able to make various modifications to the described embodiments of the invention without departing from the true spirit and scope of the invention.

Claims

1. (canceled)

2. A method comprising:

receiving, from a server, a first image tile of the plurality of image tiles that form a pan-able image;

displaying the first image tile of the plurality of image tiles on the display of the wireless computing device;

receiving, from the server, a second image tile of the plurality of image tiles, wherein the first and second image tiles share an edge;

displaying the second image tile of the plurality of image tiles in response to user input corresponding to panning in a given direction;

predicting a third image tile of the plurality of image tiles based on the second image tile and the given direction; and

requesting, from the server, the third image tile of the plurality of image tiles in response to the predicting.

3. The method of claim 2, wherein:

the wireless computing device stores a proper subset of the plurality of image tiles.

4. The method of claim 2, wherein:

the wireless computing device is a cell phone.

5. The method of claim 2, wherein:

the wireless computing device is a personal digital assistant (PDA).

6. The method of claim 2, further comprising:

receiving, prior to displaying the second image, a first set of image tiles, wherein each image tile in the first set of image tiles shares an edge with the first image tile.

7. The method of claim 6, further comprising:

receiving, subsequent to the receipt of the first set of image tiles and prior to displaying the second image, a second set of image tiles, wherein each image tile in the second set of image tiles shares a corner with the first image tile.

8. The method of claim 2, wherein:

the given direction corresponds to two directions.

9. The method of claim 2, wherein:

the given direction corresponds to a diagonal direction.

10. The method of claim 2, wherein:

the pan-able image is a map.

11. The method of claim 2, wherein:

the server and the wireless computing device communicate via a wireless carrier network.

12. A wireless computing device executing a client application that:

defines edges for a plurality of image tiles of a pan-able image, wherein each of the plurality of image tiles corresponds to a size of a display on the wireless computing device;

receives, from a server, a first image tile of the plurality of image tiles;

displays the first image tile of the plurality of image tiles on the display of the wireless computing device;

receives, from the server, a second image tile of the plurality of image tiles, wherein the first and second image tiles share an edge;

displays the second image tile of the plurality of image tiles in response to user input corresponding to panning in a given direction;

predicts a third tile of the plurality of image tiles based on the second image and the given direction; and

requests, from the server, the third tile of the plurality of image tiles in response to the predicting.

13. The wireless computing device of claim 12, wherein the client application further:

receives, prior to displaying the second image, a first set of image tiles, wherein each image tile in the first set of image tiles shares an edge with the first image tile.

14. The wireless computing device of claim 13, wherein the client application further:

receives, subsequent to the receiving of the first set of image tiles and prior to displaying the second image, a second set of image tiles, wherein each image tile in the second set of image tiles shares a corner with the first image tile.

15. The wireless computing device of claim 12, wherein:

the wireless computing device is a cell phone.

16. The wireless computing device of claim 12, wherein:

the wireless computing device is a personal digital assistant (PDA).

17. The wireless computing device of claim 12, wherein:

the given direction corresponds to two directions.

18. A non-transitory machine readable medium having machine executable instructions, the machine executable instruction causing a processor to:

define edges for a plurality of image tiles of a pan-able image, wherein each of the plurality of image tiles corresponds to a size of a display on a wireless computing device;

receive, from a server, a first image tile of the plurality of image tiles;

display the first image tile of the plurality of image tiles on the display of the wireless computing device;

receive, from the server, a first set of image tiles of the plurality of image tiles subsequent to receipt of the first image tile, wherein each image tile in the first set of image tiles shares an edge with the first image tile;

receive, from the server, a second set of image tiles of the plurality of image tiles subsequent to receipt of the first set of image tiles, wherein each image tile in the second set of image tiles shares a corner with the first image tile;

display a second image included in the first set of image tiles in response to user input corresponding to panning in a given direction;

predict a third image tile of the plurality of image tiles based on the second image and the given direction; and

requesting, from the server, the third tile of the plurality of image tiles in response to the predicting.

19. The medium of claim 18, wherein:

the given direction corresponds to two directions.

20. The medium of claim 18, wherein:

the pan-able image is a map.