HK1219145B - Device, method, and graphical user interface for displaying additional information in response to a user contact - Google Patents

Description

Device, method, and graphical user interface for displaying additional information in response to user contact

This application is a divisional application of the invention patent application with application date of May 8, 2013, application number 201380035982.1, and invention name “Device, method and graphical user interface for displaying additional information in response to user contact”.

Related patent applications

This patent application claims priority to U.S. Provisional Patent Application Serial No. 61/778,171, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Displaying Additional Information in Response to a User Contact,” U.S. Provisional Patent Application No. 61/747,278, filed on December 29, 2012, entitled “Device, Method, and Graphical User Interface for Manipulating User Interface Objects with Visual and/or Haptic Feedback,” and U.S. Provisional Patent Application No. 61/688,227, filed on May 9, 2012, entitled “Device, Method, and Graphical User Interface for Manipulating User Interface Objects with Visual and/or Haptic Feedback,” all of which are incorporated herein by reference in their entireties.

This patent application is also related to the following provisional patent applications: U.S. Provisional Patent Application Serial No. 61/778,092, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Selecting Object within a Group of Objects”; U.S. Provisional Patent Application Serial No. 61/778,125, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Navigating User Interface Hierarchies”; U.S. Provisional Patent Application Serial No. 61/778,156, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Manipulating Framed GraphicalObjects”; and U.S. Provisional Patent Application Serial No. 61/778,157, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Scrolling Nested Objects”. No. 61/778,179, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application,” U.S. Provisional Patent Application Serial No. 61/778,191, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Facilitating User Interaction with Controls in a User Interface,” U.S. Provisional Patent Application Serial No. 61/778,211, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Forgoing Generation of Tactile Output for a Multi-Contact Gesture,” U.S. Provisional Patent Application Serial No. 61/778,239, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Providing Tactile Feedback for Operations Performed in a User Interface”; U.S. Provisional Patent Application Serial No. 61/778,284, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Providing Feedback for Changing Activation States of a User Interface Object”; U.S. Provisional Patent Application Serial No. 61/778,287, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Transitioning between Touch Input to DisplayOutput Relationships”; U.S. Provisional Patent Application Serial No. 61/778,363, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Moving a User InterfaceObject Based on an Intensity of a Press Input”; U.S. Provisional Patent Application Serial No. 61/778,367, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Providing Feedback for Changing Activation States of a User Interface Object” No. 61/778,265, entitled “Transitioning between Display States in Response to a Gesture,” U.S. Provisional Patent Application Serial No. 61/778,373, filed on March 12, 2013, entitled “Device, Method, and Graphical User Interface for Managing Activation of a Control Based on Contact Intensity,” U.S. Provisional Patent Application Serial No. 61/778,412, filed on March 13, 2013, entitled “Device, Method, and Graphical User Interface for Displaying Content Associated with a Corresponding Affordance,” U.S. Provisional Patent Application Serial No. 61/778,413, filed on March 13, 2013, entitled “Device, Method, and Graphical User Interface for Selecting User Interface Objects,” U.S. Provisional Patent Application Serial No. 61/778,414, filed on March 13, 2013, entitled “Device, Method, and Graphical User Interface for Moving and Dropping a User Interface Object,” No. 61/778,418, filed on March 13, 2013, and entitled “Device, Method, and Graphical User Interface for Switching between User Interfaces,” are hereby incorporated by reference in their entirety.

This application is also related to the following provisional patent applications: U.S. Provisional Patent Application Serial No. 61/645,033, entitled “Adaptive Haptic Feedback for Electronic Devices,” filed on May 9, 2012, U.S. Provisional Patent Application Serial No. 61/665,603, entitled “Adaptive Haptic Feedback for Electronic Devices,” filed on June 28, 2012, and U.S. Provisional Patent Application Serial No. 61/681,098, entitled “Adaptive Haptic Feedback for Electronic Devices,” filed on August 8, 2012, all of which are incorporated herein by reference in their entirety.

Technical Field

This document generally relates to electronic devices with touch-sensitive surfaces, including but not limited to electronic devices with touch-sensitive surfaces that detect input for manipulating a user interface.

Background Art

The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has grown significantly in recent years. Exemplary touch-sensitive surfaces include touchpads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on displays.

Exemplary manipulations include adjusting the position and/or size of one or more user interface objects or activating a button or opening a file/application represented by a user interface object, as well as associating metadata with one or more user interface objects or otherwise manipulating the user interface. Exemplary user interface objects include digital images, videos, text, icons, control elements such as buttons, and other graphics. In some cases, a user will need to perform such manipulations of user interface objects in a file management program (e.g., Finder from Apple Inc. (Cupertino, California)), an image management application (e.g., Aperture or iPhoto from Apple Inc. (Cupertino, California)), a digital content (e.g., video and music) management application (e.g., iTunes from Apple Inc. (Cupertino, California)), a drawing application, a presentation application (e.g., Keynote from Apple Inc. (Cupertino, California)), a word processing application (e.g., Pages from Apple Inc. (Cupertino, California)), a website creation application (e.g., iWeb from Apple Inc. (Cupertino, California)), a disc editing application (e.g., iDVD from Apple Inc. (Cupertino, California)), or a spreadsheet application (e.g., Numbers from Apple Inc. (Cupertino, California)).

However, existing methods for performing these manipulations are cumbersome and inefficient. Furthermore, existing methods take longer than necessary, wasting energy. This latter consideration is particularly important in battery-powered devices.

Summary of the Invention

Therefore, electronic devices need to have faster, more efficient methods and interfaces for manipulating user interfaces. Such methods and interfaces optionally supplement or replace conventional methods for manipulating user interfaces. Such methods and interfaces reduce the cognitive burden on users and create more efficient human-computer interfaces. For battery-powered devices, such methods and interfaces save power and increase the time between battery charges.

The above-mentioned defects and other problems associated with the user interface of the electronic device with touch-sensitive surface are reduced or eliminated by the disclosed device. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a laptop, tablet computer or handheld device). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a "touch screen" or "touch screen display"). In some embodiments, the device has a graphical user interface (GUI), a memory, one or more processors and one or more modules, programs or instruction sets stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI mainly through finger contacts and gestures on the touch-sensitive surface. In some embodiments, these functions optionally include image editing, drawing, presentation, word processing, web page creation, disk editing, spreadsheet production, playing games, making and receiving calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback and/or digital video playback. The executable instructions for performing these functions are optionally included in a non-transient computer-readable storage medium or other computer program product configured for execution by one or more processors.

Electronic devices need faster, more efficient methods and interfaces for performing user interface operations on electronic devices with trackpads or touchscreens. Examples of such operations include scrolling through a list of items or deleting items from a list. Such methods and interfaces can supplement or replace conventional methods for performing user interface operations. Such methods and interfaces reduce the cognitive burden on users and create more efficient human-computer interfaces. For battery-powered devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contacts with the touch-sensitive surface. The method includes displaying a user interface on the display and detecting a gesture of a user on the touch-sensitive surface, the gesture including a contact corresponding to a focus selector at a corresponding location in the user interface, wherein the gesture is associated with a corresponding operation. The method also includes, in response to detecting the gesture, requesting confirmation that the user wants the device to perform the corresponding operation based on determining that the contact has a maximum intensity below a first intensity threshold. The method also includes performing the corresponding operation without requesting confirmation that the user wants the device to perform the corresponding operation based on determining that the maximum intensity of the contact is above the first intensity threshold.

In accordance with some embodiments, an electronic device includes a display unit configured to display a user interface on the display unit; a touch-sensitive surface unit configured to detect a gesture by a user on the touch-sensitive surface unit, the gesture comprising contact corresponding to a focus selector at a corresponding location in the user interface, wherein the gesture is associated with a corresponding operation; and a processing unit coupled to the display unit and the touch-sensitive surface unit. The processing unit is configured to: in response to detecting the gesture: request confirmation that the user wants the device to perform the corresponding operation based on determining that the contact has a maximum intensity below a first intensity threshold; and perform the corresponding operation without requesting confirmation that the user wants the device to perform the corresponding operation based on determining that the maximum intensity of the contact is above the first intensity threshold.

Thus, electronic devices having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contact with the touch-sensitive surface are provided with faster, more efficient methods and interfaces for performing user interface operations, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces can supplement or replace conventional methods for performing user interface operations.

There is a need for electronic devices to have faster, more efficient methods and interfaces for displaying additional (e.g., not initially displayed) information when a user contact is detected while a focus selector corresponding to the contact is at a location on the display associated with the additional information. Such methods and interfaces can supplement or replace conventional methods for displaying such additional information. Such methods and interfaces reduce the cognitive burden on the user and produce a more efficient human-computer interface. For battery-powered devices, such methods and interfaces save power and increase the time between battery charges.

In accordance with some embodiments, a method is performed on an electronic device having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contacts with the touch-sensitive surface. The method includes detecting a contact on the touch-sensitive surface when a focus selector corresponding to the contact is at a corresponding position on the display that is associated with additional information that was not initially displayed on the display. The method also includes, when the focus selector is at the corresponding position: based on determining that the contact had an intensity above a corresponding intensity threshold before a predefined delay time had elapsed while the focus selector was at the corresponding position, displaying the additional information associated with the corresponding position without waiting until the predefined delay time has elapsed; and based on determining that the contact had an intensity below the corresponding intensity threshold, waiting until the predefined delay time has elapsed to display the additional information associated with the corresponding position while the focus selector is at the corresponding position.

In accordance with some embodiments, an electronic device includes a display unit configured to display additional information associated with a corresponding location; a touch-sensitive surface unit configured to receive a contact on the touch-sensitive surface unit; one or more sensor units configured to detect an intensity of the contact with the touch-sensitive surface unit; and a processing unit coupled to the display unit, the touch-sensitive surface unit, and the sensor unit. The processing unit is configured to: detect the contact on the touch-sensitive surface unit when a focus selector corresponding to the contact is located at a corresponding location on the display unit associated with additional information that was not initially displayed on the display unit; and when the focus selector is at the corresponding location: based on determining that the contact had an intensity above a corresponding intensity threshold before a predefined delay time had elapsed when the focus selector was at the corresponding location, enable display of the additional information associated with the corresponding location without waiting until the predefined delay time has elapsed; and based on determining that the contact had an intensity below the corresponding intensity threshold, wait until the predefined delay time has elapsed when the focus selector is at the corresponding location to enable display of the additional information associated with the corresponding location.

Thus, electronic devices having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contact with the touch-sensitive surface are provided with faster, more efficient methods and interfaces for displaying additional (e.g., not initially displayed) information when contact is detected, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces can supplement or replace conventional methods for displaying additional (e.g., not initially displayed) information when contact is detected.

Electronic devices need faster, more efficient methods and interfaces for removing icons from predefined areas. Such methods and interfaces can supplement or replace conventional methods for removing icons from predefined areas. Such methods and interfaces reduce the cognitive burden on users and create a more efficient human-computer interface. For battery-powered devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed on an electronic device having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contacts with the touch-sensitive surface. The method includes: displaying a plurality of icons in a predefined area on the display, wherein the plurality of icons includes a respective icon; detecting a gesture, the gesture comprising: contact on the touch-sensitive surface while a focus selector is over the respective icon, and subsequent movement of the contact on the touch-sensitive surface, the subsequent movement corresponding to movement of the focus selector outside of the predefined area; and in response to detecting the gesture: displaying an indication that a gesture is being performed during the gesture based on a determination that the contact has a maximum intensity below a respective intensity threshold during the gesture, and retaining the respective icon in the predefined area after the gesture has ended; and based on a determination that the contact reaches an intensity above the respective intensity threshold during the gesture, moving the respective icon based on the movement of the contact during the gesture, and removing the respective icon from the predefined area after the gesture has ended.

In accordance with some embodiments, an electronic device includes a display unit configured to display a plurality of icons in a predefined area on the display unit, wherein the plurality of icons includes a respective icon; a touch-sensitive surface unit configured to receive a gesture; one or more sensor units configured to detect intensity of contact with the touch-sensitive surface unit; and a processing unit coupled to the display unit, the touch-sensitive surface unit, and the sensor unit. The processing unit is configured to: detect a gesture comprising contact on the touch-sensitive surface unit while a focus selector is over the respective icon, and subsequent movement of the contact on the touch-sensitive surface unit, the subsequent movement corresponding to movement of the focus selector outside of the predefined area, and in response to detecting the gesture: enable display of an indication that the gesture is being performed during the gesture based on determining that the contact has a maximum intensity below a corresponding intensity threshold during the gesture, and retain the respective icon in the predefined area after the gesture has ended; and based on determining that the contact reaches an intensity above the corresponding intensity threshold during the gesture, move the respective icon based on the movement of the contact during the gesture, and remove the respective icon from the predefined area after the gesture has ended.

Thus, electronic devices having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contact with the touch-sensitive surface are provided with faster, more efficient methods and interfaces for removing icons from predefined areas, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces can supplement or replace conventional methods for removing icons from predefined areas.

Electronic devices need faster, more efficient methods and interfaces for displaying content layers. Such methods and interfaces can supplement or replace conventional methods for displaying content layers. Such methods and interfaces reduce the cognitive burden on users and create a more efficient human-computer interface. For battery-powered devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed on an electronic device having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contacts with the touch-sensitive surface. The method includes: displaying first content on the display, wherein a portion of the first content is related to a corresponding portion of second content that is not displayed on the display; detecting an increase in intensity of the contact on the touch-sensitive surface above a corresponding intensity threshold when a focus selector is over the first portion of the first content; and in response to detecting the increase in intensity of the contact on the touch-sensitive surface above the corresponding intensity threshold: ceasing to display the first portion of the first content and displaying a first portion of the second content that corresponds to the first portion of the first content.

In accordance with some embodiments, an electronic device includes a display unit configured to display first content on the display unit, wherein a portion of the first content is related to a corresponding portion of second content that is not displayed on the display; a touch-sensitive surface unit configured to receive a contact; one or more sensors configured to detect an intensity of the contact with the touch-sensitive surface unit; and a processing unit coupled to the display unit, the touch-sensitive surface unit, and the sensors. The processing unit is configured to: when a focus selector is over a first portion of the first content, detect an increase in intensity of the contact on the touch-sensitive surface unit to above a corresponding intensity threshold; and in response to detecting an increase in intensity of the contact on the touch-sensitive surface unit to above the corresponding intensity threshold: stop displaying the first portion of the first content, and enable display of a first portion of the second content corresponding to the first portion of the first content.

Thus, electronic devices having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contact with the touch-sensitive surface are provided with faster, more efficient methods and interfaces for displaying content layers, thereby increasing the utility, efficiency, and user satisfaction of such devices. Such methods and interfaces can supplement or replace conventional methods for displaying content layers.

There is a need for electronic devices to have faster, more efficient methods and interfaces for navigating between user interfaces (e.g., between an inbox view and a conversation view in an email client application) using user input, such as input on a trackpad or touch screen, or input from a touch-sensitive surface. Such methods and interfaces can supplement or replace conventional methods for navigating between user interfaces. Such methods and interfaces reduce the cognitive burden on the user and produce a more efficient human-computer interface. For battery-powered devices, such methods and interfaces save power and increase the time between battery charges.

In accordance with some embodiments, a method is performed on an electronic device having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contacts with the touch-sensitive surface. The method includes: displaying a first user interface on the display. The method also includes detecting a first gesture on the touch-sensitive surface while a focus selector is over a first representation of an object in the first user interface, the first gesture including a first contact having a maximum intensity greater than a corresponding intensity threshold during the first gesture. The method also includes, in response to detecting the first gesture, replacing the first user interface with a second user interface including a second representation of the object. The method also includes detecting a second gesture on the touch-sensitive surface while the focus selector is over a second representation of the object in the second user interface, the second gesture including a second contact having a maximum intensity greater than a corresponding intensity threshold during the second gesture. The method also includes, in response to detecting the second gesture, replacing the second user interface with the first user interface.

In accordance with some embodiments, an electronic device includes a display unit configured to display a first user interface including a first representation of an object and a second user interface including a second representation of the object; a touch-sensitive surface unit configured to detect a first gesture on the touch-sensitive surface unit when a focus selector is over the first representation of the object in the first user interface, the first gesture including a first contact having a maximum intensity greater than a corresponding intensity threshold during the first gesture, and to detect a second gesture on the touch-sensitive surface unit when the focus selector is over the second representation of the object in the second user interface, the second gesture including a second contact having a maximum intensity greater than a corresponding intensity threshold during the second gesture; and a processing unit coupled to the display unit and the touch-sensitive surface unit. The processing unit is configured to: in response to detecting the first gesture, replace the first user interface with the second user interface; and in response to detecting the second gesture, replace the second user interface with the first user interface.

Thus, electronic devices having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contact with the touch-sensitive surface are provided with faster, more efficient methods and interfaces for navigating between displayed user interfaces, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces can supplement or replace conventional methods for navigating between displayed user interfaces.

In accordance with some embodiments, an electronic device includes a display, a touch-sensitive surface, a memory, one or more processors, one or more programs, and optionally one or more sensors for detecting intensity of contact with the touch-sensitive surface; the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing the operations of any of the methods mentioned in paragraph [0057]. In accordance with some embodiments, a graphical user interface on an electronic device having a display, a touch-sensitive surface, a memory, optionally one or more sensors for detecting intensity of contact with the touch-sensitive surface, and one or more processors executing the one or more programs stored in the memory includes one or more elements displayed in any of the methods mentioned in paragraph [0057], the one or more elements being updated in response to input, as described in any of the methods mentioned in paragraph [0057]. In accordance with some embodiments, a computer-readable storage medium has stored therein instructions that, when executed by an electronic device having a display, a touch-sensitive surface, and optionally one or more sensors for detecting intensity of contact with the touch-sensitive surface, cause the device to perform the operations of any of the methods mentioned in paragraph [0057]. According to some embodiments, an electronic device includes: a display, a touch-sensitive surface, and optionally one or more sensors for detecting intensity of contact with the touch-sensitive surface; and means for performing the operations of any of the methods mentioned in paragraph [0057]. According to some embodiments, an information processing apparatus for use in an electronic device having a display and a touch-sensitive surface, and optionally one or more sensors for detecting intensity of contact with the touch-sensitive surface, includes means for performing the operations of any of the methods mentioned in paragraph [0057].

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the various described embodiments of the present invention, reference should be made to the following description of the embodiments taken in conjunction with the following drawings, in which like reference numerals represent corresponding parts throughout the several views.

Figure 1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display in accordance with some embodiments.

FIG. 1B is a block diagram illustrating exemplary components for event processing, according to some embodiments.

FIG2 illustrates a portable multifunction device with a touch screen in accordance with some embodiments.

3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface in accordance with some embodiments.

Figure 4A illustrates an exemplary user interface for an applications menu on a portable multifunction device in accordance with some embodiments.

Figure 4B shows an exemplary user interface for a multifunction device with a touch-sensitive surface that is separate from the display in accordance with some embodiments.

5A-5I illustrate exemplary user interfaces for performing user interface operations in accordance with some embodiments.

6A-6B are flow charts illustrating methods of performing user interface operations according to some embodiments.

7 is a functional block diagram of an electronic device in accordance with some embodiments.

8A-8N illustrate exemplary user interfaces for displaying additional information associated with a location on a display based on contact intensity (eg, pressure) and contact timing (eg, duration) in accordance with some embodiments.

9A-9B are flow diagrams illustrating a method of displaying additional information associated with a location on a display based on contact intensity (eg, pressure) and contact timing (eg, duration), in accordance with some embodiments.

FIG10 is a functional block diagram of an electronic device in accordance with some embodiments.

11A-11U illustrate example user interfaces for removing icons from a predefined area, in accordance with some embodiments.

12A-12B are flow diagrams illustrating a method of removing an icon from a predefined area, according to some embodiments.

13 is a functional block diagram of an electronic device in accordance with some embodiments.

14A-14L illustrate exemplary user interfaces for displaying content layers, according to some embodiments.

15A-15C are flow diagrams illustrating methods of displaying content layers according to some embodiments.

FIG16 is a functional block diagram of an electronic device in accordance with some embodiments.

17A-17H illustrate exemplary user interfaces for navigating between displayed user interfaces in accordance with some embodiments.

18A-18C are flow diagrams illustrating methods of navigating between displayed user interfaces, according to some embodiments.

Figure 19 is a functional block diagram of an electronic device in accordance with some embodiments.

DETAILED DESCRIPTION

The methods, devices, and GUIs described herein provide visual and/or tactile feedback that makes manipulation of user interface objects more efficient and intuitive for the user. For example, in a system where a click action of a touchpad is decoupled from a contact intensity (e.g., contact force, contact pressure, or a substitute thereof) required to reach an activation threshold, the device can generate different tactile outputs (e.g., "different clicks") for different activation events (e.g., so that a click that achieves a specific result is distinguished from a click that produces no result or achieves a result different from the specific result). In addition, tactile output can be generated in response to other events that are not related to increased intensity of contact, such as when a user interface object moves to a specific position, boundary, or orientation, or when an event occurs at the device (e.g., a "stop").

Additionally, in systems where the trackpad or touchscreen display is sensitive to a range of contact intensities that includes more than one or two specific intensity values (e.g., beyond a simple on/off binary intensity determination), the user interface can provide a response (e.g., a visual or tactile cue) indicating contact intensities within this range. In some implementations, the pre-activation threshold response and/or post-activation threshold response for an input is displayed as a continuous animation. As one example of such a response, a preview of an operation is displayed in response to detecting an increase in contact intensity but still below an activation threshold for performing the operation. As another example of such a response, an animation associated with the operation continues even after the activation threshold for the operation has been reached. Both examples provide the user with a continuous response to the force or pressure of the user's contact, which provides the user with richer and more intuitive visual and/or tactile feedback. More specifically, such a continuous force response gives the user the experience of being able to press lightly to preview an operation and/or press deeply to "step through" or "pass" through a predefined user interface state corresponding to the operation.

In addition, for devices with touch-sensitive surfaces that are sensitive to a range of contact intensities, multiple contact intensity thresholds can be monitored by the device, and different functions can be mapped to different contact intensity thresholds. This serves to increase the available "gesture space," making it easier for users to access improved features, knowing that increasing the contact intensity at or above the second "deep press" intensity threshold will cause the device to perform an operation different from the operation that would have been performed if the contact intensity was between the first "activation" intensity threshold and the second "deep press" intensity threshold. An advantage of assigning additional functionality to the second "deep press" intensity threshold while maintaining familiar functionality at the first "activation" intensity threshold is that, in some cases, an inexperienced user who is confused by the additional functionality can use the familiar functionality by applying only an intensity up to the first "activation" intensity threshold, while a more experienced user can utilize the additional functionality by applying an intensity at the second "deep press" intensity threshold.

Additionally, for devices with a touch-sensitive surface that is sensitive to a range of contact intensities, the device can provide additional functionality by allowing the user to perform complex operations with a single continuous contact. For example, when selecting a group of objects, the user can move the continuous contact around the touch-sensitive surface and can press while dragging (e.g., applying an intensity greater than a "deep press" intensity threshold) to add additional elements to the selection. In this way, the user can intuitively interact with the user interface, where pressing harder with the contact makes objects in the user interface "stickier."

The following describes a number of different methods for providing an intuitive user interface on a device, where the click action is decoupled from the force required to reach an activation threshold and/or the device is sensitive to a wide range of contact strengths. Using one or more of these methods (optionally in combination with each other) helps provide a user interface that intuitively provides additional information and functionality to the user, thereby reducing the user's cognitive burden and improving the human-computer interface. Such improvements in the human-computer interface enable the user to use the device more quickly and efficiently. For battery-powered devices, these improvements save power and increase the time between battery charges. For ease of explanation, the following describes systems, methods, and user interfaces for illustrative examples including some of these methods:

Many electronic devices have user interfaces for performing user interface operations such as viewing user interface objects (photos, email messages, etc.) and deleting user interface objects. In some cases, destructive operations are performed (e.g., deleting an email), and the device requires additional input (e.g., confirming the input) before performing these operations. However, providing an input sequence can be confusing and time-consuming for the user. In the embodiments described below, the device determines whether to perform a specific destructive operation based on the contact intensity corresponding to the request to perform the specific destructive operation, thereby providing a more intuitive and efficient user interface for performing destructive operations. Specifically, Figures 5A-5I illustrate exemplary user interfaces for performing user interface operations. Figures 6A-6B are flow diagrams illustrating a method for performing user interface operations. The user interfaces in Figures 5A-5I are used to illustrate the process in Figures 6A-6B.

Many electronic devices have graphical user interfaces that include user interface objects. In some cases, additional (e.g., hidden) information associated with a user interface object is displayed at the user's request. Some devices display such additional information after, for example, detecting a focus selector over the user interface object for an extended period of time.

However, such methods are time consuming and frustrating for users. The embodiments described below provide improved methods for displaying additional information based on contact intensity (e.g., pressure) on a touch-sensitive surface, thereby providing a more convenient and more efficient user interface for displaying additional information. Specifically, Figures 8A-8N show exemplary user interfaces for displaying additional information associated with a location (e.g., corresponding to a user interface object) on a display based on contact intensity (e.g., pressure) and contact timing (e.g., duration). Figures 9A-9B are flow diagrams illustrating a method for displaying additional information based on contact intensity (e.g., pressure) and contact timing (e.g., duration). The user interfaces in Figures 8A-8N are used to illustrate the process in Figures 9A-9B.

Many electronic devices have a graphical user interface that includes an application taskbar or launcher bar. In some cases, users are able to remove or rearrange icons in the application taskbar or launcher bar. However, when icons are accidentally removed or rearranged, users may become frustrated or confused. In addition, adding back the accidentally removed icons and undoing the accidental rearrangement operation is frustrating and inefficient for the user. The embodiments described below provide an improved method for removing unwanted icons from a predefined area, such as a launcher bar, wherein the device determines whether to remove or rearrange icons in the predefined area based on contact intensity, thereby providing a more intuitive and more efficient user interface that is less frustrating for the user. Specifically, Figures 11A-11U illustrate an exemplary user interface for removing icons from a predefined area. Figures 12A-12B are flow diagrams illustrating a method for removing icons from a predefined area. The user interfaces in Figures 11A-11U are used to illustrate the process in Figures 12A-12B.

Many electronic devices include applications or user interfaces in which different corresponding content, such as layers of content, can be displayed. It is easy for users to lose their place within a layer or become confused about how to navigate within the layer. The embodiments described below provide an efficient and intuitive user interface for navigating and displaying layers of content by determining whether to reveal underlying layers, and optionally how many underlying layers to reveal, based on the intensity of contact on a touch-sensitive surface. Specifically, Figures 14A-14L illustrate exemplary user interfaces for displaying layers of content. Figures 15A-15C are flow diagrams illustrating a method of displaying layers of content. The user interfaces in Figures 14A-14L are used to illustrate the process in Figures 15A-15C.

Many electronic devices include user interfaces with different representations of the same object (e.g., content such as an email or a photo). Some methods for navigating between user interfaces on devices with a touch-sensitive surface typically require the user to locate an affordance (e.g., a key or "button"). However, such methods are problematic because they are prone to inaccuracies (e.g., accidental selection of an affordance when the affordance was intended to be selected, accidental selection of a different key). Such methods also increase the cognitive burden on the user by requiring additional, independent gestures. The embodiments described below provide convenient and intuitive methods for navigating between user interfaces corresponding to different representations of an object using similar gestures performed using touch when a focus selector is over the representation of the object. Specifically, Figures 17A-17H illustrate exemplary user interfaces for navigating between displayed user interfaces. Figures 18A-18C are flow diagrams illustrating a method for navigating between displayed user interfaces. The user interfaces in Figures 17A-17H are used to illustrate the process in Figures 18A-18C.

Exemplary devices

Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are provided to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, processes, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

It will also be understood that, although the terms first, second, etc. are used to describe various elements in some embodiments herein, these elements should not be limited by these terms. These terms are just used to distinguish an element from another element. For example, without departing from the scope of the various described embodiments, the first contact can be named as the second contact and similarly, the second contact can be named as the first contact. Both the first contact and the second contact are contacts, but they are not identical contacts.

The terms used in the description of the various described embodiments herein are only for the purpose of describing specific embodiments and are not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms "a," "an," and "said" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the terms "and/or" used herein refer to and include any and all possible combinations of one or more of the associated listed items. It will also be understood that when the terms "includes," "including," "comprises," and/or "comprising" are used in this specification, the presence of the features, integers, steps, operations, elements, and/or parts set forth are specified, but the presence or addition of one or more other features, integers, steps, operations, elements, parts, and/or groups thereof is not excluded.

As used herein, the term "if" is optionally interpreted to mean "when" or "upon," or "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [stated condition or event] is detected" are optionally interpreted to mean "when it is determined" or "in response to determining" or "when [stated condition or event] is detected" or "in response to detecting [stated condition or event]," depending on the context.

Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and/or music player functions. Exemplary embodiments of portable multifunction devices include, but are not limited to, iPods and devices from Apple Inc. (Cupertino, California). Other portable electronic devices, such as laptops or tablets with touch-sensitive surfaces (e.g., touch screen displays and/or touch pads), are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but rather a desktop computer with a touch-sensitive surface (e.g., touch screen displays and/or touch pads).

In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and/or joystick.

The device typically supports a variety of applications, such as one or more of the following applications: a drawing application, a rendering application, a word processing application, a web page creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and/or a digital video player application.

Various applications executing on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device optionally adjust and/or change from one application to the next and/or within a respective application. In this way, the common physical architecture of the device (such as a touch-sensitive surface) optionally supports the various applications with a user interface that is intuitive and clear to the user.

Attention is now directed to embodiments of portable devices with touch-sensitive displays. Figure 1A is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display 112, in accordance with some embodiments. Touch-sensitive display 112 is sometimes referred to as a "touch screen" for convenience, and is sometimes referred to as or referred to as a touch-sensitive display system. Device 100 includes memory 102 (optionally including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input/output (I/O) subsystem 106, other input or control devices 116, and external ports 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting intensity of contact on device 100 (e.g., a touch-sensitive surface, such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or to a surrogate (surrogate) for the force or pressure of a contact on the touch-sensitive surface. Contact intensity has a range of values that includes at least four different values and more typically includes hundreds of different values (e.g., at least 256). Contact intensity is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some embodiments, the force measurements from multiple force sensors are combined (e.g., a weighted average) to determine the estimated force of the contact. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and/or its change, the capacitance of the touch-sensitive surface adjacent to the contact and/or its change, and/or the resistance of the touch-sensitive surface adjacent to the contact and/or its change are optionally used as a surrogate for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the surrogate measurement of the force or pressure of the contact is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measurement). In some embodiments, the surrogate measurement of the force or pressure of the contact is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure).

As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, as detected by a user using the user's touch sensing. For example, when a device or a component of the device comes into contact with a touch-sensitive surface of a user (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical properties of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) is optionally interpreted by the user as a "down click" or "up click" on a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "down click" or "up click," even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, movement of a touch-sensitive surface may be interpreted or sensed by a user as "roughness" of the touch-sensitive surface, even when the smoothness of the touch-sensitive surface does not change. While such interpretation of touch by a user will be limited by the user's individualized perception, there are many perceptions of touch that are common to most users. Thus, when a tactile output is described as corresponding to a particular perception of a user (e.g., "down click," "up click," "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or a component thereof that would generate the described perception for a typical (or average) user.

It should be understood that device 100 is only one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. The various components shown in FIG1A are implemented in hardware, software, or a combination of hardware and software, including one or more signal processing circuits and/or application specific integrated circuits.

Memory 102 optionally includes high-speed random access memory and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of device 100 (such as CPU 120 and peripheral device interface 118) is optionally controlled by memory controller 122.

Peripherals interface 118 may be used to couple the device's input and output peripherals to CPU 120 and memory 102. The one or more processors 120 run or execute various software programs and/or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

In some embodiments, peripherals interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

RF (radio frequency) circuitry 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuitry 108 converts electrical signals into/from electromagnetic signals and communicates with a communication network and other communication devices via the electromagnetic signals. RF circuitry 108 optionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, a tuner, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, memory, one or more amplifiers, one or more oscillators, and the like. RF circuitry 108 optionally communicates with a network and other devices via wireless communications, such as the Internet (also known as the World Wide Web (WWW)), an intranet, and/or a wireless network (such as a cellular telephone network, a wireless local area network (LAN), and/or a metropolitan area network (MAN)). Wireless communication optionally uses any of a variety of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Evolution, Data Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and/or IEEE 802.11d), 802.11n), Voice over Internet Protocol (VoiP), Wi-MAX, protocols for email (e.g., Internet Message Access Protocol (IMAP) and/or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilization Extensions (SIMPLE), Instant Messaging and Presence Services (IMPS)) and/or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols that have not yet been developed on the date of filing this document.

The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into electrical signals, and transmits the electrical signals to the speaker 111. The speaker 111 converts the electrical signals into sound waves audible to humans. The audio circuit 110 also receives electrical signals converted from sound waves from the microphone 113. The audio circuit 110 converts the electrical signals into audio data and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and/or transmitted to the memory 102 and/or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., 212 in FIG. 2 ). The headset jack provides an interface between the audio circuit 110 and a removable audio input/output peripheral device, such as an output-only headset or a headset with both output (e.g., single-ear or dual-ear headphones) and input (e.g., a microphone).

The I/O subsystem 106 couples input/output peripherals on the device 100, such as the touch screen 112 and other input control devices 116, to a peripheral device interface 118. The I/O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive/send electrical signals from/to other input control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, and the like. In some alternative embodiments, the one or more input controllers 160 are optionally coupled to (or not coupled to) any of the following devices: a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. The one or more buttons (e.g., 208 in FIG. 2 ) optionally include an up/down button for volume control of the speaker 111 and/or microphone 113. The one or more buttons optionally include a push button (eg, 206 in FIG. 2 ).

The touch-sensitive display 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives and/or sends electrical signals to the touch screen 112. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects.

The touch screen 112 has a touch-sensitive surface, sensor, or sensor group that accepts input from the user based on tactile and/or tactile contact. The touch screen 112 and display controller 156 (together with any associated modules and/or instruction sets in memory 102) detect contact on the touch screen 112 (and any movement or interruption of that contact) and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In an exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to the user's finger.

The touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. The touch screen 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen 112 to detect contact and any movement or interruption thereof. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that found in the iPod and the iPhone from Apple Inc. (Cupertino, California).

The touch screen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touch screen has a video resolution of approximately 160 dpi. The user optionally uses any suitable object or appendage, such as a stylus, finger, or the like, to contact the touch screen 112. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures, which may be less precise than stylus-based input due to the larger contact area of a finger on the touch screen. In some embodiments, the device translates rough finger-based input into precise pointer/cursor positions or commands for performing the user's desired action.

In some embodiments, in addition to the touch screen, device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from touch screen 112 or an extension of the touch-sensitive surface formed by the touch screen.

Device 100 also includes a power system 162 for powering the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, power fault detection circuitry, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

The device 100 optionally also includes one or more optical sensors 164. Figure 1A shows an optical sensor coupled to the optical sensor controller 158 in the I/O subsystem 106. The optical sensor 164 optionally includes a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected from the environment through one or more lenses and converts the light into data representing an image. In conjunction with the imaging module 143 (also called a camera module), the optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the rear of the device 100, opposite the touch screen display 112 on the front of the device, so that the touch screen display can be used as a viewfinder for still images and/or video image acquisition. In some embodiments, another optical sensor is located on the front of the device so that the user can optionally obtain an image of the user for video conferencing while viewing other video conference participants on the touch screen display.

Device 100 optionally also includes one or more contact force sensors 165. FIG1A shows a contact force sensor coupled to force sensor controller 159 in I/O subsystem 106. Contact force sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of a contact on a touch-sensitive surface). Contact force sensor 165 receives contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the back of device 100, opposite touch screen display 112 located on the front of device 100.

Device 100 optionally also includes one or more proximity sensors 166. FIG1A shows proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 is coupled to input controller 160 in I/O subsystem 106. In some embodiments, the proximity sensor turns off and disables touch screen 112 when the multifunction device is placed near a user's ear (e.g., when the user is making a phone call).

Device 100 optionally also includes one or more tactile output generators 167. FIG1A shows a tactile output generator coupled to tactile feedback controller 161 in I/O subsystem 106. Tactile output generator 167 optionally includes one or more electroacoustic devices such as speakers or other audio components; and/or electromechanical devices that convert energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile outputs on the device). Contact force sensor 165 receives tactile feedback generation instructions from tactile feedback module 133 and generates tactile outputs on device 100 that can be sensed by a user of device 100. In some embodiments, at least one tactile output generator is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112) and optionally generates tactile outputs by moving the touch-sensitive surface vertically (e.g., inward/outward of the surface of device 100) or laterally (e.g., back and front in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the back of device 100 , opposite touch screen display 112 located on the front of device 100 .

The device 100 optionally also includes one or more accelerometers 168. FIG1A shows an accelerometer 168 coupled to the peripherals interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I/O subsystem 106. In some embodiments, information is displayed on the touch screen display in a portrait view or a landscape view based on an analysis of data received from the one or more accelerometers. In addition to the one or more accelerometers 168, the device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact/motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and applications (or instruction sets) 136. In addition, in some embodiments, memory 102 stores a device/global internal state 157, as shown in Figures 1A and 3. The device/global internal state 157 includes one or more of the following states: an active application state, which indicates which applications (if any) are currently active; a display state, which indicates what applications, views, or other information occupy various areas of the touch screen display 112; a sensor state, which includes information obtained from various sensors and input control devices 116 of the device; and position information regarding the position and/or posture of the device.

The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

The communication module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuitry 108 and/or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are suitable for coupling directly to other devices or indirectly via a network (e.g., the Internet, a wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, similar to, and/or compatible with the 30-pin connector used on iPod (trademark of Apple Inc.) devices.

The contact/motion module 130 optionally detects contact with the touch screen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact/motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger down event), determining the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there has been movement of the contact and tracking the movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has ceased (e.g., detecting a finger up event or a break in contact). The contact/motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and/or acceleration (change in magnitude and/or direction) of the contact point, with the movement of the contact point represented by a series of contact data. These operations are optionally applied to a single contact (e.g., a single finger contact) or multiple simultaneous contacts (e.g., "multi-touch"/multiple finger contacts). In some embodiments, contact/motion module 130 and display controller 156 detect contact on a touchpad.

In some embodiments, the contact/motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be adjusted without changing the physical hardware of the device 100). For example, a mouse "click" threshold for a touchpad or touchscreen can be set to any one of a large range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and/or by adjusting multiple intensity thresholds at once using a system-level click on an "intensity" parameter).

The contact/motion module 130 optionally detects gesture input from the user. Different gestures on the touch-sensitive surface have different contact patterns and intensities. Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). For another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and then detecting a finger lift (lift-off) event.

The graphics module 132 includes various known software components for rendering and displaying graphics on the touch screen 112 or other display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, and the like.

In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes specifying the graphics to be displayed from an application program, etc., along with coordinate data and other graphic attribute data, if necessary, and then generates screen image data for output to the display controller 156.

Haptic feedback module 133 includes various software components for generating instructions used by one or more tactile output generators 167 to produce tactile output at one or more locations on device 100 in response to user interaction with device 100 .

Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the phone 138 for location-based dialing, to the camera 143 as picture/video metadata, and to applications that provide location-based services such as the weather widget, local pages widget, and map/navigation widget).

Application 136 optionally includes the following modules (or instruction sets), or a subset or superset thereof:

Contacts module 137 (sometimes called address book or contact list);

Telephone module 138;

Video conferencing module 139;

Email client module 140;

Instant messaging (IM) module 141;

Fitness support module 142;

A camera module 143 for still and/or video images;

Image management module 144;

Browser module 147;

Calendar module 148;

A widget module 149, which optionally includes one or more of the following: a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5, and other widgets acquired by the user and a widget 149-6 created by the user;

A widget creator module 150 for forming user-created widgets 149-6;

Search module 151;

Video and music player module 152, which is optionally composed of a video player module and a music player module;

Notepad module 153;

Map module 154; and/or

Online video module 155.

Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

In conjunction with the touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contacts module 137 is optionally used to manage an address book or contact list (e.g., stored in the application internal state 192 of the contacts module 137 in memory 102 or memory 370), including: adding one or more names to the address book; deleting one or more names from the address book; associating one or more phone numbers, one or more email addresses, one or more physical addresses, or other information with a name; associating images with a name; categorizing and classifying names; providing phone numbers or email addresses to initiate and/or facilitate communications via telephone 138, video conferencing 139, email 140, or IM 141; and so on.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 is optionally used to enter a character sequence corresponding to a telephone number, access one or more telephone numbers in address book 137, modify an already entered telephone number, dial the corresponding telephone number, conduct a conversation, and disconnect or hang up when the conversation is complete. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

In combination with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and phone module 138, video conferencing module 139 includes executable instructions for initiating, conducting, and terminating a video conference between a user and one or more other participants in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.

In conjunction with the RF circuitry 108, the touch screen 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for entering a character sequence corresponding to an instant message, modifying previously entered characters, transmitting the corresponding instant message (e.g., using the Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and/or received instant messages optionally include graphics, photos, audio files, video files, and/or other attachments supported in MMS and/or Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

In combination with the RF circuitry 108, the touch screen 112, the display controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, the map module 154, and the music player module 146, the fitness support module 142 includes executable instructions for creating a fitness (e.g., with time, distance, and/or calorie burn goals); communicating with fitness sensors (sports equipment); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting and playing music for fitness; and displaying, storing, and transmitting fitness data.

In conjunction with the touch screen 112, display controller 156, optical sensor controller 158, contact module 130, graphics module 132, image management module 144, and one or more optical sensors 164, the camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in the memory 102, modifying the characteristics of the still images or videos, or deleting the still images or videos from the memory 102.

In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and/or video images.

In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving and displaying web pages or portions thereof and attachments and other files linked to web pages) in accordance with user instructions.

In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

In conjunction with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5) or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the desktop widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., the Yahoo! desktop widget).

In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134 and the browser module 147, the desktop widget creator module 150 is optionally used by a user to create a desktop widget (for example, to transfer a user-specified portion of a web page to a desktop widget).

In combination with the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos and/or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), as well as to display, present, or otherwise play back video (e.g., on touch screen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (trademark of Apple Inc.).

In conjunction with the touch screen 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do lists, etc. according to user instructions.

In combination with the RF circuitry 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, and the browser module 147, the map module 154 is optionally used to receive, display, modify, and store maps and data associated with the maps (e.g., driving directions; data about stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes instructions that allow a user to access, browse, receive (e.g., by streaming and/or downloading), play back (e.g., on a touch screen connected via external port 124 or on an external display), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats, such as H.264. In some embodiments, instant messaging module 141, rather than email client module 140, is used to send a link to a particular online video.

Each module and application program identified above corresponds to a set of executable instructions for performing one or more functions and methods described in the present application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) do not have to be implemented as separate software programs, processes, or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above modules and data structures. In addition, memory 102 optionally stores other modules and data structures not described above.

In some embodiments, device 100 is a device in which operation of a predefined set of functions on the device is performed exclusively through a touch screen and/or a touchpad. By using the touch screen and/or the touchpad as the primary input control device for operation of device 100, the number of physical input control devices (such as push buttons, dials, etc.) on device 100 is optionally reduced.

The predefined set of functions that are performed exclusively through the touch screen and/or touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to a main menu, a home menu, or a root menu. In such embodiments, the touchpad is used to implement a "menu button." In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.

FIG1B is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 (in FIG1A ) or memory 370 ( FIG3 ) includes an event classifier 170 (e.g., in operating system 126 ) and a corresponding application 136 - 1 (e.g., any of the aforementioned applications 137 - 151 , 155 , 380 - 390 ).

Event classifier 170 receives event information and determines an application 136-1 and an application view 191 of application 136-1 to which the event information is to be delivered. Event classifier 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or executing. In some embodiments, device/global internal state 157 is used by event classifier 170 to determine which application(s) are currently active, and application internal state 192 is used by event classifier 170 to determine an application view 191 to which the event information is to be delivered.

In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when the application 136-1 resumes execution, user interface state information indicating information being displayed by the application 136-1 or information ready for display by the application 136-1, a state queue for enabling a user to return to a previous state or view of the application 136-1, and a redo/undo queue of previous actions taken by the user.

Event monitor 171 receives event information from peripherals interface 118. The event information includes information about sub-events (e.g., a user contacting the touch-sensitive display 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I/O subsystem 106 or sensors (such as proximity sensor 166), microphone 113, and/or one or more accelerometers 168 (via audio circuit 110). The information received by peripherals interface 118 from I/O subsystem 106 includes information from touch-sensitive display 112 or a touch-sensitive surface.

In some embodiments, event monitor 171 sends requests to peripherals interface 118 at predetermined intervals. In response, peripherals interface 118 transmits event information. In other embodiments, peripherals interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and/or receiving an input for more than a predetermined duration).

In some embodiments, event classifier 170 also includes a hit view determination module 172 and/or an active event recognizer determination module 173 .

When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides software procedures for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to programmatic levels within the application's programmatic or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally called a hit view, and the set of events recognized as correct input is optionally determined based at least in part on the hit view that begins with the initial touch of a touch-based gesture.

Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies a hit view as the lowest view in the hierarchy that should handle the sub-event. In most cases, the hit view is the lowest-level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to the area associated with a particular view, views higher in the hierarchy will still remain actively participating views.

Event dispatcher module 174 dispatches event information to an event recognizer (e.g., event recognizer 180). In embodiments that include active event recognizer determination module 173, event dispatcher module 174 delivers the event information to the event recognizer determined by active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores the event information in an event queue, which is retrieved by corresponding event receiver module 182.

In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In another embodiment, event classifier 170 is a standalone module or part of another module stored in memory 102, such as contact/motion module 130.

In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes multiple event recognizers 180. In other embodiments, one or more of event recognizers 180 is part of a separate module, such as a user interface toolkit (not shown) or a higher-level object from which application 136-1 inherits methods and other characteristics. In some embodiments, a corresponding event handler 190 includes one or more of the following: data updater 176, object updater 177, GUI updater 178, and/or event data 179 received from event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Alternatively, one or more of application views 191 includes one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176 , object updater 177 , and GUI updater 178 are included in a corresponding application view 191 .

A corresponding event identifier 180 receives event information (e.g., event data 179) from event classifier 170 and identifies an event from the event information. Event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event identifier 180 also includes at least a subset of the following: metadata 183 and event delivery instructions 188 (optionally including sub-event delivery instructions).

Event receiver 182 receives event information from event classifier 170. Event information includes information about sub-events, such as touches or touch movements. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of a touch, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also called the device posture).

Event comparator 184 compares the event information with predefined event or sub-event definitions and determines the event or sub-event, or determines or updates the state of the event or sub-event based on the comparison. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes the definition of the event (e.g., a predefined sequence of sub-events), such as event 1 (187-1), event 2 (187-2), and other events. In some embodiments, the sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multiple touches. In one instance, the definition of event 1 (187-1) is a double-click on a displayed object. For example, the double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on the displayed object, and a second lift (touch end) of a predetermined duration. In another instance, the definition of event 2 (187-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch across the touch-sensitive display 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.

In some embodiments, event definitions 187 include definitions of events for corresponding user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view in which three user interface objects are displayed on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

In some embodiments, the definition of the corresponding event 187 also includes a delay action that delays delivery of the event information until after it has been determined whether the sequence of sub-events corresponds to the event type of the event identifier.

After it disregards subsequent sub-events of the touch-based gesture, the corresponding event recognizer 180 enters the event not possible, event failed, or event ended state when it determines that the sub-event string does not match any event in event definitions 186. In this case, other event recognizers (if any) that remain active for tapping the view continue to track and process the sub-events of the ongoing contact-based gesture.

In some embodiments, corresponding event recognizers 180 include metadata 183 with configurable properties, flags, and/or lists that indicate how the event delivery system should perform sub-event delivery for actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and/or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and/or lists that indicate whether sub-events are delivered to varying levels in a view or programmatic hierarchy.

In some embodiments, when one or more specific sub-events of an event are recognized, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating an event handler 190 is different from sending (and deferred sending) the sub-events to the corresponding hit view. In some embodiments, event recognizer 180 throws a flag associated with the recognized event, and the event handler 190 associated with the flag receives the flag and performs a predefined process.

In some embodiments, event delivery instructions 188 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver the event information to an event handler associated with the sub-event string or to an actively participating view. The event handler associated with the sub-event string or the actively participating view receives the event information and performs a predetermined process.

In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or video files stored in video player module 145. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 176 creates new user interface objects or updates the positions of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends it to graphics module 132 for display on a touch-sensitive display.

In some embodiments, one or more event handlers 190 include or have access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of a corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

It should be understood that the above discussion of event handling for user touches on a touch-sensitive display also applies to other forms of user input for operating the multifunction device 100 using an input device, and not all user input is initiated on a touch screen. For example, mouse movement and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holddowns; contact movement on a touchpad, such as taps, drags, scrolls, etc.; stylus input; movement of the device; spoken commands; detected eye movement; biometric input; and/or any combination thereof, are optionally used as input corresponding to sub-events defining the event to be recognized.

FIG2 illustrates a portable multifunction device 100 with a touch screen 112, according to some embodiments. The touch screen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment, as well as other embodiments described below, a user can select one or more of the graphics by, for example, gesturing on the graphics with one or more fingers 202 (not drawn to scale in the figures) or with one or more styluses 203 (not drawn to scale in the figures). In some embodiments, selection of the one or more graphics occurs when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, from right to left, up, and/or down), and/or a flick (from right to left, from left to right, up, and/or down) of a finger already in contact with the device 100. In some implementations or in some circumstances, inadvertent contact with a graphic does not select the graphic. For example, a swipe gesture sweeping over an application icon may not select the corresponding application when the gesture corresponding to selection is a tap.

The device 100 optionally also includes one or more physical buttons, such as a “home” button or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen 112.

In one embodiment, device 100 includes a touch screen 112, a menu button 204, a push button 206 for turning the device on and off and locking the device, a subscriber identity module (SIM) card slot 210, a headset jack 212, a docking/charging external port 124, and one or more volume adjustment buttons 208. Push button 206 is optionally used to turn the device on and off by pressing and holding the button for a predefined time interval; to lock the device by pressing and releasing the button before the predefined time interval has elapsed; and/or to unlock the device or initiate an unlocking process. In an alternative embodiment, device 100 also accepts voice input for activating or deactivating certain functions via microphone 113. Device 100 also optionally includes one or more contact force sensors 165 for detecting contact force on touch screen 112 and/or one or more tactile output generators 167 for generating tactile output for a user of device 100.

3 is a block diagram of an exemplary multifunctional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a game system, or a control device (e.g., a home or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more networks or other communication interfaces 360, a memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes a circuit (sometimes called a chipset) that interconnects system components and controls the communication between system components. Device 300 includes an input/output (I/O) interface 330 with a display 340, which is typically a touch screen display. I/O interface 330 also optionally includes a keyboard and/or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 for generating tactile output on device 300 (e.g., similar to one or more tactile output generators 167 described above with reference to FIG. 1A ), and sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch-sensitive sensors, and/or contact force sensors similar to one or more contact force sensors 165 described above with reference to FIG. 1A ). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from one or more CPUs 310. In some embodiments, memory 370 stores programs, modules, and data structures similar to those stored in memory 102 of portable multifunction device 100 ( FIG. 1A ), or a subset thereof. In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, a word processing module 384, a website creation module 386, a disk editing module 388, and/or a spreadsheet module 390, while memory 102 of portable multifunction device 100 ( FIG. 1A ) optionally does not store these modules.

Each element in the above-mentioned identified element among Fig. 3 is optionally stored in one or more previously mentioned memory devices.Each module in the above-mentioned identified module corresponds to the instruction set for performing the functions described above.The above-mentioned identified module or program (that is, instruction set) need not be implemented as independent software program, process or module, and therefore the various subsets of these modules are optionally combined or otherwise rearranged in various embodiments.In certain embodiments, memory 370 optionally stores the subset of above-mentioned modules and data structure.In addition, memory 370 optionally stores other modules and data structure not described above.

Attention is now turned to an embodiment of a user interface (“UI”) that is optionally implemented on portable multifunction device 100 .

Figure 4A shows an exemplary user interface for an application menu on portable multifunction device 100 according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

One or more signal strength indicators 402 of one or more wireless communications, such as cellular signals and Wi-Fi signals;

Time 404;

Bluetooth indicator 405;

Battery status indicator 406;

A tray 408 with icons for frequently used applications, such as:

o An icon 416 of the phone module 138 labeled "Phone," optionally including an indicator 414 of the number of missed calls or voicemail messages;

o An icon 418 of the email client module 140 labeled "Mail," optionally including an indicator 410 of the number of unread emails;

o An icon 420 of the browser module 147 labeled "Browser"; and

o An icon 422 of the video and music player module 152 labeled “iPod” (also called the iPod (trademark of Apple Inc.) module 152); and

Icons for other applications, such as:

o Icon 424 of the IM module 141 labeled "Text";

o An icon 426 of the calendar module 148 labeled "Calendar";

o Icon 428 of the image management module 144 labeled "Photos";

o An icon 430 of the camera module 143 labeled “Camera”;

o Icon 432 of the online video module 155 labeled "Online Video";

○ Icon 434 of the stock market widget 149-2 labeled "Stock Market";

o An icon 436 of the map module 154 labeled "Map";

○ Icon 438 of the weather widget 149-1 labeled "Weather";

○ Icon 440 of the alarm clock widget 149-4 labeled "Clock";

o An icon 442 of the fitness support module 142 labeled “Fitness Support”;

o An icon 444 of the Notes module 153 labeled "Notes"; and

o A settings application or module icon 446 that provides access to settings for the device 100 and its various applications 136;

It should be noted that the icon labels shown in FIG4A are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player." Other labels are optionally used for various application icons. In some embodiments, the label of a corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

4B shows an exemplary user interface on a device (e.g., device 300 in FIG. 3 ) having a touch-sensitive surface 451 (e.g., tablet computer or touchpad 355 in FIG. 3 ) that is separate from a display 450 (e.g., touch screen display 112 ). Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 357 ) for detecting intensity of contacts on touch-sensitive surface 451 and/or one or more tactile output generators 359 for generating tactile output for a user of device 300.

Although some of the examples that follow will be given with reference to input on a touchscreen display 112 (where a touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, as shown in FIG4B . In some embodiments, the touch-sensitive surface (e.g., 451 in FIG4B ) has a primary axis (e.g., 452 in FIG4B ) that corresponds to a primary axis (e.g., 453 in FIG4B ) on the display (e.g., 450 ). According to these embodiments, the device detects contacts (e.g., 460 and 462 in FIG4B ) with the touch-sensitive surface 451 at locations corresponding to corresponding locations on the display (e.g., 460 corresponds to 468 and 462 corresponds to 470 in FIG4B ). Thus, when the touch-sensitive surface (e.g., 451 in FIG4B ) is separated from the display (450 in FIG4B ) of the multifunction device, user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is used by the device to manipulate a user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

Additionally, while the examples below are primarily given with reference to finger inputs (e.g., finger contacts, single-finger tap gestures, finger swipe gestures), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of contact) followed by movement of a cursor along the path of the swipe (e.g., instead of movement of the contact). For another example, a tap gesture is optionally replaced by a mouse click (e.g., instead of detection of contact followed by cessation of detecting contact) when the cursor is over the location of the tap gesture. Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously or one mouse and multiple finger contacts are optionally used simultaneously.

As used herein, the term "focus selector" refers to an input element that indicates the current portion of a user interface with which a user is interacting. In some embodiments that include a cursor or other position marker, the cursor acts as a "focus selector" such that when input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4B ) while the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the particular user interface element is adjusted according to the detected input. In some embodiments that include a touch screen display (e.g., touch-sensitive display system 112 in FIG. 1A or touch screen 112 in FIG. 4A ) that enables direct interaction with user interface elements on the touch screen display, a contact detected on the touch screen acts as a "focus selector" such that when input (e.g., a press input by a contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touch screen display, the particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one area of a user interface to another area of the user interface without corresponding movement of a cursor or movement of a contact on a touch screen display (e.g., by using a tab key or arrow keys to move focus from one button to another); in these implementations, a focus selector moves in accordance with the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user interface element (or contact on a touch screen display) that is controlled by the user to convey the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user intends to interact). For example, the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touch screen) will indicate that the user intends to activate the corresponding button (as opposed to other user interface elements shown on the device's display).

The user interface diagrams described below include various intensity diagrams that illustrate the current intensity of a contact on the touch-sensitive surface relative to one or more intensity thresholds (e.g., a contact detection intensity threshold IT ₀ , a light press intensity threshold IT _L , a deep press intensity threshold IT _D , and/or one or more other intensity thresholds). This intensity diagram is not typically part of the displayed user interface, but is provided to aid in explaining the diagrams. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device performs an action typically associated with clicking a button on a physical mouse or trackpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device performs an action different from the action typically associated with clicking a button on a physical mouse or trackpad. In some embodiments, when a contact is detected with an intensity below the light press intensity threshold (e.g., and above a nominal contact detection intensity threshold IT ₀ , below which contacts are no longer detected), the device moves the focus selector in accordance with the movement of the contact on the touch-sensitive surface without performing the action associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise noted, these intensity thresholds are consistent across different sets of user interface diagrams.

An increase in contact intensity from an intensity below a light press intensity threshold IT _L to an intensity between the light press intensity threshold IT _L and the deep press intensity threshold IT _D is sometimes referred to as a "light press" input. An increase in contact intensity from an intensity below a deep press intensity threshold IT _D to an intensity above the deep press intensity threshold IT _D is sometimes referred to as a "deep press" input. An increase in contact intensity from an intensity below a contact detection intensity threshold IT ₀ to an intensity between the contact detection intensity threshold IT ₀ and the light press intensity threshold IT _L is sometimes referred to as detecting a contact on the touch surface. A decrease in contact intensity from an intensity above the contact detection intensity threshold IT ₀ to an intensity below the contact intensity threshold IT ₀ is sometimes referred to as detecting liftoff of the contact from the touch surface. In some embodiments, IT ₀ is zero. In some embodiments, IT ₀ is greater than zero. In some illustrations, shaded circles or ellipses are used to represent contact intensities on the touch-sensitive surface. In some illustrations, unshaded circles or ellipses are used to represent corresponding contacts on the touch-sensitive surface without specifying corresponding contact intensities.

In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a corresponding press input, or in response to detecting a corresponding press input performed with a corresponding contact (or contacts), where the corresponding press input is detected based at least in part on detecting an increase in intensity of the contact (or contacts) above a press input intensity threshold. In some embodiments, the corresponding operation is performed in response to detecting an increase in intensity of the corresponding contact above the press input intensity threshold (e.g., a "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in intensity of the corresponding contact above the press input intensity threshold and a subsequent decrease in intensity of the contact below the press input intensity threshold, and the corresponding operation is performed in response to detecting a subsequent decrease in intensity of the corresponding contact below the press input threshold (e.g., an "up stroke" of the corresponding press input).

In some embodiments, the device employs intensity hysteresis to avoid unexpected inputs, sometimes referred to as "jitter," where the device defines or selects a hysteresis intensity threshold that has a predefined relationship to a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable fraction of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in intensity of the corresponding contact above the press input intensity threshold and a subsequent decrease in the intensity of the contact below a hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting the subsequent decrease in intensity of the corresponding contact below the hysteresis intensity threshold (e.g., an "upstroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects an increase in contact intensity from an intensity at or below the hysteresis intensity threshold to an intensity at or above the press input intensity threshold, and optionally a subsequent decrease in contact intensity to an intensity at or below the hysteresis intensity, and a corresponding operation is performed in response to detecting the press input (e.g., an increase in contact intensity or a decrease in contact intensity, depending on the circumstances).

For ease of explanation, descriptions of operations performed in response to a press input associated with a press input intensity threshold, or in response to a gesture including a press input, are optionally triggered in response to detecting any of: an increase in the intensity of the contact above the press input intensity threshold, an increase in the intensity of the contact from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, a decrease in the intensity of the contact below the press input intensity threshold, and/or a decrease in the intensity of the contact below a hysteresis intensity threshold that corresponds to the press input intensity threshold. Additionally, in instances where an operation is described as being performed in response to detecting a decrease in the intensity of the contact below the press input intensity threshold, the operation is optionally performed in response to detecting a decrease in the intensity of the contact below a hysteresis intensity threshold that corresponds to and is less than the press input intensity threshold.

User interface and associated processes

Performing UI actions

Many electronic devices have user interfaces for performing operations such as viewing user interface objects (photos, email messages, etc.) and deleting user interface objects. Some user interface operations, such as deleting a user interface object from a device or a separate server, are destructive, meaning that the operation is difficult or impossible to reverse. For example, in some cases, a user who has accidentally deleted an email message will have a limited amount of time to recover the email message from a trash folder before the email message is permanently deleted. In order to prevent such unintentional operations from occurring, some existing methods cause the electronic device to request confirmation from a user who has entered an input (e.g., into the electronic device) requesting such a destructive operation. One disadvantage of these existing methods is that they are time consuming and require the user to enter additional input to confirm that the operation should be performed.

In the embodiments described below, an improved method for performing user interface operations is provided. A gesture associated with a corresponding operation by a user on a touch-sensitive surface is detected (e.g., swiping over an email message to delete the email message). In response to detecting the gesture, if the gesture has a maximum intensity on the touch-sensitive surface that is above a first intensity threshold, the device performs the operation without requesting confirmation from the user (e.g., a deep press swipe is understood by the device to indicate that the request to perform the operation was truly intentional). On the other hand, if the maximum intensity of the gesture is below the first intensity threshold, the device requests confirmation that the user wants to perform the operation, and thus still prevents unintentional destructive operations from being performed. Some operations, such as scrolling, are performed without requesting confirmation from the user, regardless of the intensity of the gesture associated with the request to perform the operation.

Figures 5A-5I illustrate exemplary user interfaces for performing user interface operations in accordance with some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 6A-6B. Where applicable, Figures 5A-5I include intensity graphs that illustrate the current contact intensity on the touch-sensitive surface relative to multiple intensity thresholds including a first intensity threshold (e.g., "IT _D "). In some embodiments, operations similar to those described below with reference to "IT _D " are performed with reference to different intensity thresholds (e.g., "IT _L ").

In some embodiments, the device is an electronic device having an independent display (e.g., display 450) and an independent touch-sensitive surface (e.g., touch-sensitive surface 451). In some embodiments, the device is portable multifunction device 100, the display is touch-sensitive display system 112, and the touch-sensitive surface includes tactile output generator 167 ( FIG. 1A ) on the display. For ease of explanation, the embodiments described with reference to FIG. 5A-5I and FIG. 6A-6B will be discussed with reference to touch screen 112. However, when the user interface shown in FIG. 5A-5I is displayed on display 450, similar operations are optionally performed on the device having display 450 and independent touch-sensitive surface 451 in response to detecting the contacts described in FIG. 5A-5I on touch-sensitive surface 451. In such embodiments, the focus selector is optionally a cursor or pointer displayed at a location corresponding to (e.g., at or near) the locations of the contacts shown in FIG. 5A-5I (e.g., 1806, 1808, and 1810).

Figure 5A illustrates an exemplary user interface for performing user interface operations on an electronic device 300 having a touch screen display 112. In some embodiments, the device displays one or more user interface objects (e.g., user interface objects 1804-1 through 1804-4 correspond to email conversations 1 through 4, respectively). Specifically, Figures 5A-5I illustrate an example of deleting an email message from a list of email messages, and also illustrate examples of other user interface operations according to some embodiments, such as scrolling through a list of emails.

5B shows the detection of contact 1806. In some embodiments, a specific gesture is associated with a specific corresponding operation. For example, contact 1806 is included in a swipe gesture above user interface object 1804-1. FIG. 5B shows an example of an embodiment in which a swipe gesture above an email message (user interface item 1804-1) is understood as a request to delete or archive the message. In some embodiments, if the gesture begins in the predefined boundaries of the email message, the swipe gesture is considered to be above the email message. In some embodiments, if the main part of the gesture occurs within the predefined boundaries (or within a predefined area in which the representation of the email message is displayed), the swipe gesture is considered to be above the email message.

In the example shown in Figure 5B, contact 1806 has an intensity that is below a corresponding intensity threshold (e.g., IT _D ). In this example, the device detects the gesture and, based on predefined criteria, requests confirmation that the user wants to delete the email message (e.g., if the criteria are not met) or deletes the email message without requesting confirmation from the user (e.g., if the criteria are met). In some embodiments, the predefined criteria include a corresponding intensity (e.g., maximum intensity) of the gesture being above a predefined threshold (e.g., IT _D ). It is contemplated that the intensity of contact 1806 when a gesture is detected over the email message can be defined and measured in any number of ways. For example, in some embodiments, the intensity is measured at the start of the gesture. In some embodiments, the average intensity of the gesture is used. Other metrics for measuring and defining the intensity of a gesture will be apparent to those skilled in the art.

Figure 5C shows a continuation of contact 1806 forming a swipe gesture on user interface object 1804-1. Contact 1806 in both Figures 5B and 5C has an intensity below the deep press intensity threshold. Thus, the device requests confirmation that the user wants to delete the email message (e.g., displaying an affordance with the word "Delete?"). Requesting confirmation for a low-intensity gesture to perform an operation such as a destructive operation (such as deleting a user interface object) enables the device to prevent unintentional operations that are difficult or impossible to reverse. In some embodiments, the user optionally selects the affordance (using a separate contact or a continuation of the same gesture) to notify the device to proceed with the operation. In some embodiments, a separate gesture other than the selection of the affordance cancels the operation. For example, in some embodiments, a second swipe over the email cancels the request to delete the email. In some embodiments, any one of a plurality of gestures other than the gesture including the selection of the affordance cancels the delete operation. In this example, contact 1806 is also below the light press intensity threshold. In this example, no additional functionality is associated with the light press intensity threshold. However, in some other embodiments, additional functionality is associated with the light press input (e.g., the light press intensity threshold is an intensity threshold for identifying a light press input for activating an enable indication such as the delete enable indication in Figure 5B, the edit enable indication in Figure 5B, and/or opening a session in response to detecting a light press input on a user interface object 1804, wherein the light press input includes increasing from an intensity below IT _L to a contact intensity above IT _L ).

Figure 5D is similar to Figure 5B, except that contact 1808 shown in Figure 5D has an intensity greater than the deep press intensity threshold IT _D (e.g., contact 1806 in Figure 5B has an intensity less than IT _D ). Figure 5E shows a continuation of the gesture including contact 1808. Figure 5E is similar to Figure 5C, except that contact 1808 shown in Figure 5E has an intensity greater than the deep press intensity threshold (e.g., contact 1806 in Figure 5C). For the sake of simplicity, other details already described with reference to Figures 5B and 5C will not be repeated here with reference to Figures 5D and 5E, respectively. In some embodiments, because contact 1808 has a maximum intensity greater than IT _D when over the email message, the email message is deleted without requesting confirmation from the user. In this example, the device determines that the gesture was not accidental based on the high intensity of the contact, and therefore continues with the operation (e.g., deletion) without requesting further user input (e.g., confirmation input), thereby saving the user time and improving efficiency. In some embodiments, contact 1808 has an intensity above IT _D throughout the gesture shown in Figures 5D-5E . In some embodiments, contact 1808 has an intensity above IT _D at the beginning of the gesture shown in Figures 5D-5E , and has an intensity below IT _D at the end of the gesture shown in Figures 5D-5E . In some embodiments, contact 1808 has an intensity below IT _D at the beginning of the gesture shown in Figures 5D-5E , and has an intensity above IT _D at the end of the gesture shown in Figures 5D-5E .

In some embodiments, it is helpful to display a dramatic animation that draws the user's attention (e.g., an animation of glass breaking or a representation of an email message being "crumpled") to notify the user that a destructive operation has been performed. In some embodiments, after displaying the animation corresponding to the destructive operation, the user is provided with an option to undo the destructive operation (e.g., displaying an "undo" affordance or a shake-to-undo instruction). FIG5F shows a still frame illustration of such an animation. In this case, the animation displays the word "deleting" in a dramatic manner.

5G shows the removal of the email message from the list after the animation described with reference to FIG5F. In some embodiments, one or more additional email messages that were previously not visible (e.g., the user would have to scroll through the list of email messages to see them) are now visible due to the removal of user interface object 1804-1. For example, user interface object 1804-5 is now visible to the user without scrolling.

In some embodiments, certain inputs associated with certain operations can be performed without requesting confirmation from the user, regardless of the contact intensity corresponding to the input. For example, Figure 5H shows a vertical swipe input including an upward movement of contact 1810. In this example, the vertical swipe input performed using contact 1810 is interpreted by the device as a request to scroll through a list of email messages. Figure 5I shows the resulting changes in the displayed email messages (the list has been scrolled so that user interface objects 1804-5 to 1804-7 are now displayed). The maximum intensity of the vertical swipe gesture performed using contact 1810 is lower than the deep press intensity threshold (and, for that matter, the light press intensity threshold). However, in this example, because the scrolling operation does not require user confirmation, the list is scrolled without requesting user confirmation. In some embodiments, the operation associated with the light press gesture is classified as requiring user confirmation or not requiring user confirmation based on the ease with which the operation can be reversed (e.g., whether it is a destructive operation or a non-destructive operation).

Figures 6A-6B are flow diagrams illustrating method 1900 for performing user interface operations, according to some embodiments. Method 1900 is performed on an electronic device having a display and a touch-sensitive surface (e.g., device 300, Figure 3, or portable multifunction device 100, Figure 1A). In some embodiments, the display is a touchscreen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 1900 may optionally be combined and/or the order of some operations may optionally be changed.

As described below, method 1900 provides an intuitive way to perform user interface operations. The method reduces the cognitive burden on the user when performing user interface operations, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to perform user interface operations faster and more efficiently saves power and increases the time between battery charges.

The device displays (1902) a user interface on a display of the device (e.g., a user interface displayed on touch screen 112 of device 300, FIG. 5A ). The device detects (1904) a user gesture on the touch-sensitive surface, the gesture including a contact (e.g., contact 1806, FIG. 5B ) corresponding to a focus selector at a corresponding location in the user interface. The gesture is associated with a corresponding operation.

In some embodiments, the gesture is a swipe gesture of the contact (1906), the swipe gesture of the contact corresponding to movement of the focus selector over a representation of an item in a list of items (e.g., a message or conversation in an email inbox). In some embodiments, the user interface includes (1908) a plurality of user interface objects, and the corresponding position corresponds to a corresponding object in the plurality of user interface objects. In some embodiments, the corresponding operation is a destructive operation (e.g., a delete operation) (1910).

In response to detecting a gesture, based on determining that the contact has a maximum intensity below a first intensity threshold, the device requests (1912) confirmation that the user wants the device to perform a corresponding action (e.g., to confirm that the user did not accidentally perform the gesture). For example, Figures 5C-5D illustrate an example of a gesture having a maximum intensity below IT _D and a subsequent request for confirmation to delete an email message. In response to the gesture, based on determining that the maximum intensity of the contact is above the first intensity threshold, the device performs the corresponding action without requesting confirmation that the user wants the device to perform the action. For example, Figures 5D-5E illustrate an example of a gesture having a maximum intensity above IT _D and the subsequent deletion of an email message without requesting confirmation. In some embodiments, the first intensity threshold is an intensity threshold that is higher than the input detection intensity threshold at which the contact was initially detected. In some embodiments, the device determines whether to request confirmation or perform the action without requesting confirmation based on whether the gesture satisfies confirmation criteria. In some embodiments, the confirmation criteria is satisfied when the contact (e.g., contact 1808 in Figures 5D-5E) has an intensity that is maintained (e.g., continuously maintained) above IT _D throughout the gesture. In some embodiments, the confirmation criterion is satisfied when the contact (e.g., contact 1808 in Figures 5D-5E ) has an intensity above IT _D at the start of the gesture (or before detecting movement of the contact), regardless of whether the contact has an intensity above IT _D at the end of the gesture. In some embodiments, the confirmation criterion is satisfied when the contact (e.g., contact 1808 in Figures 5D-5E ) has an intensity above IT _D at the end of the gesture (or after ceasing to detect movement of the contact), regardless of whether the contact had an intensity above IT _D at the start of the gesture.

In some embodiments, requesting confirmation that the user wants the device to perform the corresponding operation includes (1914) displaying a confirmation message that includes an option to continue performing the corresponding operation. In some embodiments, the confirmation message includes an explicit option to cancel performance of the corresponding operation. In some embodiments, the confirmation message does not include an explicit option to cancel performance of the corresponding operation, but rather tapping/pressing when the focus selector is outside of the confirmation message cancels performance of the corresponding operation. In some embodiments, the confirmation identifies (1916) the corresponding operation to be performed (e.g., displaying a dialog box asking "Do you want to delete this message? Yes/No" or simply displaying an affordance saying "Delete?" as shown in FIG5C ).

In some embodiments, it is helpful to display a dramatic animation that attracts the user's attention to notify the user that a destructive operation has been performed. In some embodiments, the corresponding operation is (1918) a delete operation, and performing the operation includes displaying an animation that causes the corresponding user interface object to wrinkle (e.g., wrinkle the object like a piece of paper). In some embodiments, the operation is (1920) a delete operation, and performing the operation includes displaying an animation that causes the corresponding user interface object to shatter (e.g., shatter the object like a piece of glass). For example, Figure 5F shows a still frame of an animation of the word "deleting" shattering like glass. In some embodiments, after displaying the animation corresponding to the destructive operation, the user is provided with an option to undo the destructive operation (e.g., display an "undo" affordance or a shake to undo instruction). In some embodiments, the operation is a delete operation, and performing the operation includes displaying an animation that causes the corresponding user interface object to fold up (e.g., fold the corresponding user interface object onto itself and/or fold up multiple user interface objects that include the corresponding user interface object). In some embodiments, the operation is a delete operation, and performing the operation includes displaying an animation of the corresponding user interface object moving back into the display (eg, the corresponding user interface object moves back and gradually fades out).

In some embodiments, the device detects (1922) one or more inputs, the one or more inputs including a contact with a maximum intensity below a first intensity threshold. In response to detecting the one or more inputs, the device performs (1924) one or more corresponding operations (e.g., non-destructive operations such as a scrolling operation or an email viewing operation) without requesting confirmation from the user. For example, a user can view or scroll a list of email messages using one or more contacts with an intensity below the first intensity threshold, as shown in Figures 5H-5I.

It should be understood that the specific order of the operations described in Figures 6A-6B is merely exemplary and is not intended to indicate that the order is the only order in which these operations can be performed. Those skilled in the art will think of various ways to reorder the operations described herein. In addition, it should be noted that the details of the other processes described herein with respect to other methods described herein (e.g., those listed in paragraph [0057]) are also applicable in a similar manner to the method 1900 described above with respect to Figures 6A-6B. For example, the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described above with reference to method 1900 optionally have one or more characteristics of the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described herein with reference to other methods described herein (e.g., those listed in paragraph [0057]). For the sake of brevity, these details are not repeated here.

According to some embodiments, FIG7 shows a functional block diagram of an electronic device 2000 configured according to the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that implements the principles of various described embodiments. Those skilled in the art will appreciate that the functional blocks described in FIG7 can be optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Therefore, the description herein optionally supports any possible combination, separation, or further definition of the functional blocks described herein. As shown in FIG7 , the electronic device 2000 includes a display unit 2002 configured to display a user interface; a touch-sensitive surface unit 2004 configured to detect a user gesture on the touch-sensitive surface unit, the gesture comprising contact corresponding to a focus selector at a corresponding location in the user interface, wherein the gesture is associated with a corresponding operation; one or more sensor units 2006 configured to detect the intensity of contact with the touch-sensitive surface unit 2004; and a processing unit 2008 coupled to the display unit 2002, the touch-sensitive surface unit 2004, and the one or more sensor units 2006. In some embodiments, the processing unit 2008 includes a confirmation request unit 2010 , an operation execution unit 2012 , and an animation unit 2014 .

In some embodiments, the user interface includes a plurality of user interface objects; and the respective positions correspond to respective objects in the plurality of user interface objects.

Processing unit 2008 is configured to: in response to detecting a gesture: request confirmation (e.g., using confirmation request unit 2010) that the user wants the device to perform the corresponding operation based on determining that the contact has a maximum intensity below a first intensity threshold; and perform the corresponding operation (e.g., using operation execution unit 2012) without requesting confirmation that the user wants the device to perform the corresponding operation based on determining that the maximum intensity of the contact is above the first intensity threshold.

In some embodiments, the gesture is a swipe gesture of contact that corresponds to movement of the focus selector over the representation of the item in the list of items.

In some embodiments, requesting confirmation (eg, using confirmation request unit 2010) that the user wants to perform the corresponding operation includes displaying a confirmation message that includes an option to continue performing the corresponding operation.

In some embodiments, the confirmation identifies a corresponding operation to be performed.

In some embodiments, the corresponding operation is a destructive operation.

In some embodiments, processing unit 2008 is further configured to: detect one or more inputs, the one or more inputs comprising a contact having a maximum intensity below a first intensity threshold; and in response to detecting the one or more inputs, perform one or more corresponding operations without requesting confirmation from the user.

In some embodiments, the corresponding operation is a delete operation; and performing the operation includes displaying an animation that wrinkles the corresponding user interface object (eg, using animation unit 2014).

In some embodiments, the operation is a delete operation; and performing the operation includes displaying an animation (eg, utilizing animation unit 2014) that causes the corresponding user interface object to break apart.

The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above with respect to Figures 1A and 3) or an application-specific chip.

The operations described above with reference to Figures 6A-6B may optionally be implemented by the components depicted in Figures 1A-1B or Figure 7. For example, display operation 1902, detection operation 1904, and wrinkle animation operation 1918 may optionally be implemented by event classifier 170, event recognizer 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. The corresponding event recognizer 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as the selection of an object in a user interface. Upon detecting the corresponding predefined event or sub-event, event recognizer 180 activates event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally utilizes or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly know how other processes can be implemented based on the components shown in Figures 1A-1B.

Displaying additional information in response to user contact

Many electronic devices have a graphical user interface that includes a user interface object displayed at a specific location on the display. In some cases, one or more user interface objects are associated with hidden information that is revealed when a user places a cursor or contact over the user interface object for a predefined time period (e.g., a tooltip) or clicks and remains on the user interface object for a predefined time period (e.g., a back button on a browser). These situations require the user to wait for a predefined time period to see the initially hidden information. Providing the user with an option to avoid waiting and obtain the immediate display of additional information would be advantageous. In some cases, a single activation area or user interface object is associated with multiple operations of the device (e.g., a back button on a browser is associated with navigating to a previous web page and displaying a pop-up list of previously visited web pages). In these cases, time delays are sometimes introduced as a way to determine which of multiple operations is to be performed in response to user input associated with the activation area or user interface object. However, such delays interrupt the user's workflow. Providing an intuitive way to immediately perform a specific operation can make the interaction between the user and the device more effective.

The embodiments described below provide new and improved methods for displaying additional information (or activating user interface objects) using a touch-sensitive surface and a sensor that detects contact intensity (e.g., pressure) on the touch-sensitive surface. To avoid waiting for a predefined delay time in order to view additional information associated with a user interface object or area of the user interface, the user has the option of applying a contact intensity (e.g., pressure) greater than a predefined intensity threshold to immediately see the additional information. For example, a user can immediately display a tooltip or pop-up menu by pressing the touch-sensitive surface with an intensity greater than the intensity threshold without having to wait for the delay time. Similarly, for the back button on a browser, a user can immediately display a list of previously visited web pages by pressing the touch-sensitive surface with an intensity greater than the intensity threshold. On the other hand, if the user does not press with enough intensity, the additional information is not displayed until the delay time has elapsed. Thus, the embodiments provide users with a more efficient way to interact with user interface objects.

Figures 8A-8N illustrate exemplary user interfaces for displaying additional information associated with a location on a display based on contact intensity (e.g., pressure) and contact timing (e.g., duration) in accordance with some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 9A-9B. Figures 8A-8N include an intensity graph that illustrates the current intensity of a contact on the touch-sensitive surface relative to multiple intensity thresholds including a corresponding threshold (e.g., “IT _D ”). In some embodiments, operations similar to those described below with reference to IT _D are performed with reference to a different intensity threshold (e.g., “IT _L ”).

Figure 8A shows a user interface (e.g., a web page) displayed on display 450 with multiple selectable objects (e.g., 3606-1 and 3606-2). Figures 8A-8C show that when the contact has an intensity above (e.g., greater than) an intensity threshold, additional information is displayed in response to detecting a contact (e.g., by scrolling a web page) without waiting for a predefined delay time from the time the contact is detected.

In Figure 8A, at a time before T0, no contact is present or detected on touch-sensitive surface 451, and focus selector 3607-a is away from scroll arrow 3606-4 in scroll bar 3606-3. In Figure 8B, at time T0, a contact is detected on touch-sensitive surface 451 (e.g., contact 3605 in Figure 8B). In response to detecting the contact on touch-sensitive surface 451, the device directs display 450 to position the focus selector (e.g., focus selector 3607-a in Figure 8B) at a location on display 450 (e.g., above scroll arrow 3606-4 in scroll bar 3606-3 in Figure 8B). In Figure 8C, after time T1 and before time T2, while focus selector 3607-a is still positioned at a location on display 450 (e.g., over scroll arrow 3606-4 in scroll bar 366-3 in Figure 8C), the intensity of a contact (e.g., contact 3605 in Figure 8C) on touch-sensitive surface 451 exceeds intensity threshold IT _D. When the contact has an intensity above a predefined intensity threshold, additional information (e.g., additional selectable objects 3606-6 and 3606-7) is displayed on display 450 (e.g., by scrolling a web page) without waiting for a predefined delay time to pass (e.g., without waiting until time T2 or without waiting until a predefined delay time, T _delay = T2 - T0, has elapsed). In other words, when focus selector 3607-a is located over scroll arrow 3606-4, if the user places a contact with an intensity above the intensity threshold, the user interface is scrolled even if the delay time has not yet passed. In some embodiments, the portion of the user interface that controls scrolling of the user interface includes a displayed scrolling user interface (e.g., as shown in FIG8A ). In some embodiments, the portion of the user interface that controls scrolling of the user interface is a hidden hit area that is not displayed but exists at the edge of the scrollable window of the application. For example, if the user moves the cursor 3607 to the bottom edge of the display while moving a corresponding icon or other user interface object or while selecting text, the device scrolls the user interface in response to detecting that the cursor remains in the hidden hit area for longer than a delay time or in response to detecting that the intensity of the contact increases to above a predefined intensity threshold.

Figures 8D-8G show that when a contact has an intensity below (e.g., less than) an intensity threshold (e.g., “IT _D ”), waiting until a predefined delay time has elapsed from the time the contact is detected on touch-sensitive surface 451 (or from the time the contact reaches a lower intensity threshold such as IT _L ) to display additional information in response to the detected contact (e.g., by scrolling a web page).

In Figure 8D, at a time before T0, there is no contact or no contact detected on touch-sensitive surface 451 and focus selector 3607-a is away from scroll arrow 3606-4 in scroll bar 3606-3. In Figure 8E, at time T0, a contact is detected on touch-sensitive surface 451 (e.g., contact 3608 in Figure 8E). In response to detecting the contact on touch-sensitive surface 451, the device directs display 450 to position the focus selector (e.g., focus selector 3607-b in Figure 8E) at a corresponding location on display 450 (e.g., at edge 3606-4 of scroll bar 3606-3 in Figure 8E). In Figure 8F , at a time before time T2, while focus selector 3607-b is still positioned at a corresponding location on display 450 (e.g., over scroll arrow 3606-4 in scroll bar 3606-3 in Figure 8F ), the intensity of the contact on touch-sensitive surface 451 (e.g., contact 3608 in Figure 8F ) is below intensity threshold IT _D . When contact 3608 has an intensity below the intensity threshold and while focus selector 3607-b is still positioned at the corresponding location on display 450, the device does not display additional information associated with the corresponding location of focus selector 3607-b until a predefined delay time has elapsed (e.g., predefined delay time T _delay = T2 - T0) or until time T2. In Figure 8G, after a predefined delay time ( _Tdelay = T2-T0) has elapsed or after time T2, when the intensity of the contact on touch-sensitive surface 451 (e.g., contact 3608 in Figure 8G) is still below intensity threshold IT _D , and when focus selector 3607-b is still positioned at a location on display 450 (e.g., above scroll arrow 3606-4 in scroll bar 3606-3 in Figure 8G), additional information (e.g., additional selectable objects 3606-6 and 3606-7) is displayed on display 450 (e.g., by scrolling a web page). For example, if the user places a contact with an intensity below the intensity threshold above the scroll arrow in the scroll area and waits for more than the delay time, the user interface is scrolled accordingly. In some embodiments, the delay time _Tdelay = T2-T1, and the contact reaches a lower intensity threshold (e.g., "IT _L ") at time T1, and the additional information is displayed on the display after waiting until the intensity of the contact has been maintained above intensity threshold IT _L from time T1 until time T2.

In other words, in some embodiments, the device determines whether the contact intensity is above an intensity threshold when the focus selector is at the corresponding position on the display. If the contact intensity is above the intensity threshold, the device does not wait for a predefined delay time to display the additional information on the display, but instead displays the additional information immediately. On the other hand, if the contact intensity is below the intensity threshold, the device waits until a predefined delay time to display the additional information on the display, and thus displays the additional information at a later time (e.g., determined according to the predefined delay time). Therefore, if the user increases the intensity of the contact to above the intensity threshold, the user can display additional (e.g., initially not displayed) information associated with the position corresponding to the contact on the display earlier without having to wait for the predefined delay time.

In some embodiments, when the focus selector is displayed at the corresponding position, the device determines whether a predefined delay time has passed since the contact was detected. If the predefined delay time has elapsed since the contact was detected, the device displays additional information on display 450. On the other hand, if the predefined delay time has not elapsed, the device determines (e.g., repeatedly, repeatedly, or multiple times) whether the contact intensity is above a predefined threshold. If the device determines that the contact intensity is above the predefined threshold, the device displays the additional information without waiting until the predefined delay time has elapsed.

Figure 8H shows a user interface (e.g., a web page) displayed on touch screen 112. Figures 8H-8J also show that when the contact has an intensity (e.g., pressure) above (e.g., greater than) an intensity threshold (e.g., IT _D ), additional information (e.g., a browsing history, including a list of previously visited web pages) is displayed in response to a contact being directly detected on touch screen 112 without waiting for a predefined delay time from the time the contact is detected.

In FIG8H , at a time before T0, no contact is present or detected on touch screen 112 at a location corresponding to browser back button 3610 (however, a contact is optionally present at another location on touch screen 112). In FIG8I , at time T0, a contact (e.g., contact 3609 in FIG8I ) is detected on touch screen 112 at a corresponding location (e.g., at the location of back button 3610 in FIG8I ). In some embodiments, when the device has a touch screen display and a contact is detected on the touch screen, the location of the focus selector coincides with the location of the contact on the touch screen. In FIG8J , after time T1 and before time T2, while contact 3609 is still positioned at a corresponding location on touch screen 112 (e.g., at the location of back button 3610 in FIG8J ), the intensity of the contact on touch screen 112 (e.g., contact 3609 in FIG8J ) exceeds intensity threshold IT _D . When the contact has an intensity above a predefined intensity threshold, additional information (e.g., browsing history, including a list 3611 of previously visited web pages) is displayed on touch screen 112 without waiting for a predefined delay time to elapse (e.g., without waiting until time T2 or without waiting until a predefined delay time _Tdelay =T2-T0 has elapsed). In other words, if the user places a contact on the browser back button with an intensity above intensity threshold IT _D , the browser history is displayed even if the delay time has not yet elapsed.

Figures 8K-8N show that when a contact has an intensity below (e.g., less than) an intensity threshold IT _D , waiting until a predefined delay time has elapsed from the time the contact is detected at the corresponding location on the touch screen 112 to display additional information corresponding to the detected contact (e.g., a browsing history, including a list 3611 of previously visited web pages, sometimes referred to as a browser history).

In FIG8K , at a time before T0, no contact is present or detected on touch screen 112 at a location corresponding to browser back button 3610 (however, a contact is optionally present at another location on touch screen 112). In FIG8L , at time T0, a contact is detected on touch screen 112 at a location corresponding to browser back button 3610 (e.g., contact 3613 in FIG8L ). In some embodiments, the location of the contact on touch screen 112 coincides with the location of a focus selector on touch screen 112 (e.g., at the location of back button 3610 in FIG8L ). In FIG8M , at a time before time T2, while the contact (or focus selector) is still positioned at a corresponding location on touch screen 112 (e.g., at the location of back button 3610 in FIG8M ), the intensity of the contact on touch screen 112 (e.g., contact 3613 in FIG8L ) is below intensity threshold IT _D . When the contact has an intensity below the intensity threshold and the contact (or focus selector) is still positioned at the corresponding position on the touch screen 112, the device does not display the additional information until a predefined delay time (e.g., a predefined delay time _Tdelay = T2-T0) has elapsed or until time T2. In FIG8N , after the predefined delay time ( _Tdelay = T2-T0) has passed or after time T2, when the contact (or the focus selector corresponding to the contact) is still positioned at the corresponding position on the touch screen 112 (e.g., at the position of the back button 3610 in FIG8N ), additional information (e.g., a browsing history, including a list 3611 of previously visited web pages) is displayed on the touch screen 112. In other words, if the user places a contact with an intensity below the intensity threshold IT _D on the browser back button and waits for more than the delay time, the browser history is displayed.

Figures 9A-9B are flowcharts illustrating method 3700 for displaying additional information associated with a location on a display based on contact intensity (e.g., pressure) and contact timing (e.g., duration) in accordance with some embodiments. Method 3700 is performed at an electronic device having a display and a touch-sensitive surface (e.g., device 300, Figure 3, or portable multifunction device 100, Figure 1A). In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. The device includes one or more sensors for detecting the intensity of contact with the touch-sensitive surface. Some operations in method 3700 are optionally combined, and/or the order of some operations is optionally changed.

As described below, method 3700 provides an intuitive way to display additional (e.g., not initially displayed) information associated with a location on a display based on contact strength (e.g., pressure) and contact timing (e.g., duration). The method reduces the cognitive burden on the user when displaying additional (e.g., not initially displayed) information by enabling the user to skip the delay time and immediately access the additional information, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to display additional (e.g., not initially displayed) information faster and more efficiently saves power and increases the time between battery charges.

When the focus selector corresponding to the contact is at a respective location on the display that is associated with additional information that was not initially displayed on the display (e.g., the additional information was not immediately displayed prior to detecting movement of the focus selector to the respective location on the display or detecting contact on the touch screen at the respective location), the device detects (3702) the contact on the touch-sensitive surface. In some embodiments, the additional information is "associated with the respective location" because the additional information is displayed on the display when a predefined gesture (e.g., a finger press input) is performed at the respective location.

In some embodiments, the device detects a gesture that includes movement of a contact on the touch-sensitive surface, the movement corresponding to movement of the focus selector to a corresponding location on the display (e.g., movement of a cursor on the display in response to movement of the contact on the touchpad). In such embodiments, additional information is not immediately displayed before detecting movement of the focus selector to the corresponding location on the display. For example, as explained with reference to Figures 8A-8G, the device detects contact 3605 on touch-sensitive surface 451 that is separate from display 450. In this example, a focus selector (e.g., cursor 3607 in Figures 8A-8G) corresponding to contact 3608 on touch-sensitive surface 451 is positioned at a corresponding location on display 450 (e.g., above scroll arrow 3606-4 in scroll bar 3606-3 in Figure 8B) and moves in accordance with the movement of contact 3605. Additional information associated with the position of focus selector 3607 - a (eg, additional selectable objects 3606 - 6 and 3606 - 7 displayed on display 450 , FIG. 8C ) is not initially displayed on display 450 .

In some embodiments, the contact is a finger contact. In some embodiments, additional information is not immediately displayed before a contact on the touch screen at the corresponding location is detected. In some embodiments, the device detects a contact on a portion of the touch-sensitive surface corresponding to the corresponding location (e.g., the device detects a contact at the location of a button on the touch screen display associated with the additional information). For example, as explained with reference to Figures 8I-8N, the device detects contact 3609 on touch screen 112 at a corresponding location (e.g., at the location of back button 3610 in Figure 8I). In some embodiments, as shown in Figure 8I, when the device has a touch screen display (e.g., touch screen 112) and a contact is detected on the touch screen (e.g., contact 3609, Figure 8I), the position of the focus selector coincides with the position of the contact. Additional information associated with the position of the focus selector (e.g., a browsing history including a list 3611 of previously visited web pages, Figure 8J) is not initially displayed on touch screen 112.

In some embodiments, the device displays (3704) a user interface object on the display. In some embodiments, the user interface object (3706) is associated with additional information that is not initially displayed on the display; and the corresponding position corresponds to the position of the user interface object on the display (e.g., the additional information is associated with an icon or other displayed element of the user interface). In some embodiments, the corresponding position is not associated with a specific user interface object on the display (e.g., the device detects a contact or cursor in an activation area or a hidden hit area of the touch-sensitive surface and/or display that triggers display of the additional information when activated by waiting a delay time or increasing the intensity of the contact above an intensity threshold).

In some embodiments, the additional information includes (3708) help for the user interface object (e.g., information describing the user interface object and one or more options, such as a hyperlink or a search box, for retrieving further help related to the user interface object). In some embodiments, the additional information includes (3710) a description of one or more functions of the user interface object. In some embodiments, the additional information includes a brief description of the user interface object and/or a description of the functions associated with the user interface object. For example, if the user interface object is a "create bookmark" icon in a browser, the additional information optionally reads: "This creates a new bookmark; bookmarks are a way to save your favorite sites."

In some embodiments, the additional information includes (3712) a menu with selectable options related to the user interface object. For example, if the user interface object is a file icon, the additional information includes a menu with selectable options such as "Open," "Edit," "Print," "Copy," "Delete," "Rename," etc.

In some embodiments, the user interface object is (3714) a back button. For example, the user interface object is a back button for a web browser or file browser, such as back button 3610 shown in Figures 8H-8N. In some embodiments (e.g., where the user interface object is a back button), the additional information includes a browsing history that includes a list of representations of previously visited information. For example, as shown in Figures 8J and 8N, the additional information includes a browsing history that includes a list 3611 of previously visited web pages. In this example, when the focus selector is over the back button, in response to detecting that the intensity of the contact is above a corresponding intensity threshold (e.g., "IT _D ") or detecting that the contact persists for more than a predefined delay time, the device displays a list of previously visited websites or folders. In some embodiments, as described with reference to Figure 8J, in response to detecting that the intensity of contact 3609 on touch screen 112 is above a corresponding intensity threshold, the device displays a list 3611 of previously visited websites or folders on touch screen 112. In some embodiments, as described with reference to FIGURE 8N , when the focus selector (coinciding with contact 3613 on touch screen 112) is located above the back button 3610, in response to detecting that the contact 3613 persists for more than a predefined delay time ( _Tdelay = T2-T0), the device displays a list 3611 of previously visited websites or folders.

In some embodiments, the corresponding position corresponds to (3718) an edge of a user interface for navigating an electronic document. In some embodiments, the edge of the user interface for navigating an electronic document is a scroll bar. For example, as shown in Figures 8A-8E, the corresponding position corresponds to a scroll arrow 3606-4 in a scroll bar 3606-3 for navigating an electronic document, where the scroll bar 3606-3 is located at the bottom edge of the user interface shown on the display 450. In some embodiments, the edge of the user interface for navigating an electronic document is a portion of the electronic document that is adjacent to an edge of an application window displaying the electronic document. For example, in some cases, a user selecting a portion of an electronic document (e.g., by dragging a cursor or touch over text in a word processing document or on a web page) will reach the edge of the electronic document on the display. In this example, if the device would otherwise wait for a delay period (e.g., 0.1, 0.2, 0.3, 0.4, or 0.5 seconds) to elapse before scrolling the electronic document, the device will instead begin scrolling the electronic document in response to an increase in intensity above a corresponding intensity threshold (e.g., IT _D ) before the delay period has elapsed.

In some embodiments (e.g., where the corresponding position corresponds to a hidden hit zone at an edge of a user interface for navigating within an electronic document), the additional information includes a portion of the electronic document that was not displayed immediately before the contact was detected when the focus selector was at the corresponding position. For example, when the focus selector is at or near an edge of an application window while the user is selecting an object (e.g., a file icon in a file manager application) or content (e.g., text in a word processing document) in the application window or moving a user interface object (e.g., a file icon in a file manager application) within the application window, the application window is scrolled when the focus selector pauses at the edge of the user interface for more than a predefined delay time or if a press input above a corresponding intensity threshold is detected while the focus selector is at or near an edge of the application window. Scrolling the application window in this manner enables the user to keep moving the user interface object or, alternatively, enables the user to keep selecting an object or content without interruption (e.g., without dropping a currently selected user interface object or without deselecting previously selected content).

For example, when the focus selector is located at an edge of a user interface for navigating an electronic document, in response to detecting a contact intensity greater than a corresponding intensity threshold or detecting a contact duration exceeding a predefined delay time, the device scrolls the electronic document. Specifically, as explained with reference to FIG8C , when focus selector 3607-a is located over scroll arrow 3606-4 in scroll bar 3606-3, in response to detecting an intensity of contact 3605 greater than a corresponding intensity threshold IT _D , where scroll bar 3606-3 is located at the bottom edge of the user interface shown on display 450, the device scrolls the electronic document to display additional information (e.g., additional selectable objects 3606-6 and 3606-7). Similarly, as explained with reference to FIG8G , when focus selector 3607-b is located over scroll arrow 3606-4 in scroll bar 3606-3, in response to detecting a contact 3608 duration exceeding a predefined delay time (e.g., exceeding a predefined delay time T _delay = T2 - T0), the device scrolls the electronic document.

For another example, when the focus selector is adjacent to a scroll bar or near an edge of an application window in which an electronic document is displayed, the device scrolls the document when the focus selector is paused in that position for more than a predefined delay time or if a press input above a corresponding intensity threshold is detected. In some embodiments, when the press input is detected, the electronic document is scrolled based on determining that a user interface object is currently selected and is moving in accordance with movement of the focus selector. For example, if a user is dragging a representation of a file or program (e.g., a file or program icon) in a window of a file manager application (e.g., a system file manager such as the Finder from Apple Inc. (Cupertino, California), or a media manager such as iPhoto from Apple Inc. (Cupertino, California) or iTunes from Apple Inc. (Cupertino, California)), and the user drags the representation of the file or program to the edge of the file manager application's window and pauses for more than a predefined delay time while at the edge of the virtual desktop or increases the intensity of the contact to above a corresponding intensity threshold, the device scrolls the window of the file manager application.

In some embodiments, the corresponding position corresponds to (3720) an edge of the display. For example, the corresponding position is located on an area or strip within a predefined distance (e.g., 0.1, 0.5, or 1.0 mm, or 5, 10, or other suitable number of pixels) of the edge of the display. In some embodiments (e.g., where the corresponding position corresponds to an edge of the display), the additional information includes a virtual desktop that was not immediately displayed before the contact was detected when the focus selector was at the corresponding position (e.g., switching between virtual desktops in response to detecting that the contact intensity exceeds a corresponding intensity threshold or detecting that the contact continues for more than a predefined delay time when the focus selector is at the edge of the display). For example, if a user is dragging a window on a workspace or virtual desktop of an operating system with multiple virtual desktops, and the user drags the window of an application to the edge of the virtual desktop and pauses when at the edge of the virtual desktop or increases the intensity of the contact to exceed a corresponding intensity threshold, the device will optionally switch from the current/first virtual desktop to a different/second virtual desktop associated with the corresponding edge of the current virtual desktop. In this example, the edge of the display is associated with the virtual desktop because pressing the input when the focus selector is adjacent to the edge of the display causes a different virtual desktop to be displayed. In some embodiments, when a press input is detected, a different virtual desktop is displayed based on determining that a user interface object is currently selected and is being moved based on movement of a focus selector. In other words, a different virtual desktop is displayed based on determining that a user has performed a gesture associated with moving a user interface object from a current virtual desktop to a different virtual desktop.

In some embodiments, the corresponding location corresponds to (3722) a hyperlink (e.g., a hyperlink in a web page), and the additional information includes content associated with the hyperlink. For example, in response to detecting a contact intensity exceeding a corresponding intensity threshold or detecting a contact duration exceeding a predefined delay time on a hyperlink in a web page displayed in a web browser, the device loads the web page associated with the hyperlink in a new application window or a current application window of the web browser (e.g., immediately after detecting an increase in contact intensity or after detecting a lift-off of the contact after detecting an increase in contact intensity).

When the focus selector is located (3724) at the corresponding location: based on determining (3726) that the contact had an intensity above the corresponding intensity threshold (e.g., "IT _D " in Figures 8A-8N) before a predefined delay time has elapsed while the focus selector is located at the corresponding location, the device displays additional information associated with the corresponding location without waiting until the predefined delay time has elapsed. For example, as explained with reference to Figure 8C, when the focus selector (e.g., cursor 3607-a) is located at the corresponding location (e.g., at scroll arrow 3606-4 in scroll bar 3606-3), based on determining that the contact 3605 had an intensity above the corresponding intensity threshold before a predefined delay time (e.g., T _delay = T2-T0) has elapsed while the focus selector 3607-a is located at the corresponding location, the device displays additional information associated with the corresponding location (e.g., additional selectable objects 3606-6 and 3606-7) without waiting until the predefined delay time has elapsed (e.g., waiting until time T2).

When the focus selector is at (3724) the corresponding position: based on determining (3730) that the contact has an intensity below the corresponding intensity threshold, when the focus selector is at the corresponding position, the device waits until a predefined delay time has elapsed to display additional information associated with the corresponding position. For example, as explained with reference to Figures 8F-8G, when the focus selector (e.g., cursor 3607-b) is at the corresponding position (e.g., scroll arrow 3606-4 in scroll bar 3606-3), based on determining that contact 3608 has an intensity below the corresponding intensity threshold, when the focus selector (e.g., cursor 3607-b) is at the corresponding position, the device waits until a predefined delay time has elapsed (e.g., _Tdelay = T2-T0) to display additional information associated with the corresponding position (e.g., additionally selectable objects 3606-6 and 3606-7). For example, the device waits until time T2 to display the additional information associated with the corresponding position.

In some embodiments, when the focus selector is at the corresponding location, the device detects (3734) a tap gesture on the touch-sensitive surface. For example, the device detects a contact on the touch-sensitive surface that has a duration less than a predefined maximum tap gesture duration and/or a maximum intensity between a minimum contact detection intensity (e.g., IT ₀ ) and a corresponding intensity threshold IT _D (e.g., a brief regular press gesture that is not a “deep press” input). In some embodiments, in response to detecting the tap gesture, the device performs (3736) an operation associated with the user interface object (e.g., without displaying additional information associated with the corresponding location). In other words, in some embodiments, a regular press input (e.g., a non-deep press interaction) corresponding to the user interface object performs an operation that is different from a deep press input, which immediately displays additional information associated with the corresponding location (e.g., a non-deep press interaction causes navigating “back” in the browser by loading the most recently visited web page in the browser rather than displaying a list of previously visited websites).

It should be understood that the particular order in which the operations described in Figures 9A-9B are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art would appreciate various ways to reorder the operations described herein. In addition, it should be noted that the details of the other processes described herein with respect to other methods described herein (e.g., those listed in paragraph [0057]) also apply in a similar manner to method 3700 described above with respect to Figures 9A-9B. For example, the contacts, gestures, user interface objects, intensity thresholds, and focus selectors described above with reference to method 3700 optionally have one or more characteristics of the contacts, gestures, user interface objects, intensity thresholds, and focus selectors described herein with reference to other methods described herein (e.g., those listed in paragraph [0057]). For the sake of brevity, these details are not repeated here.

According to some embodiments, Figure 10 shows a functional block diagram of an electronic device 3800 configured according to various described embodiment principles. The functional block of this equipment is optionally realized by hardware, software, or a combination of hardware and software that performs the principles of various described embodiments. Those skilled in the art will appreciate that the functional block described in Figure 10 is optionally combined or separated into sub-blocks to realize the principles of various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further qualification of functional block described herein.

10 , electronic device 3800 includes a display unit 3802 configured to display additional information associated with a corresponding location, a touch-sensitive surface unit 3804 configured to receive a contact on the touch-sensitive surface unit, one or more sensor units 3805 configured to detect an intensity of contact with touch-sensitive surface unit 3804, and a processing unit 3806 coupled to display unit 3802, touch-sensitive surface unit 3804, and sensor unit 3805. In some embodiments, processing unit 3806 includes a detection unit 3808, a waiting unit 3810, and a display enabling unit 3814.

The processing unit 3806 is configured to: detect a contact on the touch-sensitive surface unit 3804 (for example, using the detection unit 3808) when the focus selector corresponding to the contact is at a corresponding position on the display unit 3802 associated with additional information that was not initially displayed on the display unit 3802; and when the focus selector is at the corresponding position: based on determining that the contact has an intensity higher than a corresponding intensity threshold before a predefined delay time has elapsed when the focus selector is at the corresponding position, enable the display of additional information associated with the corresponding position (for example, using the display enabling unit 3814) without waiting until the predefined delay time has elapsed; and based on determining that the contact has an intensity lower than the corresponding intensity threshold, wait until the predefined delay time has elapsed when the focus selector is at the corresponding position (for example, using the waiting unit 3810) to enable the display of additional information associated with the corresponding position (for example, using the display enabling unit 3814).

In some embodiments, processing unit 3806 is further configured to enable display of a user interface object on display unit 3802 (e.g., utilizing display enabling unit 3814), wherein the user interface object is associated with additional information that is not initially displayed on display unit 3802; and the corresponding position corresponds to the position of the user interface object on display unit 3802.

In some embodiments, the additional information is information selected from the group consisting of: help for the user interface object, a description of one or more functions of the user interface object, and a menu with selectable options related to the user interface object.

In some embodiments, processing unit 3806 is further configured to detect a tap gesture on the touch-sensitive surface unit (e.g., using detection unit 3808) when the focus selector is at the corresponding position; and in response to detecting the tap gesture, perform an operation associated with the user interface object.

In some embodiments, the user interface object is a back button; and the additional information includes a browsing history including a list of representations of previously accessed information.

In some embodiments, the corresponding location corresponds to an edge of a user interface for navigating in the electronic document; and the additional information includes a portion of the electronic document that was not displayed immediately before the contact was detected when the focus selector was at the corresponding location.

In some embodiments, the corresponding location corresponds to an edge of the display unit 3802; and the additional information includes a virtual desktop that was not displayed immediately before the contact was detected when the focus selector was at the corresponding location.

In some embodiments, the corresponding location corresponds to a hyperlink; and the additional information includes content associated with the hyperlink.

The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above with respect to Figures 1A and 3) or an application-specific chip.

The operations described above with reference to Figures 9A-9B are optionally implemented by the components depicted in Figures 1A-1B or Figure 10. For example, detection operation 3702, display operations 3704 and 3726, and wait operation 3730 are optionally implemented by event classifier 170, event recognizer 180, and event handler 190. Event monitor 171 in event classifier 170 detects a contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. The corresponding event recognizer 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at the first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as the selection of an object on a user interface. When a corresponding predefined event or sub-event is detected, event recognizer 180 activates event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally utilizes or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, event handler 190 accesses corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly understand how other processes can be implemented based on the components shown in Figures 1A-1B.

Remove an icon from a predefined area

Many electronic devices have a graphical user interface that includes an application taskbar or launcher bar. The application taskbar or launcher bar includes application icons that users can activate to launch applications. Users can add application icons corresponding to their favorite or frequently used applications to the application taskbar for quick access and remove application icons from the application taskbar. Users can also rearrange application icons within the application taskbar. Some methods require multiple inputs to remove application icons from the application taskbar, or sometimes result in icons being accidentally removed from the application taskbar. The embodiments described below improve on these methods by allowing users to perform gestures on a touch-sensitive surface and move application icons based on the contact intensity in the gesture for rearrangement or removal from the application taskbar based on intensity. When the user performs the gesture with a heavy touch, the user can remove the application icon from the application taskbar. When the user performs the gesture with a light touch, the application icon remains in the application taskbar. This provides users with a more convenient way to remove application icons from the application taskbar and reduces the likelihood of accidentally removing icons from the application taskbar.

Figures 11A-11U illustrate exemplary user interfaces for removing an icon from a predefined area, in accordance with some embodiments. The user interfaces in these figures are used to illustrate the process described below, including the process in Figures 12A-12B. Figures 11A-11U include an intensity graph that illustrates the current intensity of a contact on the touch-sensitive surface relative to multiple intensity thresholds, including a light press intensity threshold (e.g., "IT _L ") and a deep press intensity threshold (e.g., "IT _D "). In some embodiments, operations similar to those described below with reference to IT _D are performed with reference to a different intensity threshold (e.g., "IT _L ").

FIG11A shows a user interface 3904 displayed on a display 450 of a device (e.g., device 300). Displayed in the user interface 3904 is an application taskbar or tray area 3916. Optionally, one or more icons are displayed in the application taskbar area 3916. For example, in FIG11A , application icons 3906, 3908, 3910, 3912, and 3914 are displayed in the application taskbar 3912. Each application icon 3906, 3908, 3910, 3912, and 3914 is associated with a corresponding application. For example, icon 3908 is associated with a media player or audio application. In some embodiments, the application taskbar or tray area 3916 is a predefined area displayed on the display 450, in which one or more application icons (such as icons 3906, 3908, 3910, 3912, and 3914) are optionally placed for quick access.

Also displayed on display 450 is a cursor 3922. Cursor 3922 is an example of a focus selector. The user optionally moves cursor 3922 on display 450 (e.g., using touch-sensitive surface 451 of the device) to point to a user interface object (e.g., icons 3906, 3908, 3910, 3912, or 3914) and utilize additional input to select and manipulate the object.

Figures 11B-11C illustrate a gesture detected on touch-sensitive surface 451 of the device when cursor 3922 is over icon 3908. The gesture is performed using contact 3920. The gesture includes movement 3924 of contact 3920 across touch-sensitive surface 451, as shown in Figure 11C. Movement 3924 corresponds to movement of cursor 3922 outside of application taskbar area 3916; cursor 3922 moves outside of application taskbar area 3916 in accordance with movement 3924.

The intensity of contact 3920 is detected by one or more sensors in the device. The intensity is detected throughout the duration of the gesture. In response to detecting a gesture including contact 3920, and based on a determination by the device that contact 3920 had a maximum intensity below a deep press intensity threshold (e.g., "IT _D ") during the gesture, an indication that the gesture is being performed is displayed during the gesture, and icon 3908 (the icon over which cursor 3922 was positioned when the gesture was detected) remains in application taskbar area 3916 after the gesture ends. The indication is optionally, for example, a display of movement of icon 3908 based on movement of cursor 3922, as shown in FIG11C . After detecting the end of the gesture (e.g., detecting liftoff of contact 3920 or detecting a decrease in the intensity of contact 3920 from an intensity above IT _L to an intensity below IT _L ), icon 3908 remains in application taskbar area 3916, as shown in FIG11D .

In some embodiments, displaying an indication that a gesture is being performed includes changing the appearance of icon 3908 to, for example, display a ghost image 3926 of icon 3908, as shown in Figures 11E-11F. While ghost image 3926 is displayed, a representation of icon 3908 is displayed as moving in accordance with movement 3924 of contact 3920. The representation of icon 3908 that moves in accordance with movement 3924 is optionally ghost image 3926 (as shown in Figure 11F) or a duplicate icon 3928 of icon 3908 (as shown in Figure 11G). After the gesture ends, icon 3908 is again displayed in application taskbar area 3916, as shown in Figure 11D.

In some embodiments, showing an indication that the gesture is being performed includes showing movement of cursor 3922 while leaving icon 3908 in place in application taskbar area 3916. For example, Figure 11H shows cursor 3922 moving outside of application taskbar area 3916 in accordance with movement 3924 of contact 3920, while icon 3908 remains fixed in its position in application taskbar area 3916.

In some embodiments, displaying an indication that a gesture is being performed includes displaying movement of icon 3908 during the gesture, but constraining the movement of icon 3908 to application taskbar area 3916, as shown in FIG11I . In other words, icon 3908 does not move completely outside of application taskbar area 3916 (or, optionally, does not move outside of application taskbar area 3916 at all). After the gesture ends, icon 3908 is again displayed in its original position in application taskbar area 3916, as shown in FIG11D . In some embodiments, icon 3908 moves to the left or right within application taskbar area 3916.

Figure 11J shows user interface 3904, icons 3906, 3908, 3910, 3912, and 3914, application taskbar area 3916, and cursor 3922 displayed on display 450 of the device, as shown in Figure 11A. A gesture performed with contact 3930 and movement 3934 of contact 3930 are detected on touch-sensitive surface 451. In Figure 11J, the device detects that contact 3930 reaches an intensity above a deep press intensity threshold (e.g., "IT _D ") during the gesture.

In response to detecting a gesture including contact 3930, and in accordance with a determination by the device that contact 3930 reached an intensity above a deep press intensity threshold (e.g., “IT _D ”) during the gesture, icon 3908 (the icon over which cursor 3922 was positioned when the gesture was detected) moves in accordance with movement 3934 of contact 3930, as shown in FIG11K , and icon 3908 is removed from application taskbar area 3916 after the device detects the end of the gesture (e.g., detecting liftoff of contact 3930 or detecting a decrease in the intensity of contact 3930 below IT _L ), as shown in FIG11M . Icon 3908 removed from application taskbar area 3916 is optionally displayed outside of application taskbar area 3916 or deleted. In some embodiments, when icon 3908 is moved, an indication that icon 3908 will be removed from application taskbar area 3916 is displayed before the actual removal. This indication is optionally that icon 3936 replaces icon 3908 in application taskbar area 3916, as shown in FIG11L .

Figure 11N shows a gesture performed with contact 3938 detected on touch-sensitive surface 451 when cursor 3922 is over icon 3908. The gesture performed with contact 3938 is, optionally, a tap gesture or a press input (or stationary press input) that includes an increase in the intensity of the contact above a light press intensity threshold (e.g., "IT _L ") and a subsequent decrease. In response to detecting the gesture performed with contact 3938 in Figure 11N, the application corresponding to icon 3908 (Figure 11N) is launched, as shown in Figure 11O.

11P-11U illustrate examples of the user interfaces described above with reference to FIG. 11A-11O implemented on touch screen 112 of a device (e.g., device 100). FIG. 11P illustrates a user interface 3944 displayed on touch-sensitive display 112 of the device. Displayed in user interface 3944 is an application taskbar or tray area 3916. Optionally, one or more icons are displayed in application taskbar area 3916. For example, in FIG. 11P , application icons 3906, 3908, 3910, 3912, and 3914 are displayed in application taskbar 3916.

11P-11Q illustrate a gesture detected on the device's touch-sensitive display 112. The gesture is performed using contact 3946 detected on icon 3908. The gesture includes movement 3948 of contact 3946 across touch-sensitive display 112, as shown in FIG11Q. Movement 3948 includes movement outside of application taskbar area 3916.

The intensity of contact 3946 is detected by one or more sensors in the device. The intensity is detected throughout the duration of the gesture. In response to detecting a gesture including contact 3946, and based on a determination by the device that contact 3946 had a maximum intensity below a deep press intensity threshold (e.g., "IT _D ") during the gesture, an indication that a gesture is being performed is displayed during the gesture, and icon 3908 (the icon on which contact 3946 was detected when the gesture was detected) remains in application taskbar area 3916 after the gesture ends. The indication is optionally, for example, display of icon 3908 in accordance with movement 3948, as shown in FIG11Q . After the gesture ends (e.g., lift-off of contact 3946 is detected or the intensity of contact 3946 is detected to decrease from an intensity above IT _L to an intensity below IT _L ), icon 3908 remains in application taskbar area 3916, as shown in FIG11R .

11S-11T illustrate a gesture detected on the device's touch-sensitive display 112. The gesture is performed using contact 3950 detected on icon 3908. The gesture includes movement 3952 of contact 3950 across touch-sensitive display 112, as shown in FIG11T. Movement 3952 includes movement outside of application taskbar area 3916.

The intensity of contact 3950 is detected by one or more sensors in the device. The intensity is detected throughout the duration of the gesture. In response to detecting the gesture including contact 3950, and based on a determination by the device that contact 3950 reaches an intensity above a deep press intensity threshold (e.g., “IT _D ”) during the gesture, icon 3908 (the icon over which contact 3950 was positioned when the gesture was detected) moves in accordance with movement 3952 of contact 3950, as shown in FIG11T , and icon 3908 is removed from application taskbar area 3916 after the gesture ends, as shown in FIG11U .

Figures 12A-12B are flow diagrams illustrating method 4000 for removing an icon from a predefined area, according to some embodiments. Method 4000 is performed on an electronic device having a display and a touch-sensitive surface (e.g., device 300, Figure 3, or portable multifunction device 100, Figure 1A). In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 4000 may optionally be combined, and/or the order of some operations may optionally be changed.

As described below, method 4000 provides an intuitive way to remove icons from a predefined area. This method reduces the cognitive burden on the user when removing icons from a predefined area, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to remove icons from a predefined area more quickly and efficiently saves power and increases the time between battery charges.

The device displays a plurality of icons in a predefined area (e.g., an application taskbar or a quick launch tray) on the display, wherein the plurality of icons includes (4002) corresponding icons. For example, in FIG. 11A or FIG. 11P , icons 3906, 3908, 3910, 3912, and 3914 are displayed in the application taskbar area 3916.

The device detects a gesture comprising (4004) a contact (e.g., a finger contact) on the touch-sensitive surface while the focus selector is over a corresponding icon and subsequent movement of the contact on the touch-sensitive surface, the subsequent movement corresponding to movement of the focus selector outside of a predefined area. For example, Figures 11B-11C illustrate a gesture comprising contact 3920 and movement 3924 of contact 3920 detected on touch-sensitive surface 451. The gesture is detected when cursor 3922 is over icon 3908. Movement 3924 corresponds to movement of cursor 3922 outside of application taskbar area 3916. As another example, Figures 11J-11K illustrate a gesture comprising contact 3930 and movement 3934 of contact 3930 detected on touch-sensitive surface 451. The gesture is detected when cursor 3922 is over icon 3908. Movement 3934 corresponds to movement of cursor 3922 outside of application taskbar area 3916.

As another example, Figures 11P-11Q show a gesture including contact 3946 and movement 3948 of contact 3946 detected on touch-sensitive display 112. Contact 3946 is detected over icon 3908, and movement 3948 includes movement of contact 3946 outside of application taskbar area 3916. Figures 11S-11T show a gesture including contact 3950 and movement 3952 of contact 3950 detected on touch-sensitive display 112. Contact 3950 is detected over icon 3908, and movement 3952 includes movement of contact 3950 outside of application taskbar area 3916. In these examples, the location of the contact (contact 3946 or 3950) is similar to the location of the focus selector.

In response to detecting (4006) a gesture and in accordance with determining (4008) that the contact had a maximum intensity below a corresponding intensity threshold during the gesture, the device displays (4010) an indication that the gesture is being performed (e.g., displaying movement of the corresponding icon or a ghost of the corresponding icon) during the gesture and retains (4018) the corresponding icon in the predefined area after the gesture ends (e.g., in response to detecting lift-off of the contact, the corresponding icon bounces or otherwise returns to its original position or stops moving at a boundary of the predefined area). In response to detecting a gesture performed with contact 3920 and in accordance with determining that contact 3920 had a maximum intensity below a defined intensity threshold during the gesture, the device displays, for example, movement of icon 3908 (Figure 11C) or ghost 3926 (Figure 11F) or a copy icon 3928 (Figure 11G) in accordance with movement 3924 of contact 3920 and retains icon 3908 in application taskbar area 3916 after the gesture ends, as shown in Figure 11D. As another example, in response to detecting a gesture performed using contact 3946 (Figure 11P), icon 3908 moves according to movement 3948 of contact 3946 and then remains in application taskbar area 3916 after the gesture ends, as shown in Figure 11R.

In some embodiments, displaying an indication that a gesture is being performed includes (4012) changing the appearance of a corresponding icon in a predefined area (e.g., replacing the corresponding icon with a ghost of the icon), and while the corresponding icon having the changed appearance is displayed on the display, moving a representation of the corresponding icon in accordance with movement of the contact during the gesture. For example, in response to detecting a gesture performed using contact 3920, icon 3908 optionally changes to ghost 3926, and while ghost 3926 is displayed, a representation of icon 3908 (e.g., copy icon 3928 or ghost 3926) moves in accordance with movement 3924, as shown in Figures 11E-11G.

In some embodiments, displaying an indication that a gesture is being performed includes (4014) moving a cursor on the display in accordance with movement of the contact while the corresponding icon is held in a fixed position in a predefined area (e.g., icons in a taskbar are stationary or "stuck" in place unless the user presses hard enough). For example, in response to detecting a gesture performed with contact 3920, while icon 3908 is held in place, cursor 3922 optionally moves in accordance with movement 3924, as shown in FIG11H .

In some embodiments, displaying an indication that a gesture is being performed includes (4016) moving the corresponding icon in accordance with movement of the contact during the gesture when movement of the corresponding icon is constrained to a predefined area (e.g., the icon can still be repositioned within the taskbar during the gesture, but cannot be moved outside the taskbar unless the user presses harder). In some embodiments, if the icon is constrained to the predefined area, the icon moves along the edge of the predefined area closest to the contact. For example, if the predefined area is a horizontal strip near the bottom edge of the display and the focus selector is above the predefined area, the movement of the icon will track horizontal movement of the contact, but will not track vertical movement of the contact unless the focus selector is moved into the predefined area. For example, icon 3908 is optionally constrained to within application taskbar area 3916 in accordance with movement 3924 of contact 3920, as shown in FIG11I .

In response to (4006) detecting a gesture, and in accordance with determining (4020) that the contact reached an intensity above a corresponding intensity threshold during the gesture, the device moves (4022) a corresponding icon in accordance with the movement of the contact during the gesture (e.g., moves the corresponding icon outside the predefined area during the gesture so that it follows the movement of the contact), and removes (4024) the corresponding icon from the predefined area after the gesture ends (e.g., in response to detecting lift-off of the contact, deletes the icon or moves the icon to a location outside the predefined area). For example, in Figures 11J-11K, in response to detecting a gesture performed with contact 3930 and in accordance with determining that contact 3930 reached an intensity above a defined intensity threshold during the gesture, the device displays, for example, movement of icon 3908 in accordance with movement 3934 of contact 3930, and removes icon 3908 from application taskbar area 3916 after the gesture ends, as shown in Figure 11M. As another example, in Figures 11S-11T, in response to detecting a gesture performed using contact 3950, icon 3908 moves according to movement 3952 of contact 3950 and is then removed from application taskbar area 3916 after the gesture ends, as shown in Figure 11U.

In some embodiments, before detecting the end of the gesture, based on determining that the contact reached an intensity above a corresponding intensity threshold during the gesture, before removing the corresponding icon from the predefined area, the device displays (4026) a visual indication that the corresponding icon will be removed from the taskbar (e.g., before detecting the end of the gesture, displaying the corresponding icon moving out of the predefined area and/or displaying an icon removal symbol, such as a “cloud” or a red “X”, in the case where the corresponding icon will be permanently removed from the predefined area). Figure 11L shows, for example, icon 3936 (e.g., an application icon removal indicator) displayed in application taskbar area 3916 where icon 3908 was located, before a gesture performed with contact 3930 ends.

In some embodiments, when the focus selector is over the corresponding icon, the device detects (4028) a tap gesture on the touch-sensitive surface, and in response to detecting the tap gesture, the device launches (4030) a corresponding application associated with the corresponding icon. In some embodiments, the corresponding icon is a document icon that opens a specific document using a default application. In some embodiments, the corresponding icon is an application icon that opens an application without a document or with an initially displayed default/blank document. For example, in Figures 11N-11O, in response to detecting a tap gesture 3938 on touch-sensitive surface 451 when cursor 3922 is over icon 3908, the application associated with icon 3908 is launched and a corresponding interface 3940 is displayed.

In some embodiments, instead of determining whether to remove the respective icon from the predefined area based on whether the contact had a maximum intensity above or below a corresponding intensity threshold during a gesture corresponding to movement outside the predefined area, the device determines whether to rearrange the plurality of icons within the predefined area based on whether the contact had a maximum intensity above or below the corresponding intensity threshold. Thus, in some embodiments, the device displays a plurality of icons within a predefined area on the display, wherein the plurality of icons includes the respective icon. The device detects a gesture comprising a contact on the touch-sensitive surface while a focus selector is over the respective icon; and subsequent movement of the contact on the touch-sensitive surface, the subsequent movement corresponding to movement of the focus selector within the predefined area. In response to detecting the gesture: based on determining that the contact had a maximum intensity below the corresponding intensity threshold during the gesture, the device displays an indication that the gesture is being performed during the gesture; and does not rearrange (e.g., reorder) the respective icon within the predefined area after the gesture ends. Instead, based on determining that the contact reached an intensity above the corresponding intensity threshold during the gesture, the device moves the respective icon in accordance with the movement of the contact during the gesture; and rearranges (e.g., reorders) the respective icon within the predefined area relative to other icons in the plurality of icons after the gesture ends. For example, in Figure 11P, if the device detects contact 3946 having a maximum intensity below IT _D moving to a position between icon 3910 and icon 3912, then when the contact intensity drops below IT _L , icon 3908 will return to its original position, and the icons in predefined area 3916 will not be rearranged, as shown in Figure 11R (e.g., because contact 3946 has a maximum intensity below IT _D ). Conversely, in Figure 11S, if the device detects contact 3950 having a maximum intensity above IT _D moving to a position between icon 3910 and icon 3912, then when the contact intensity drops below IT _L , icon 3908 will be placed between icon 3910 and icon 3912, and the icons in predefined area 3916 will be rearranged accordingly (e.g., because contact 3950 has a maximum intensity above IT _D ). Inadvertently rearranging icons in a predefined area can be frustrating for a user who may not realize that the icons have been rearranged and then have difficulty finding icons that have been moved from their usual locations. Using a higher intensity threshold (eg, IT _D ) to control whether a user gesture rearranges icons in a predefined area helps prevent users from inadvertently rearranging icons within the predefined area, thereby improving the efficiency and predictability of the user interface.

It should be understood that the particular order in which the operations described in Figures 12A-12B are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art will appreciate various ways to reorder the operations described herein. In addition, it should be noted that the details of the other processes described herein with respect to other methods described herein (e.g., those listed in paragraph [0057]) also apply in a similar manner to the method 4000 described above with respect to Figures 12A-12B. For example, the contacts, gestures, user interface objects, intensity thresholds, and focus selectors described above with reference to method 4000 optionally have one or more characteristics of the contacts, gestures, user interface objects, intensity thresholds, and focus selectors described herein with reference to other methods described herein (e.g., those listed in paragraph [0057]). For the sake of brevity, these details are not repeated here.

According to some embodiments, Figure 13 shows a functional block diagram of an electronic device 4100 configured according to the principle of the various described embodiments. The functional block of this equipment is optionally realized by hardware, software, or a combination of hardware and software that performs the principle of the various described embodiments. Those skilled in the art will appreciate that the functional block described in Figure 13 is optionally combined or separated into sub-blocks to realize the principle of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further qualification of the functional block described herein.

13 , electronic device 4100 includes a display unit 4102 configured to display a plurality of icons in a predefined area on display unit 4102, wherein the plurality of icons includes a corresponding icon; a touch-sensitive surface unit 4104 configured to receive a gesture; one or more sensor units 4105 configured to detect intensity of contact with touch-sensitive surface unit 4104; and a processing unit 4106 coupled to display unit 4102, touch-sensitive surface unit 4104, and sensor unit 4105. In some embodiments, processing unit 4106 includes a detection unit 4108, a display enabling unit 4110, a retaining unit 4112, a moving unit 4114, a removing unit 4116, and an activating unit 4118.

Processing unit 4106 is configured to: detect a gesture, which includes contact on touch-sensitive surface unit 4104 when the focus selector is over the corresponding icon and subsequent movement of the contact on touch-sensitive surface unit 4104, which subsequent movement corresponds to movement of the focus selector outside of the predefined area (for example, using detection unit 4108), and in response to detecting the gesture: based on determining that the contact has a maximum intensity below the corresponding intensity threshold during the gesture, enable display of an indication that the gesture is being performed during the gesture (for example, using display enabling unit 4110) and retain the corresponding icon in the predefined area after the gesture ends (for example, using retention unit 4112); and based on determining that the contact reaches an intensity above the corresponding intensity threshold during the gesture, move the corresponding icon according to the movement of the contact during the gesture (for example, using movement unit 4114) and remove the corresponding icon from the predefined area after the gesture ends (for example, using removal unit 4116).

In some embodiments, displaying an indication that a gesture is being performed includes changing the appearance of a corresponding icon in a predefined area, and when the corresponding icon with the changed appearance is displayed on the display unit 4102, moving the representation of the corresponding icon according to movement of the contact during the gesture.

In some embodiments, displaying an indication that a gesture is being performed includes moving a cursor on the display unit 4102 in accordance with movement of the contact while the corresponding icon is maintained in a fixed position in the predefined area.

In some embodiments, displaying an indication that a gesture is being performed includes moving the respective icon in accordance with movement of the contact during the gesture while constraining movement of the respective icon to a predefined area.

In some embodiments, the processing unit 4106 is configured to detect a tap gesture on the touch-sensitive surface unit 4104 when the focus selector is over the corresponding icon (e.g., using the detection unit 4108), and in response to detecting the tap gesture, launch the corresponding application associated with the corresponding icon (e.g., using the launch unit 4118).

In some embodiments, processing unit 4106 is configured to, before detecting the end of the gesture, enable display of a visual indication that the corresponding icon will be removed from the taskbar (e.g., using display enabling unit 4110) based on determining that the contact reaches an intensity above a corresponding intensity threshold during the gesture and before removing the corresponding icon from the predefined area.

The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above with respect to Figures 1A and 3) or an application-specific chip.

The operations described above with reference to Figures 12A-12B are optionally implemented by the components depicted in Figures 1A-1B or Figure 13. For example, detection operation 4004, display operation 4010, retention operation 4018, move operation 4022, and removal operation 4024 are optionally implemented by event classifier 170, event recognizer 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. The corresponding event recognizer 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at the first location on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as the selection of an object on a user interface or the rotation of the device from one orientation to another. When a corresponding predefined event or sub-event is detected, event recognizer 180 activates the event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally utilizes or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, event handler 190 accesses corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly understand how other processes can be implemented based on the components shown in Figures 1A-1B.

Display content layer

Many electronic devices include applications or user interfaces in which different corresponding content, such as content layers, can be displayed. For example, the corresponding content may be an image of a location at different times or a map of different types of locations. In some approaches, in order to display different content layers, the user must open a menu and select the desired content layer, which can be cumbersome and time consuming. The embodiments described below improve upon these approaches. When the focus selector is over a content layer, the user increases the intensity of the contact on the touch-sensitive surface. If the intensity increases above a threshold, corresponding portions of the different content layers are displayed. The different content layers can also be gradually revealed based on the intensity of the contact, or panned by moving the focus selector. These features make the display and manipulation of content layers more efficient.

Figures 14A-14L illustrate exemplary user interfaces for displaying content layers, in accordance with some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 15A-15C. Figures 14A-14L include an intensity graph that illustrates the current intensity of a contact on the touch-sensitive surface relative to multiple intensity thresholds, including a light press intensity threshold (e.g., "IT _L ") and a transition-start intensity threshold (e.g., "IT ₁ "). In some embodiments, operations similar to those described below with reference to "IT _L " are performed with reference to a different intensity threshold (e.g., "IT _D ").

In some embodiments, the device is an electronic device having an independent display (e.g., display 450) and an independent touch-sensitive surface (e.g., touch-sensitive surface 451). In some embodiments, the device is portable multifunction device 100, the display is touch-sensitive display system 112, and the touch-sensitive surface includes tactile output generator 167 ( FIG. 1A ) on the display. For ease of explanation, the embodiments described with reference to FIG. 14A-14L and FIG. 15A-15C will be discussed with reference to display 450 and independent touch-sensitive surface 451. However, similar operations are optionally performed on the device having touch-sensitive display system 112 in response to detecting the contacts described in FIG. 14A-14L on touch-sensitive display system 112 when the user interface shown in FIG. 14A-14L is displayed on touch-sensitive display system 112. In such embodiments, the focus selector is optionally: the corresponding contact, a representative point corresponding to the contact (e.g., the centroid of the corresponding contact or a point associated with the corresponding contact), or the centroid of two or more contacts detected on touch-sensitive display system 112 instead of cursor 6904.

14A shows the device displaying content 6902 on display 450. In some embodiments, content 6902 is an image (e.g., an aerial or satellite image of the location) of a location at a first time (e.g., at a specific date and time). In some other embodiments, content 6902 is a first type of map (e.g., a street map) of the location.

The device also displays a focus selector (e.g., cursor 6904) on display 450. In some embodiments, cursor 6904 is a pointer (e.g., a mouse pointer). In FIG14A , cursor 6904 is located at position 6904-a above content 6902. Cursor 6904 is associated with the portion of content 6902 over which cursor 6904 is located. For example, in FIG14A , cursor 6904 at position 6904-a is associated with portion 6902-1. As cursor 6904 moves over content 6902, the associated portion of content 6902 also changes with the position of cursor 6904 (e.g., the portion of content 6902 associated with the focus selector corresponds to content within a predefined radius or distance of cursor 6904 or within a predefined area centered on cursor 6904).

When the portion of content with which cursor 6904 is associated has a finite size, the extent of the portion (e.g., size and boundaries) is not displayed to the user, so that the demarcation of the portion from the rest of content 6902 is not optionally apparent to the user until the device begins replacing the portion of the first content with the corresponding portion of the second content, as described below. For example, in Figure 14A, the boundaries of portion 6902-1 are not displayed to the user on display 450.

In Figure 14A , when cursor 6904 is over portion 6902-1, the device detects contact 6906 on touch-sensitive surface 451. While cursor 6904 is over portion 6902-1, the device detects an increase in the intensity of contact 6906 from below a light press intensity threshold (e.g., IT _L ) to above a light press intensity threshold (e.g., IT _L ), as shown in Figures 14B-14C . In response to detecting an increase in the intensity of contact 6906 (e.g., from an intensity below IT _L in Figure 14B to an intensity above IT _L in Figure 14C ), the device stops displaying portion 6902-1 and displays portion 6908-1 of content 6908, as shown in Figure 14C . Portion 6908-1 has the same size and position as portion 6902-1. In Figure 14C , all of content 6908 is not displayed; only portion 6908-1 is displayed. In some embodiments, portion 6908 - 1 is displayed to give the visual appearance of being viewed through a virtual magnifying glass or viewport, as if the user were looking through a hole in content 6902 to view content 6908 “underneath” content 6902 .

In some embodiments where content 6902 is an image of a location at a first time, content 6908 is an image of the same location at a second time different from the first time (e.g., a different time and date). In some embodiments where content 6902 is a map of a first type of location, content 6908 is a map of the same location of a second type different from the first type (e.g., a public transportation map). Portions of content 6908 are associated with corresponding portions of content 6902. Thus, portion 6908-1 corresponds to portion 6902-1. For example, if content 6902 and 6908 are different types of maps of the same location, portions 6902-1 and 6908-1 are portions of respective maps near the same street intersection. In some embodiments, content 6902 is a first image and content 6908 is a second image corresponding to the first image (e.g., an image of the same object or group of objects that differs from the first image based on media capture settings such as exposure, white balance, focus, etc., or based on image capture conditions such as time of day, lighting, etc.). In some embodiments, content 6902 is a first document and content 6908 is an earlier version of the first document (e.g., an earlier version of a word processing, spreadsheet, presentation, or PDF document). Although a number of examples have been provided, these examples are not limiting, and similar operations may optionally be performed on any set of first and second content, for which it may be advantageous to replace a particular portion of the first content with the second content while continuing to display other portions of the first content.

In some embodiments, in response to detecting that the intensity of contact 6906 drops below a light press intensity threshold (e.g., IT _L ), display of content 6908-1 stops, and content 6902-1 is redisplayed as part of content 6902 as a whole (e.g., if contact 6906 drops back to an intensity below IT _L as shown in FIG. 14B , display of the first portion 6908-1 of the second content stops).

In FIG14C , portion 6908-1 has a first size that is the same as the size of portion 6902-1 in FIG14A-14B . In some embodiments, the first size is based on the intensity of contact 6906 (e.g., when the intensity of contact 6906 increases above a predefined intensity threshold, the size of the area of first content 6902 that is replaced with second content 6908 increases in a corresponding manner). In response to detecting an increase in the intensity of contact 6906, the device increases the size of portion 6902-1 to a second size. If an increase in intensity is detected while portion 6908-1 is displayed, the size of portion 6908-1 is increased to the second size to match the increase in size of portion 6902-1.

For example, FIG14D shows that the intensity of contact 6906 has increased to a level higher than that shown in FIG14C . In response to detecting the increase in intensity, the sizes of portion 6902-1 and portion 6908-1 are increased by an amount corresponding to the increase in the intensity of contact 6906. The increase in size of portion 6908-1 is displayed on display 450, but the increase in size of portion 6902-1 is displayed as the size of the hole in content 6902 in FIG14D . Similarly, FIG14E shows that the intensity of contact 6906 has increased to a level higher than that shown in FIG14D . In response to detecting the increase in intensity, the sizes of portion 6902-1 and portion 6908-1 are further increased from the size shown in FIG14D . Similarly, the increase in size of portion 6908-1 is displayed on display 450, but the increase in size of portion 6902-1 is displayed as the size of the hole in content 6902 in FIG14E . In some embodiments, portions 6902-1 and 6908-1 are reduced in size in response to a decrease in the intensity of contact 6906 (eg, the device reversal performed in response to the increase in intensity shown in Figures 14C-14E).

Figure 14F shows content 6902, including portion 6902-1 and cursor 6904, displayed on display 450, as shown in Figure 14A. The device detects contact 6910 with an intensity below a light press intensity threshold (e.g., IT _L ) and below a transition-start intensity threshold (e.g., IT ₁ ) on touch-sensitive surface 451. In Figures 14A-14L , the transition-start intensity threshold (e.g., IT ₁ ) is lower than the light press intensity threshold (e.g., IT _L ).

The device detects that the intensity of contact 6910 increases to a higher intensity that is still below the transition-start intensity threshold (e.g., IT ₁ ), as shown in Figure 14G. Based on determining that the new intensity is below the transition-start intensity threshold (e.g., IT ₁ ), portion 6902-1 continues to be displayed and portion 6908-1 is not displayed.

In Figure 14H, the device detects that the intensity of contact 6910 has further increased to an intensity that is above the transition-start intensity threshold (e.g., _IT1 ) and below the light press intensity threshold (e.g., IT _L ). Based on determining that the new intensity is above the transition-start intensity threshold (and below the light press intensity threshold IT _L ), portion 6908-1 is displayed, and a reduced-opaque (e.g., visually more transparent) version of portion 6902-1 is displayed overlying portion 6908-1. Thus, from the user's perspective, both portion 6908-1 and portion 6902-1 are visible, with portion 6908-1 visible through the more transparent portion 6902-1.

In some embodiments, the opacity of reduced-opacity portion 6902-1 is based on an intensity between a transition-start intensity threshold (e.g., IT ₁ ) and a light press intensity threshold (e.g., IT _L ) of contact 6910. The opacity is reduced in response to an increase in the intensity of contact 6910, and the opacity is increased in response to a decrease in the intensity of contact 6910. Thus, portion 6908-1 optionally has the appearance of gradually revealing on display 450 and gradually replacing portion 6902-1.

In Figure 14I, the device detects that the intensity of contact 6910 has increased above a light press intensity threshold (e.g., IT _L ). In response to detecting that the intensity of contact 6910 has increased above the light press intensity threshold (e.g., IT _L ), the device stops displaying portion 6902-1 (with reduced opacity or otherwise) and displays content portion 6908-1. Although the previous example of gradually stopping displaying the first portion of first content 6902 has been described with reference to transition initiation threshold IT ₁ and light press intensity threshold IT _L , in some embodiments, IT _L is the intensity threshold at which the first portion of first content 6902 begins to gradually fade to reveal the first portion of second content 6908, and the deep press intensity threshold IT _D is the intensity threshold at which the first portion of the first content completely fades to reveal the first portion of second content 6908.

FIG14J shows a cursor 6904 at position 6904-1 above portion 6908-1, and contact 6910 at position 6910-a. As shown in FIG14J-FIG14L, the device detects movement 6912 (FIG14K-FIG14L) of contact 6910 from position 6910-a toward position 6910-b on touch-sensitive surface 451. In response to detecting movement of contact 6926, cursor 6904 moves from position 6904-a (FIG14J) to position 6904-b (FIG14K-FIG14L) above portion 6902-2 (as shown in FIG14K) of content 6902. The device stops displaying portion 6902-2 and displays portion 6908-2 corresponding to portion 6902-2, as shown in FIG14L.

In Figures 14J-14L , cursor 6904 moves in response to movement 6912 of contact 6910, with a portion of content 6908 still displayed and the displayed portion of content 6908 changing as cursor 6904 moves over the new portion of content 6902; the displayed portion of content 6908 follows cursor 6904. For example, if a portion of content 6908 is displayed as if viewed through a virtual magnifying glass, viewport, or hole in content 6902, the virtual magnifying glass, viewport, or hole in content 6902 moves along with cursor 6904, and the portion of content 6908 viewed through the virtual magnifying glass or viewport changes. In Figure 14K , the displayed portion of content 6908 is not shown to facilitate understanding of the movement of cursor 6904 over different portions of content 6902. Additionally, as shown in Figure 14K , portions 6902-1 and 6902-2 optionally at least partially overlap.

Figures 15A-15C are flow diagrams illustrating method 7000 for displaying content layers according to some embodiments. Method 7000 is performed at an electronic device having a display and a touch-sensitive surface (e.g., device 300, Figure 3, or portable multifunction device 100, Figure 1A). In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 7000 may optionally be combined, and/or the order of some operations may optionally be changed.

As described below, method 7000 provides an intuitive way to display content layers. The method reduces the cognitive burden on the user when displaying content layers, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to display content layers faster and more efficiently saves power and increases the time between battery charges.

The device displays (7002) first content on the display, where a portion of the first content is related to a corresponding portion of second content that was not (initially) displayed on the display (e.g., the corresponding portion of the second content is not displayed until an increase in contact intensity on the touch-sensitive surface is detected, as described below). For example, Figures 14A and 14F show content 6902 being displayed on display 450, where a portion of content 6902 is related to a corresponding portion of content 6908 that was not (initially) displayed.

When the focus selector is over the first portion of the first content, the device detects (7014) that the intensity of the contact on the touch-sensitive surface increases above a corresponding intensity threshold (e.g., IT _L ). For example, Figures 14A-14B illustrate detecting that the intensity of contact 6906 increases above a corresponding intensity threshold (e.g., IT _L ) when cursor 6904 is over portion 6902-1. Figures 14G-14I illustrate detecting that the intensity of contact 6910 increases above a corresponding intensity threshold (e.g., IT _L ) when cursor 6904 is over portion 6902-1.

In response to detecting (7018) that the intensity of the contact on the touch-sensitive surface increases above a corresponding intensity threshold (e.g., IT _L ), the device ceases (7020) displaying the first portion of the first content and displays (7022) a first portion of the second content corresponding to the first portion of the first content. For example, in Figures 14A-14C , in response to detecting that the intensity of contact 6906 increases above a corresponding intensity threshold (e.g., IT _L ), the device ceases displaying portion 6902-1 ( Figures 14A and 14B ) and displays portion 6908-1 ( Figure 14C ) corresponding to portion 6902-1. In Figures 14G-14I , in response to detecting that the intensity of contact 6910 increases above a corresponding intensity threshold (e.g., IT _L ), the device ceases displaying portion 6902-1 ( Figure 14G ) and displays portion 6908-1 ( Figure 14I ).

In some embodiments, before displaying the first portion of the second content (7004), the device detects (7006) an increase in intensity of the contact from a first intensity below a corresponding intensity threshold (e.g., IT _L ) to a second intensity below the corresponding intensity threshold (e.g., IT _L ). In response to detecting (7008) the increase in intensity of the contact to a second intensity below the corresponding intensity threshold (e.g., IT _L ), based on determining that the second intensity is below a transition-start intensity threshold (e.g., IT ₁ ), the device continues (7010) displaying the first portion of the first content without displaying the first portion of the second content; and based on determining that the intensity of the contact is above the transition-start intensity threshold (e.g., IT ₁ ), the device displays (7012) a reduced-opacity version of the first portion of the first content overlaid on the first portion of the second content. In some embodiments, the opacity of the first portion of the first content decreases as the intensity increases to the corresponding threshold. As shown in Figures 14F-14G , for example, before displaying portion 6908-1, the device detects that the intensity of contact 6910 increases to an intensity below the corresponding intensity threshold (e.g., IT _L ). Based on the determination that the new intensity is below the transition-start intensity threshold IT ₁ , as shown in FIG 14G , the device continues to display portion 6902-1 without displaying portion 6908-1. Conversely, based on the determination that the new intensity is above the transition-start intensity threshold IT ₁ , as shown in FIG 14H , the device displays a reduced-opacity version of portion 6902-1 overlaid on portion 6908-1.

In some embodiments, the size (e.g., area) of the first portion of the first content (and the corresponding first portion of the second content) is determined (7016) based on the intensity of the contact. After displaying (7028) the first portion of the second content, the device detects (7030) an increase in the intensity of the contact and, in response to detecting the increase in the intensity of the contact, increases (7032) the size of the first portion of the second content (and the corresponding first portion of the first content). For example, in Figures 14C-14E, after displaying portion 6908-1, the device detects an increase in the intensity of contact 6906. In response to detecting the increase in intensity, the device increases the size of portion 6902-1 (in the background) and increases the size of portion 6908-1 to match the new size of portion 6902-1, as shown in Figure 14E.

In some embodiments, the first content includes (7024) an image of a location (e.g., a physical location) at a first time, the second content includes an image of the location at a second time different from the first time, and the first portion of the first content and the first portion of the second content correspond to the same portion of the location. For example, content 6902 includes an image of the location at a first time, and content 6908 includes an image of the location at a second time different from the first time. Portion 6902-1 corresponds to portion 6908-1; they display the same portion of the location at different times.

In some embodiments, the first content includes (7026) a first map of a location (e.g., a physical location) and the second content includes a second map of the location, wherein the first map is different from the second map and the first portion of the first content and the first portion of the second content correspond to the same portion of the location. For example, the first map includes map features that are different from map features included in the second map (e.g., a 1902 map of San Francisco instead of a 2012 map of San Francisco or a street map of San Francisco overlaid on a public transportation map of San Francisco). For example, content 6902 includes a map of a first type of location (e.g., a street map) and content 6908 includes a map of a second type of location that is different from the first type (e.g., a public transportation map). Portion 6902-1 corresponds to portion 6908-1; they show different map types for the same portion of the location.

In some embodiments, after displaying (7028) the first portion of the second content, the device detects (7034) movement of the contact. In response to detecting (7036) the movement of the contact, the device moves (7038) the focus selector over the second portion of the first content, stops (7040) displaying the second portion of the first content, and displays (7042) a second portion of the second content that corresponds to the second portion of the first content (e.g., the "revealed" area of the second content moves in accordance with the movement of the contact on the touch-sensitive surface). For example, in Figures 14J-14L, after displaying portion 6908-1, the device detects movement 6912 of contact 6910 on touch-sensitive surface 451. In response to detecting movement 6912 of contact 6910 , the device moves the cursor 6904 on the display 450 and then stops displaying portion 6902 - 2 and displays portion 6908 - 2 , which is the portion of the first content 6902 and the second content 6908 that corresponds to the new position of the cursor 6904 on the display 450 .

In some embodiments, the second portion of the first content at least partially overlaps the first portion of the first content (7044). In some embodiments, the second portion of the second content at least partially overlaps the first portion of the second content (e.g., the "revealed" area of the second content changes smoothly based on the movement of the contact, such that when a row of pixels is revealed on the right edge of the "revealed" area, the corresponding row of pixels stops being revealed on the left edge of the "revealed" area). For example, as shown in Figure 14K, portions 6902-1 and 6902-2 overlap.

It should be understood that the particular order in which the operations described in Figures 15A-15C are described is merely exemplary and is not intended to indicate that the order is the only order in which these operations can be performed. One of ordinary skill in the art will appreciate various ways to reorder the operations described herein. In addition, it should be noted that the details of the other processes described herein with respect to other methods described herein (e.g., those listed in paragraph [0057]) are also applicable in a similar manner to the method 7000 described above with respect to Figures 15A-15C. For example, the contacts, intensity thresholds, and focus selectors described above with reference to method 7000 optionally have one or more characteristics of the contacts, intensity thresholds, and focus selectors described herein with reference to other methods described herein (e.g., those listed in paragraph [0057]). For the sake of brevity, these details are not repeated here.

According to some embodiments, Figure 16 shows a functional block diagram of an electronic device 7100 configured according to the principle of the various described embodiments. The functional block of this equipment is optionally realized by hardware, software, or a combination of hardware and software that performs the principle of the various described embodiments. Those skilled in the art will appreciate that the functional block described in Figure 16 is optionally combined or separated into sub-blocks to realize the principle of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further qualification of functional block described herein.

16 , electronic device 7100 includes a display unit 7102 configured to display first content on display unit 7102, wherein a portion of the first content is associated with a corresponding portion of second content that is not displayed on display unit 7102; a touch-sensitive surface unit 7104 configured to receive a contact; one or more sensors 7106 configured to detect intensity of contact with touch-sensitive surface unit 7104; and a processing unit 7108 coupled to display unit 7102, touch-sensitive surface unit 7104, and sensors 7106. In some embodiments, processing unit 7108 includes a detecting unit 7110, a stopping unit 7112, a display enabling unit 7114, an enlarging unit 7116, and a moving unit 7118.

The processing unit 7108 is configured to: when the focus selector is over the first portion of the first content, detect that the intensity of the contact on the touch-sensitive surface unit 7104 increases to above the corresponding intensity threshold (e.g., IT _L ) (e.g., using the detection unit 7110); and in response to detecting that the intensity of the contact on the touch-sensitive surface unit 7104 increases to above the corresponding intensity threshold: stop displaying the first portion of the first content (e.g., using the stopping unit 7112), and enable display of a first portion of the second content corresponding to the first portion of the first content (e.g., using the display enabling unit 7114).

In some embodiments, the size of the first portion of the first content is determined based on the contact strength; and the processing unit 7108 is configured to, after enabling display of the first portion of the second content: detect an increase in the strength of the contact (e.g., using the detection unit 7110), and in response to detecting the increase in the strength of the contact, increase the size of the first portion of the second content (e.g., using the increase unit 7116).

In some embodiments, the processing unit 7108 is configured to, before enabling display of the first portion of the second content: detect an increase in contact intensity from a first intensity that is below a corresponding intensity threshold to a second intensity that is below the corresponding intensity threshold (e.g., using the detection unit 7110); and in response to detecting the increase in intensity of the contact to the second intensity that is below the corresponding intensity threshold: based on determining that the second intensity is below the transition start intensity, continue to enable display of the first portion of the first content without enabling display of the first portion of the second content (e.g., using the display enabling unit 7114); and based on determining that the contact intensity is above the transition start intensity threshold (e.g., IT ₁ ), enable display of a reduced-opaque version of the first portion of the first content overlying the first portion of the second content (e.g., using the display enabling unit 7114).

In some embodiments, the processing unit 7108 is configured to, after enabling display of the first portion of the second content: detect movement of the contact (e.g., using the detection unit 7110); and in response to detecting movement of the contact: move the focus selector over the second portion of the first content (e.g., using the movement unit 7118); stop enabling display of the second portion of the first content (e.g., using the stop unit 7112); and enable display of a second portion of the second content corresponding to the second portion of the first content (e.g., using the display enabling unit 7114).

In some embodiments, the second portion of the first content at least partially overlaps with the first portion of the first content.

In some embodiments, the first content includes an image of a location at a first time, the second content includes an image of the location at a second time different from the first time, and the first portion of the first content and the first portion of the second content correspond to the same portion of the location.

In some embodiments, the first content includes a first map of a location and the second content includes a second map of the location, wherein the first map is different from the second map, and the first portion of the first content and the first portion of the second content correspond to the same portion of the location.

The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above with respect to Figures 1A and 3) or an application-specific chip.

The operations described above with reference to Figures 15A-15C are optionally implemented by the components depicted in Figures 1A-1B or Figure 16. For example, the detection operation 7014, the stop operation 7020, and the display operation 7022 are optionally implemented by event classifier 170, event recognizer 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. The corresponding event recognizer 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as the selection of an object on a user interface. Upon detecting the corresponding predefined event or sub-event, event recognizer 180 activates the event handler 190 associated with the detected event or sub-event. Event handler 190 optionally utilizes or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly know how other processes can be implemented based on the components shown in Figures 1A-1B.

Navigate between displayed user interfaces

Many electronic devices include user interfaces with different representations of the same object (e.g., content such as emails or photos). In some cases, different representations of the same object are organized into a hierarchical structure (e.g., a conversation including multiple messages or an album including multiple photos). Such groups require convenient methods for navigating between user interfaces corresponding to the levels of the hierarchical structure. Some methods for navigating between user interfaces on devices with touch-sensitive surfaces typically require the user to locate an affordance (e.g., a key or "button"). However, such methods are problematic because they are prone to inaccuracies (e.g., accidental selection of an affordance when intended to select the affordance, accidental selection of a different key). Such methods also increase the user's cognitive burden by requiring additional independent gestures. The embodiments described below provide convenient and intuitive methods for navigating between user interfaces corresponding to different representations of an object using similar gestures performed using touch when a focus selector is located over the representation of the object. In the embodiments described below, an improved method for navigating between a first user interface and a second user interface is provided, the first user interface including at least a first representation of the object and the second user interface including at least a second representation of the object. When a first user interface is displayed, when the device detects a gesture that satisfies certain navigation criteria of the object, the device replaces the first user interface with a second user interface. When a second user interface is displayed, when the device detects a gesture that satisfies certain navigation criteria, the device replaces the second user interface with the first user interface.

Figures 17A-17H illustrate exemplary user interfaces for navigating between displayed user interfaces in accordance with some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 18A-18C. Figures 17A-17H include an intensity graph that illustrates the current intensity of contacts on the touch-sensitive surface relative to a plurality of intensity thresholds, including a corresponding threshold (e.g., "IT _L "). In some embodiments, operations similar to those described below with reference to "IT _L " are performed with reference to different intensity thresholds (e.g., "IT _D ").

In some embodiments, the device is an electronic device having an independent display (e.g., display 450) and an independent touch-sensitive surface (e.g., touch-sensitive surface 451). In some embodiments, the device is portable multifunction device 100, the display is touch-sensitive display system 112, and the touch-sensitive surface includes tactile output generator 167 ( FIG. 1A ) on the display. For ease of explanation, the embodiments described with reference to FIG. 17A-17H and FIG. 18A-18C will be discussed with reference to touch screen 112. However, when the user interface shown in FIG. 17A-17H is displayed on display 450, similar operations are optionally performed on the device having display 450 and independent touch-sensitive surface 451 in response to detecting the contacts described in FIG. 17A-17H on touch-sensitive surface 451. In such embodiments, the focus selector is optionally a cursor or pointer displayed at a location corresponding to (e.g., at or near) the locations of the contacts (e.g., 8406, 8410, and 8412) shown in FIG. 17A-17H.

Figure 17A shows an exemplary user interface (e.g., a first user interface) on an electronic device 300 with a touchscreen display 112, which is replaced with a second user interface in response to a specific gesture described below, the second user interface being different from the first user interface not currently displayed in Figure 17A. In some embodiments, the device displays representations of objects (e.g., media objects such as songs, pictures, and/or electronic documents). In some embodiments, some or all of the objects are electronic messages (e.g., email messages, SMS messages, instant messages, etc.) or groups of messages (e.g., conversations). In some embodiments, the device displays a list with multiple entries, and the entries are representations of objects. The example shown in Figure 17A shows a displayed list of representations of conversations (user interface objects 8404-1 to 8404-4 represent conversations 1 to 4, respectively). Each conversation contains one or more messages. In some cases, it is desirable to change the display shown in Figure 17A to a different display that includes a list of messages within a particular conversation (e.g., navigating from an inbox view to a conversation view). Figures 17A-17H show examples of improved user interfaces for navigating between these different views, for example.

17B shows detection of contact 8406 when the focus selector is over user interface object 8404-1. In some embodiments, contact 8406 is part of a gesture that optionally includes other movement and pressure inputs, for example, on a touch-sensitive surface.

In some embodiments, when the navigation criteria are met, the device displays the second user interface. In some embodiments, the navigation criteria include the gesture having a maximum intensity greater than a corresponding intensity threshold (e.g., an intensity threshold greater than a nominal intensity measurement indicating that the contact is touching the touch-sensitive surface, such as IT _L greater than a contact detection intensity threshold IT ₀ ), as shown by contact 8406 in Figure 17B. In some embodiments, the navigation criteria include detecting an increase in intensity from below a corresponding intensity threshold (e.g., IT _{L ) to above a corresponding intensity threshold (e.g., detecting a "down stroke") while the focus selector (e.g., while the gesture) is continuously over a representation of an object. In some embodiments, the navigation criteria include detecting a decrease in intensity from above a corresponding intensity threshold (e.g., IT L )} to below a corresponding intensity threshold (e.g., detecting an "up stroke") while the focus selector is continuously over a representation of an object. In some embodiments, the navigation criteria include, while the focus selector is continuously positioned over a representation of the object, detecting an increase in intensity from below a corresponding intensity threshold (e.g., IT _L ) to above a corresponding intensity threshold (e.g., IT _L ), and subsequently detecting a decrease in intensity from above a corresponding intensity threshold (e.g., IT _L ) to below the corresponding intensity threshold (e.g., detecting both an upstroke and a downstroke without leaving the representation).

Figures 17A-17H illustrate an example in which detecting a contact with a maximum intensity above the IT _L when the focus selector is positioned over a particular representation of an object is sufficient to satisfy the navigation criteria. Thus, contact 8406 satisfies the criteria. Consequently, contact 8406 in Figure 17B will initiate display of the second user interface. However, it will be apparent to those skilled in the art that other additional and/or alternative criteria may optionally be used in place of or in addition to the criteria described herein.

In some embodiments, the device optionally displays an animated transition between the first user interface and the second user interface. Figure 17C shows an example of such a transition. In the example shown, the first message set (e.g., user interface objects 8404-1 and 8404-3) slides aside in one direction, and the second message set (e.g., user interface objects 8404-2 and 8404-4) slides aside in the opposite direction, thereby revealing the second user interface (e.g., conversation view) "below" the first user interface.

Figure 17D shows an example of an animation according to an embodiment, wherein the animation includes a visual cue that indicates that a particular representation is expanding to become a corresponding document, message, or message list. Figure 17D shows an example of a selected conversation that expands to reveal a message list corresponding to the selected conversation. In some embodiments, an "expand message" animation, such as the animation shown in Figure 17D, follows a "slide message" animation (e.g., such as the animation shown in Figure 17C). In some embodiments, the expand message animation or the slide message animation appear independently of each other. In addition, it is contemplated that any number of possible animations may be optionally utilized, and different animations for different types of objects and/or different situations may sometimes be appropriate. An additional specific example of an animation, a "trampoline" animation, is described with respect to method 8500 in Figures 18A-18C.

Figure 17E shows the result of navigating from the first user interface in Figure 17B to the second user interface. Specifically, Figure 17E shows that the user has now navigated to the conversation view. In addition, Figure 17E shows contact 8410 included in the gesture and meeting the navigation criteria.

In some embodiments, because a conversation view is an instance of a user interface that is not at either extreme of a hierarchy of user interfaces (e.g., a user can navigate up in the hierarchy to return to an inbox view or down in the hierarchy to view individual messages), the device is configured to respond independently to upward and downward navigation criteria. For example, in some embodiments, the upward navigation criteria include a gesture comprising a deep press (e.g., a press input having an intensity greater than a deep press intensity threshold IT _D ) and the downward navigation criteria include a gesture having a maximum intensity between IT _L and IT _D. In some embodiments, the downward navigation criteria include a gesture comprising a deep press (e.g., a press input having an intensity greater than a deep press intensity threshold IT _D ) and the upward navigation criteria include a gesture having a maximum intensity between IT _L and IT _D. In some embodiments, other gestures, movements, gestures with multiple fingers, or a combination of light press and deep press inputs are used, alone and/or in combination, to distinguish between upward and downward navigation inputs.

For the purpose of providing an example, assume that the gesture including contact 8410 in Figure 17E is a downward navigation input. In some embodiments, an animation that is the same as or different from the animation displayed in the previously described transitions (e.g., the transition from the inbox view to the conversation view with reference to Figures 17B-17E) is displayed.

Figure 17F continues this example by showing that contact 8410 in Figure 17E has caused the device to respond by displaying a third user interface (i.e., a message view) due to satisfying the navigation criteria and/or the upward navigation criteria. Figure 17F includes contact 8412. In this example, the gesture including contact 8412 satisfies the upward navigation criteria by having at least a maximum intensity above the IT _L.

Figure 17G shows an example of some embodiments in which the device displays an animation of a first set of messages "sliding in" from a first direction and a second set of messages "sliding in" from a second direction (e.g., the opposite direction), thereby "overlaying" the previously displayed messages and returning to the second user interface (or in some cases, the first user interface).

Figure 17H shows the result of navigating upward from the message view shown in Figure 17F. In this example, the conversation view is displayed again.

Figures 18A-18C are flow diagrams illustrating method 8500 for navigating between displayed user interfaces, according to some embodiments. Method 8500 is performed on an electronic device having a display and a touch-sensitive surface (e.g., device 300, Figure 3, or portable multifunction device 100, Figure 1A). In some embodiments, the display is a touchscreen display, and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 8500 may optionally be combined, and/or the order of some operations may optionally be changed.

As described below, method 8500 provides an intuitive way to navigate between displayed user interfaces. The method reduces the cognitive burden on the user when navigating between displayed user interfaces, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to navigate between displayed user interfaces faster and more efficiently saves power and increases the time between battery charges.

The device displays (8502) a first user interface (e.g., an inbox view, Figure 17A) on the display. When the focus selector is over a first representation of an object in the first user interface (e.g., user interface object 8404-1, Figure 17B), the device detects (8504) a first gesture on the touch-sensitive surface, the first gesture including a first contact (e.g., contact 8406, Figure 17B) having a maximum intensity above a corresponding intensity threshold during the first gesture.

In some embodiments, the corresponding intensity threshold (e.g., IT _L ) corresponds ( 8506 ) to an intensity measurement that is greater than a nominal intensity measurement (e.g., IT ₀ ) indicating that the contact is touching the touch-sensitive surface. In some embodiments, when one or more sensors that detect contact intensity (e.g., "intensity sensors") are capable of measuring four or more intensity quantization values, the corresponding intensity threshold is at least one level higher than N, where N is the lowest intensity quantization value (e.g., the intensity value corresponding to the presence of a contact on the touch-sensitive surface). For example, if there are four intensity quantization values and 1 is the lowest level, the corresponding intensity threshold is 2 or a level greater than 2. For another example, if there are 256 intensity quantization values and 1 is the lowest level, the corresponding intensity threshold is 2 or a level greater than 2, such as 5, 10, or 15.

In response to detecting the first gesture, the device (8508) replaces the first user interface with a second user interface that includes a second representation of the object (e.g., a conversation view, FIG. 17E, or a message view, FIG. 17F).

In some embodiments, the object is (8510) an electronic document (e.g., a word processing document, a spreadsheet document, a presentation document, or a PDF), the first representation is an icon corresponding to the electronic document, and the second representation is a view of the electronic document (e.g., an application window for viewing and/or responding to the electronic document, a document editing user interface for editing the electronic document, or another interface for interacting with the electronic document).

In some embodiments, the object is (8512) an electronic message (e.g., an email, text message, instant message, or SMS, MMS, or social network message) with a first representation as a message list entry in an electronic message list (e.g., a representation of an email in an inbox, such as user interface object 8404-1, FIG17A), and a second representation as a view of the electronic message corresponding to the message list entry (e.g., a message view, FIG17F). Although many of the embodiments described herein are illustrated using the example of a conversation/message hierarchy, where the first representation of the object is a representation of a message within a conversation and the second representation of the object is a view of the full message, the principles described herein apply in a similar manner to items in other hierarchies. Specifically, the object can be another hierarchical item, such as a discussion thread (e.g., having a first representation of a top-level discussion and a second representation of a sub-discussion), an album (e.g., having a first representation of a song in an album view and a second expanded representation of the song), or a news story (e.g., having a first representation as a headline with an associated summary or image and a second representation as the full news story).

In some embodiments, the first user interface includes (8514) a list having a plurality of entries representing corresponding objects (e.g., a list of electronic messages as described with reference to operation 8512, or alternatively, a list of songs in an album, etc.), wherein a first representation of the object is one of the entries in the list, and replacing the first user interface with the second user interface includes sliding a first set of one or more of the entries away from the first representation of the object in a first direction and sliding a second set of one or more of the entries away from the first representation of the object in a second direction different from the first direction. In some embodiments, replacing the first user interface with the second user interface also includes displaying an animation of the first representation (list entry) expanding to become a corresponding document or message (e.g., a message list expanding, FIG. 17D ). In some embodiments, the first direction is right and the second direction is left, or vice versa. In some embodiments, the first direction is upward and the second direction is downward, or vice versa.

When the focus selector is over a second representation of an object in the second user interface, the device detects (8516) a second gesture on the touch-sensitive surface that includes a second contact (e.g., contact 8412, Figure 17F) having a maximum intensity above a corresponding intensity threshold during the second gesture. In response to detecting the second gesture, the device replaces the second user interface with the first user interface (e.g., displaying the conversation view, Figure 17G).

In some embodiments, the second gesture is (8520) a repetition of the first gesture (eg, the second gesture includes the same basic movement and/or intensity change components as the first gesture, although the second gesture occurs at a later time).

In some embodiments, the first user interface includes (8522) a list having a plurality of entries representing corresponding objects, wherein a first representation of the object is one of the entries in the list, and replacing the second user interface with the first user interface includes sliding a first set of one or more of the entries toward the first representation of the object in a first direction and sliding a second set of one or more of the entries toward the first representation of the object in a second direction different from the first direction (e.g., sliding the list entries in FIG17G ). In some embodiments, replacing the second user interface with the first user interface also includes displaying an animation of the document or message collapsing to become the corresponding first representation (the list entry).

In some embodiments, before replacing the first user interface with the second user interface, the device detects (8524) an increase in the strength of the first contact above a corresponding strength threshold. In response to detecting the increase in the strength of the first contact, the device displays (8526) a first animation of the first representation distorting in accordance with the increase in the strength of the first contact (e.g., stretching back into the display like a trampoline with a weight placed on it). In some embodiments, the first user interface is replaced with the second user interface in response to detecting a decrease in the strength and/or lift-off of the first contact, and this includes displaying a reverse animation of the distortion of the first representation (e.g., the first representation rebounds like a trampoline after a weight is lifted off the trampoline). In some embodiments, before replacing the second user interface with the first user interface, the device detects (8528) an increase in the strength of the second contact above a corresponding strength threshold. In response to detecting the increase in the strength of the second contact, the device displays (8530) a second animation of the second representation distorting in accordance with the increase in the strength of the second contact (e.g., stretching back into the display like a trampoline with a weight placed on it). In some embodiments, the second user interface is replaced with the first user interface in response to detecting a decrease in strength and/or lift-off of the second contact, and this includes displaying an animation of a reverse of the distortion of the second representation (e.g., the second representation bounces back like a trampoline after a weight is lifted off the trampoline).

In some embodiments, the first animation includes (8532) adjusting the display of the first representation by displaying multiple visual cues (e.g., visual cues associated with corresponding physical metaphors) based on the first contact strength (e.g., moving the center of the first representation downward and showing a distortion of the edges of the first representation to indicate that the first representation is being stretched downward based on the first contact strength). The second animation includes adjusting the display of the second representation by displaying multiple visual cues (e.g., visual cues associated with the same corresponding physical metaphors) based on the second contact strength (e.g., moving the center of the second representation downward and showing a distortion of the edges of the second representation to indicate that the second representation is being stretched downward based on the second contact strength). In some embodiments, the first animation and the second animation include similar distortions of corresponding user interface elements displayed on the display. It is intended that many other transitions and/or animations fall within the scope of the present disclosure. For example, several alternative animations are discussed herein with reference to Figures 17A-17H.

It should be understood that the specific order of the operations described in Figures 18A-18C is merely exemplary and is not intended to indicate that the order is the only order in which these operations can be performed. A person of ordinary skill in the art will think of various ways to reorder the operations described herein. In addition, it should be noted that the details of the other processes described herein with respect to other methods described herein (e.g., those listed in paragraph [0057]) are also applicable in a similar manner to the method 8500 described above with respect to Figures 18A-18C. For example, the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described above with reference to method 8500 optionally have one or more characteristics of the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described herein with reference to other methods described herein (e.g., those listed in paragraph [0057]). For the sake of brevity, these details are not repeated here.

According to some embodiments, Figure 19 shows a functional block diagram of an electronic device 8600 configured according to the principle of the various described embodiments. The functional block of this equipment is optionally realized by hardware, software, or a combination of hardware and software that performs the principle of the various described embodiments. Those skilled in the art will appreciate that the functional block described in Figure 19 is optionally combined or separated into sub-blocks to realize the principle of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further qualification of the functional block described herein.

As shown in FIG19 , electronic device 8600 includes a display unit 8602 configured to display a first user interface including a first representation of an object and a second user interface including a second representation of an object; a touch-sensitive surface unit 8604 configured to detect a first gesture on the touch-sensitive surface unit when a focus selector is located over the first representation of the object in the first user interface, the first gesture including a first contact having a maximum intensity greater than a corresponding intensity threshold during the first gesture, and to detect a second gesture on the touch-sensitive surface unit when the focus selector is located over the second representation of the object in the second user interface, the second gesture including a second contact having a maximum intensity greater than a corresponding intensity threshold during the second gesture; one or more sensor units 8606 configured to detect intensity of contact with touch-sensitive surface unit 8604; and a processing unit 8608 coupled to display unit 8602, touch-sensitive surface unit 8604, and one or more sensor units 8606. In some embodiments, processing unit 8608 includes a replacement unit 8610, an animation unit 8612, and a display enabling unit 8614.

Processing unit 8608 is configured to: replace the first user interface with the second user interface in response to detecting the first gesture (for example, using replacement unit 8610); and replace the second user interface with the first user interface in response to detecting the second gesture.

In some embodiments, the corresponding intensity threshold corresponds to an intensity measurement that is greater than a nominal intensity measurement, which indicates that the contact is touching touch-sensitive surface unit 8604.

In some embodiments, the second gesture is a repetition of the first gesture.

In some embodiments, the object is an electronic message; the first representation is a message list entry in a list of electronic messages; and the second representation is a view of the electronic message corresponding to the message list entry.

In some embodiments, the object is an electronic document; the first representation is an icon corresponding to the electronic document; and the second representation is a view of the electronic document.

In some embodiments, the first user interface includes a list having multiple entries representing corresponding objects, wherein a first representation of the object is one of the entries in the list; and replacing the first user interface with the second user interface includes: sliding a first set of one or more of the entries away from the first representation of the object in a first direction; and sliding a second set of one or more of the entries away from the first representation of the object in a second direction different from the first direction.

In some embodiments, the first user interface includes a list having multiple entries representing corresponding objects, wherein a first representation of the object is one of the entries in the list; and replacing the second user interface with the first user interface includes: sliding a first set of one or more entries in the entries along a first direction toward the first representation of the object; and sliding a second set of one or more entries in the entries toward the first representation of the object along a second direction different from the first direction.

In some embodiments, the processing unit 8608 is further configured to: before replacing the first user interface with the second user interface: detect that the intensity of the first contact increases to above the corresponding intensity threshold; and in response to detecting the increase in the intensity of the first contact, display a first animation of the first representation that is distorted according to the increase in the intensity of the first contact; before replacing the second user interface with the first user interface: detect that the intensity of the second contact increases to above the corresponding intensity threshold; and in response to detecting the increase in the intensity of the second contact, display a second animation of the second representation that is distorted according to the increase in the intensity of the second contact.

In some embodiments, the first animation includes adjusting display of the first representation by displaying a plurality of visual cues as a function of the first contact intensity; and the second animation includes adjusting display of the second representation by displaying a plurality of visual cues as a function of the second contact intensity.

The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above with respect to Figures 1A and 3) or an application-specific chip.

The operations described above with reference to Figures 18A-18C are optionally implemented by the components depicted in Figures 1A-1B or Figure 19. For example, detection operation 8504, replacement operation 8508, and animation operation 8532 are optionally implemented by event classifier 170, event recognizer 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. The corresponding event recognizer 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at the first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selecting an object on the user interface. When a corresponding predefined event or sub-event is detected, event recognizer 180 activates the event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally utilizes or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, event handler 190 accesses corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly understand how other processes can be implemented based on the components shown in Figures 1A-1B.

It should be understood that the specific order of the operations described above is merely exemplary and is not intended to indicate that the order is the only order in which these operations can be performed. A person of ordinary skill in the art will think of various ways to reorder the operations described herein. In addition, it should be noted that the various processes described independently herein (e.g., those listed in paragraph [0057]) can be combined with each other in different arrangements. For example, the contact, user interface object, touch, intensity threshold, and/or focus selector described above with reference to any of the various processes described independently herein (e.g., those listed in paragraph [0057]) optionally have one or more of the contact, gesture, user interface object, touch, intensity threshold, and focus selector characteristics described herein with reference to one or more of the other methods described herein (e.g., those listed in paragraph [0057]). For the sake of simplicity, not all possible combinations are specifically enumerated here, but it should be understood that the claims described above can be combined in any way that is not excluded by mutually exclusive claim features.

For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the foregoing exemplary discussion is not intended to be exhaustive or to limit the various described embodiments to the precise forms disclosed herein. Numerous modifications and variations are possible in light of the above teachings. The embodiments have been chosen and described in order to best illustrate the principles of the various described embodiments and their practical application, thereby enabling others skilled in the art to best utilize the various described embodiments with various modifications as are appropriate for the particular application contemplated.

Claims (9)

1. A method for displaying content, comprising: In an electronic device having a touch-sensitive display, said electronic device includes one or more sensors for detecting the intensity of contact with said touch-sensitive display: Display the first content on the touch-sensitive display; When a touch is detected at a first location on the touch-sensitive display associated with a bounded area of the first content, the intensity of the detected touch on the touch-sensitive display increases to above a corresponding intensity threshold. In response to detecting that the intensity of the touch on the touch-sensitive display increases to above the corresponding intensity threshold, a portion of the first content that was previously displayed in the bounded area is replaced by a visual appearance being viewed through a virtual magnifying glass in the bounded area. The touch is detected to move to a second position on the touch-sensitive display: In response to detecting movement of the contact on the touch-sensitive display, the bounded area is moved according to the movement of the contact, and the content displayed in the bounded area is changed as the contact is moved, such that the content in the bounded area continues to give the visual appearance that is being viewed through the virtual magnifying glass; Detecting a decrease in the intensity of the contact without detecting the contact being lifted off the touch-sensitive display; and In response to the detection of a decrease in the intensity of the contact, the portion of the first content that was previously displayed in the bounded area is re-displayed before the contact moves to that position. 2. The method of claim 1, wherein the step of re-displaying the portion of the first content is performed in response to detecting that the intensity of the contact has decreased to a hysteresis intensity threshold corresponding to and below the corresponding intensity threshold. 3. The method according to claim 1, wherein: The size of the bounded region is determined based on the strength of the contact; and The method includes, after displaying the content that gives the visual appearance being viewed through a virtual magnifying glass: Detecting an increase in the strength of the contact; and In response to the detection of an increase in the intensity of the contact, the size of the bounded region is increased. 4. The method according to any one of claims 1 to 2, further comprising, before displaying the content giving the visual appearance being viewed through the virtual magnifying glass: The intensity of the contact is detected to increase from a first intensity below a corresponding intensity threshold to a second intensity below the corresponding intensity threshold; and In response to the detection that the intensity of the contact increases to a second intensity below the corresponding intensity threshold: Based on the determination that the second intensity is below the transition initiation intensity threshold, the portion of the first content continues to be displayed within the bounded area; and Based on the determination that the intensity of the contact is higher than the transition initiation intensity threshold, the content gradually reveals the visual appearance being viewed through a virtual magnifying glass as the intensity of the contact increases. 5. The method according to any one of claims 1 to 2, wherein the bounded region in the first content is a circular region. 6. The method according to any one of claims 1 to 2, wherein when the content, which gives the visual appearance being viewed through a virtual magnifying glass, is displayed in the bounded area, the display of the first content in the portion outside the bounded area is maintained. 7. The method according to any one of claims 1 to 2, wherein the first content is an image of an object/collection of objects, and the content giving the visual appearance of being viewed through a virtual magnifying glass is an image of the object/collection of objects. 8. A computer-readable storage medium storing one or more programs, said one or more programs including instructions that, when executed by an electronic device having a touch display and one or more sensors for detecting the intensity of contact with said touch display, cause said electronic device to perform the method of any one of claims 1 to 7. 9. An electronic device, comprising: Touch-sensitive display; One or more sensors used to detect the intensity of contact with the touch-sensitive display; and Apparatus for performing the method according to any one of claims 1 to 7.