CN106164815A - For being postponed the system and method for power consumption by aftertreatment sensors data - Google Patents

Abstract

This invention discloses systems and methods for determining the battery life of an electronic device and conserving battery charge in the electronic device. Battery consumption of the electronic device can be reduced when the user or the device learns, via user input or by prediction, that the battery will be depleted before the next possible charging cycle. Reducing battery consumption can be achieved by accessing applications that require less power consumption and/or delaying the post-processing of sensor data associated with such applications, such as camera applications. Predicting battery life may include determining the time until the next expected battery charge and delaying the processing of sensor data until the electronic device is plugged in and charging or has reached a predetermined charge level.

Description

Systems and methods for deferring power consumption by post-processing sensor data

technical field

The systems and methods disclosed herein relate generally to conserving battery charge and, in particular, to deferring power consumption of an electronic device until a later time by post-processing sensor data.

Background technique

Nowadays, user experience and battery life are the two most important metrics for electronic devices (such as mobile phones) in the rapid mobile technology research and development of mobile business units. Some applications running on electronic devices, such as camera applications, can quickly drain the battery charge. Depletion of the battery charge can cause the electronic device to malfunction, which may be inconvenient to the user. Actively managing the power consumption of features or applications can increase the value of the system by promoting better battery life through intelligent use of features. Additionally, by tailoring the response of the mobile device based on more context-aware methods or systems, the device can provide a more compelling user experience.

Contents of the invention

The systems, methods, and devices of the disclosure each have several innovative aspects, no single one of which is solely responsible for the desirable attributes disclosed herein. Combinations of innovations, aspects, and features described herein can be incorporated in various embodiments of systems, methods, and devices and such combinations are not limited to the examples of embodiments described herein.

Embodiments and innovations described herein relate to systems and methods operable in a processor of an electronic device for deferring battery depletion when the user or device knows or knows that the battery will be depleted before the next possible charge cycle. Postponing battery drain to a later time can be accomplished by accessing applications that require less power consumption and/or delaying post-processing of sensor data associated with such applications (e.g., image or video capture applications or audio applications). accomplish. Aspects of the invention also relate to predicting battery life of an electronic device. Predicting battery life may include determining the time until the next expected battery charge, and delaying the processing of sensor data until the electronic device is plugged in and charged or has reached a predetermined charge. Other aspects of the invention relate to processes, applications, daemons and libraries that may be modified depending on the mode of operation of the electronic device.

As discussed below, some embodiments incorporate a lookup table to control which application is launched when a user selects an application icon. Those skilled in the art will appreciate that other embodiments can be used to control which application is launched under a given battery life condition.

One innovation relates to systems and methods for reducing power consumption of an electronic device when the user or device knows that the battery will be depleted before the next possible charge cycle. For example, some methods to reduce battery consumption may include determining whether a full or reduced power version of an application is running, and/or delaying post-processing of sensor data until a time when saving power is no longer an issue.

One aspect relates to a system for deferring power consumption of an electronic device. The system includes a memory component configured to store sensor data and a processor coupled to the memory component. a processor configured to retrieve sensor data from the memory component and to perform processing of the sensor data in at least two operational modes including a sensor data processing normal power mode of operation and a sensor data processing low power mode of operation, The low power mode of operation consumes less power than the normal power mode of operation. The system also includes a control module stored in the memory component. The control module includes instructions configured to operate the processor to determine a low power condition based on a threshold battery level below which the electronic device will enter a low power mode of operation or after which the device will enter a low power At least one of the threshold times of the operating modes, and operating the device in the low power operating mode based on whether a low power condition occurs. Operating the device in the low power operating mode includes storing sensor data in the memory component and performing less processing of the sensor data than when operating the device in the normal power operating mode. The system further includes an imaging device in communication with the processor and configured to generate image data, the imaging device including at least one imaging sensor. The control module is further configured to accept a user preference for a low power operating mode of the electronic device. The control module is further configured to store the sensor data in the memory component for later processing while in the low power operating mode. The control module is further configured to store the sensor data for delayed high quality post-processing during a later charge cycle of the electronic device. A low power mode of operation may include operating a camera application for a light field or plenoptic camera. A low power mode of operation may include operating a camera application for a stereo camera. A low power mode of operation may include operating a heart rate monitor application. A low power mode of operation may include operating an audio application. A low power mode of operation may include operating a camera application for the mobile device.

In another aspect, a method for deferring power consumption of an electronic device comprises the steps of: storing sensor data in a memory component of the electronic device; retrieving the sensor data from the memory component and processing of sensor data is performed in at least two operable modes of a sensor data processing low power operating mode that consumes less power than normal power operation; determining a low power condition based on electronic at least one of a threshold battery level below which the device will enter the low power mode of operation or a threshold time after which the device will enter the low power mode of operation; and in the low power mode of operation based on whether a low power condition occurs operating device. Operating the device in the low power operating mode may include storing sensor data in a memory component and performing less processing of the sensor data than when operating the device in the normal power operating mode. The method may further include the step of accepting a user preference for a low power operating mode of the electronic device. The method may further comprise the step of storing the sensor data in the memory component for later processing during low power operation of the electronic device. The method may further include the step of storing the sensor data in a memory component of the electronic device for delayed post-processing during a later charge cycle of the electronic device. A low power mode of operation may include operating a camera application for a light field or plenoptic camera. A low power mode of operation may include operating a camera application for a stereo camera. A low power mode of operation may include operating a heart rate monitor application. A low power mode of operation may include operating an audio application. A low power mode of operation may include operating a camera application for the mobile device.

In yet another aspect, an apparatus for deferring power consumption of an electronic device may comprise: means for storing sensor data; means for performing processing of sensor data in at least two of the operable modes of a low power operating mode that consumes less power than normal power operation; means for determining a low power condition, the low power condition being Based on at least one of a threshold battery level below which the electronic device will enter a low power operating mode or a threshold time after which the device will enter a low power operating mode; A means of operating a device in power mode of operation.

In another aspect, a non-transitory computer-readable medium stores instructions that, when executed, cause at least one physical computer processor to perform a method of deferring power operations of an electronic device. The method may comprise the steps of: storing sensor data in a memory component of the electronic device; retrieving the sensor data from the memory component and operating in at least two modes including a sensor data processing normal power operating mode and a sensor data processing low power operating mode In performing processing of sensor data, the low power mode of operation consumes less power than normal power operation; determining a low power condition based on a threshold battery level below which the electronic device will enter the low power mode of operation or at least one of the threshold times after which the device will enter the low power mode of operation; and operating the device in the low power mode of operation based on whether the low power condition occurs. Operating in the low power operating mode may include storing sensor data in the memory component and performing less processing of the sensor data than when operating the device in the normal power operating mode. The method may further include accepting a user preference for a low power mode of operation of the electronic device. The method may further include storing the sensor data in a memory component for later processing. The method may further include storing the sensor data in the memory component for delayed high quality post-processing during a later charge cycle of the electronic device. A low power mode of operation may include operating a camera application for a light field or plenoptic camera. A low power mode of operation may include operating a camera application for a stereo camera. A low power mode of operation may include operating a heart rate monitor application. A low power mode of operation may include operating an audio application.

Description of drawings

The disclosed aspects will be described hereinafter with reference to the accompanying drawings, which are provided for purposes of illustration but not limitation of the disclosed aspects, wherein like symbols represent like elements.

1 is a block diagram depicting a system implementing some operational elements for reducing power consumption of an electronic device.

2 is a flow diagram illustrating a process for reducing power consumption of an electronic device by post-processing sensor data.

3 is an example depicting a user interface for partial processing and delayed processing of image data to reduce power consumption.

4 is a flow diagram illustrating a process for reducing power consumption of an electronic device by post-processing sensor data.

5 is an example of a user interface illustrating one embodiment of a photo library of an electronic device implementing delayed post-processing of sensor data.

Figure 6 is an example of an image processing pipeline that may be implemented as shortened by an electronic device with a camera operable in regular and low power modes.

detailed description

Depletion of battery charge can be a problem or inconvenience to users of electronic devices. Sometimes circumstances may prevent a user from recharging an electronic device in a timely manner, such as when the user is traveling or actively away from a charging station. Prolonged use of an electronic device without access to a charging facility can result in the battery draining before the user is able to recharge the device. The methods and systems discussed below provide a solution to reduce or delay battery drain depending on the expected timing of the next possible charge cycle.

In some embodiments, a device may include a power consumption deferral process, which may include a configuration phase and a run phase. During the configuration phase, the device may provide an interface to receive user selections, and if the battery charge is low, the user-selected features may be limited. Additionally, during the configuration phase, the device's user interface may allow the device to receive a user-selectable threshold upon which the device enters a low power mode. During the run phase, the electronic device has been instructed (or configured) to enter a low power mode. In low power mode, the device can be configured to limit available features and/or functionality, run alternative low power applications, adjust which libraries are accessed, modify other processing functions including background processing (e.g., daemons), or adjust current Any other function running on the device's processor. In some embodiments, the processor of the electronic device may be instructed (or configured) to delay post-processing of sensor data until it is time to recharge the battery.

It should also be noted that the examples may be described as processes depicted as flowcharts, flow diagrams, finite state diagrams, structure diagrams, or block diagrams. Although a flowchart may describe operations as a sequential process, many operations may be performed in parallel or simultaneously, and the process may be repeated. Additionally, the order of operations may be rearranged. A process terminates when its operations are complete. A process may correspond to a method, function, procedure, subroutine, subroutine, or the like. When a process corresponds to a software function, termination of the process corresponds to a return of the function to the calling or main function.

Embodiments may be implemented in a system on a chip (SoC) or external hardware, software, firmware, or any combination thereof. Those of skill in the art would understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof and chips.

In the following description, specific details are given to provide a thorough understanding of examples. However, it will be understood by those skilled in the art that the described examples may be practiced without these specific details. For example, electrical components/devices may be shown in block diagrams in order not to obscure the examples with unnecessary detail. In other instances, such components, other structures and techniques may be shown in detail to further explain aspects.

System Overview

1 illustrates one example of a power management system 100 configured to estimate power consumption of an electronic device and implement power reduction strategies based on the estimated power consumption or remaining battery charge. The illustrated embodiments are not meant to be limiting, but to illustrate certain components in some embodiments. The power management system 100 may include various other components for other functions, which are not shown for clarity of the illustrated components.

Power management system 100 may include imaging device 110 and electronic display 130 . Certain embodiments of electronic display 130 may be any flat panel display technology, such as LED, LCD, plasma, or projection screen. Electronic display 130 may be coupled to processor 120 for receiving information for presenting a visual display to a user. This information may include, but is not limited to, visual representations of files stored in memory locations, software applications installed on processor 120, user interfaces, and network-accessible content objects.

Imaging device 110 may include one or a combination of imaging sensors. The processor 120 of the power management system 100 may also be coupled to the imaging device 110 and be in data communication with the imaging device 110 . The power management system 100 may also include a working memory 135 and a program memory 140 also in communication with the processor 120 . The power management system 100 may be a mobile device, such as a portable wireless device, including but not limited to a tablet computer, a laptop computer, or a cellular telephone such as a smartphone.

In some embodiments, the processor 120 may be a general-purpose processing unit, or in some embodiments, the processor 120 may be specially designed for power management or image processing applications of handheld electronic devices. In some embodiments, processor 120 may include an image signal processor (ISP) for digital processing of image data in a digital camera, mobile phone, or other device with a camera. As shown, processor 120 is coupled to, and in data communication with, program memory 140 and working memory 135 . In some embodiments, working memory 135 may be incorporated into processor 120, such as a cache memory. Working memory 135 may also be a component separate from and coupled to processor 120, such as one or more RAM or DRAM components. In other words, although FIG. 1 illustrates two memory components, including a memory component 140 comprising several modules and a separate memory 135 comprising working memory, those skilled in the art will recognize several embodiments utilizing different memory architectures. For example, a design may utilize ROM or static RAM memory for storing processor instructions that implement the modules contained in memory 140 . The processor instructions can then be loaded into RAM for execution by the processor. Working memory 135 may be, for example, RAM memory where instructions are loaded into working memory 135 prior to execution by processor 120 .

In the illustrated embodiment, program memory 140 stores image capture module 145 , battery level determination module 150 , post-processing determination module 155 , low power applications module 160 , operating system 165 , and user interface module 170 . These modules may contain instructions that configure processor 120 to perform various image processing and device management tasks. Program memory 140 may be any suitable computer-readable storage medium, such as a non-transitory storage medium. Working memory 135 may be used by processor 120 to store a working set of processor instructions contained in modules of memory 140 . Alternatively, working memory 135 may also be used by processor 120 to store dynamic data created during operation of power management system 100 .

As mentioned above, the processor 120 may be configured by several modules stored in the memory 140 . In other words, process 120 may execute instructions stored in modules in memory 140 . Image capture control module 145 may include instructions that configure processor 120 to obtain images from an imaging device. Thus, processor 120, along with image capture control module 145, imaging device 110, and working memory 135 represent one means for obtaining image sensor data.

Still referring to FIG. 1 , the memory 140 may also contain a battery level determination module 150 . The battery level determination module 150 may include instructions that configure the processor 120 to determine the amount of charge remaining in the electronic device, as will be described in further detail below. Thus, the processor 120 together with the battery charge determination module 150 and the working memory 135 represent one means for estimating the amount of battery charge or charge remaining on the battery of the electronic device.

The memory 140 may also contain a post-processing determination module 155 . Post-processing determination module 155 may include instructions that configure processor 120 to perform limited post-processing or delay post-processing of acquired image data based on the amount of battery charge remaining. For example, if the remaining battery charge is less than a predetermined threshold level or a threshold determined during operation (e.g., a dynamically determined threshold), processor 120 may be instructed by post-processing determination module 155 to delay post-processing functions, such as robust demosaicing Filtering, motion stabilization, skin tone correction, and more. Thus, processor 120, along with battery level determination module 150, post-processing determination module 155, and working memory 135 represent one means for determining which post-processing functions to apply to the acquired images and when to apply such functions. In some embodiments, dynamically determined thresholds may be determined based on one or more factors such as, but not limited to, the rate at which the battery is depleted, how quickly the battery has been depleted during previous use, and/or What other processes are running on the electronic device or processor. Such thresholds may be dynamically determined for limited post-processing or delayed post-processing, or other processes and/or functionality described herein.

Memory 140 may also contain low power application modules 160 . The low power applications module 160 illustrated in FIG. 1 may include instructions that configure the processor 120 to switch from applications that consume a large amount of power to applications that consume a small amount of power based on remaining battery charge. Some embodiments incorporate information into, for example, a lookup table, a file, a database, or another hardware or software storage component (all of which are referred to as lookup tables for ease of reference) to control which application icon is launched when the user selects an application icon. application. In other embodiments, the operating process may use system attributes to identify whether to operate in low power mode or normal mode. For example, system properties may contain information about the current battery charge. Procedures can contain arguments that can be introduced to determine the mode of operation. In some embodiments, if the remaining battery charge is less than a predetermined threshold level, the processor 120 may be instructed by the low power application module 160 to access information stored in a lookup table that controls when the user selects an application icon. Whether to start a low power application or a full power application and start the appropriate application. Thus, processor 120 together with battery level determination module 150, low power application module 160 and working memory 135 represent one means for determining whether to launch a low power application or a full power application.

The memory 140 may also contain a user interface module 170 . The user interface module 170 illustrated in FIG. 1 may include instructions that configure the processor 120 to provide a set of soft controls for displaying objects and allowing a user to interact with the device. An operating system module 165 may also reside in the memory 140 and operate with the processor 120 to manage the memory and processing resources of the system 100 . For example, operating system 165 may include device drivers to manage hardware resources, such as electronic display 130 or imaging device 110 . In some embodiments, the instructions contained in battery level determination module 150 and post-processing determination module 155 may not directly interact with these hardware resources, but instead interact via standard subroutines or APIs located in operating system 165 . Instructions within operating system 165 may then directly interface with these hardware components.

The processor 120 can write data to the storage module 125 . Although storage module 125 is graphically represented as a conventional disk drive, those skilled in the art will appreciate that embodiments may include disk-based storage or one of several other types of storage media, including memory disks, USB drives, flash drives, remotely attached storage media, virtual disk drives, or the like.

Although FIG. 1 depicts a device that includes separate components to include a processor, imaging device, electronic display, and memory, those skilled in the art will recognize that these separate components can be combined in various ways to achieve particular design goals. For example, in alternative embodiments, memory components may be combined with processor components to save cost and improve performance.

Additionally, while FIG. 1 illustrates two memory components, including a memory component 140 comprising several modules and a separate memory 135 comprising working memory, those skilled in the art will recognize several embodiments utilizing different memory architectures. For example, a design may utilize ROM or static RAM memory for storing processor instructions that implement the modules contained in memory 140 . Alternatively, the processor instructions may be read at system startup from a disk storage device integrated into the power management system 100 or connected via an external device port. The processor instructions can then be loaded into RAM for execution by the processor. Working memory 135 may be, for example, RAM memory where instructions are loaded into working memory 135 prior to execution by processor 120 .

Method overview

Some of the functionality of the examples of embodiments described herein involves predicting how long the battery charge of an electronic device will last, and based on that prediction, performing full or limited functions on the electronic device, such as starting and running low power or full charge. power applications or perform full or limited post-processing of sensor data such as images. An instance may be described as a process, which may be described as a flowchart, flow diagram, finite state diagram, structure diagram, or block diagram. Although a flowchart may describe operations as a sequential process, many operations may be performed in parallel or simultaneously, and the process may be repeated. Additionally, the order of operations may be rearranged. A process terminates when its operations are complete. A process may correspond to a method, function, procedure, subroutine, subroutine, or the like. When a process corresponds to a software function, termination of the process corresponds to a return of the function to the calling or main function.

FIG. 2 illustrates one embodiment of a process 200 to configure an electronic device into a low power or full or normal power mode of operation, which may be implemented in one or more of the modules depicted in FIG. 1 . The low power mode of operation desirably consumes less power than the normal power mode of operation. The electronic device may be a handheld communication device such as a cellular telephone or "smart phone" or a mobile personal data assistant (PDA) including a tablet computer. In some examples, process 200 may run on a processor, such as processor 120 (FIG. 1), and run on other components illustrated in FIG. 1 stored in memory 140 or incorporated in other in hardware or software. Configuration process 200 begins at start block 202 and proceeds to block 204, where a user first indicates one or more low power operating preferences. In some embodiments, the user may select which feature or features of the electronic device may be limited during low power operation. For example, a user may indicate that post-processing of sensor data, such as camera data acquired by an imaging device such as imaging device 110 ( FIG. 1 ) of the electronic device, may be delayed if the device is in a low power mode of operation. In another example, a user may indicate that a low-power application, such as a less graphics-intensive game, rather than a full game or a more graphics-intensive version, may be launched during a low-power mode of operation. In other embodiments, the electronic device may run low power applications and processes by default. In this embodiment, the user can select which applications, features or processes to run in regular or high power mode. For example, a user may indicate that complete post-processing of an image is desired by selecting an image icon or text displayed in the photo library (as illustrated in FIG. 5 and discussed in more detail below). Process 200 then moves to block 206, where the user indicates a threshold beyond which the device has entered an "available but low power" mode. In some embodiments, the threshold may be a battery level threshold. The user may, via selection, indicate a battery level threshold below which the device will enter a low power mode of operation. In some embodiments, the battery level threshold may be 30% battery charge, 25% battery charge, 20% battery charge, or any other user-defined threshold battery charge percentage. In other embodiments, the user may indicate the time. The time threshold may represent the minimum amount of time a device must continue to function at the current power consumption. For example, a user may indicate that the device needs to operate at the current power consumption for a specified time to accommodate the user's itinerary, such as air travel. In another example, the user may indicate that the device needs to operate at the current power consumption for a specified time to accommodate use of the device while the user is in a business meeting. In another example, a user may indicate a time threshold when the user is in a theme park and will want to use the device to take a photo or video within a specified length of time. Once the user has indicated a threshold or battery level or time or both, process 200 moves to block 208 and ends.

After completing the configuration phase, such as upon completion of process 200 , in some embodiments the device may run a battery level determination process, such as process 300 shown in FIG. 3 . In some embodiments, process 300 may be used to estimate the amount of battery life remaining in an electronic device, or to predict how long the electronic device's battery charge will last. This prediction may be based on data representing historical utilization and charging patterns of the electronic device or from location indications or other factors. In some examples, process 300 may run on a processor, such as processor 120 (FIG. 1), and on other components illustrated in FIG. 1 stored in memory 140 or incorporated in other hardware or software. The battery level determination process 300 begins at start block 302 and proceeds to block 304 where a decision is made regarding the battery charge level of the electronic device. If the battery is low (as defined by the user-defined threshold established during configuration process 200), process 300 moves to block 310, where the device enters an "available but low power" mode. In this mode, features or applications may be limited or disabled, as identified by the user in the configuration process 200 described above. Additional details of the operation of the device during the low power mode will be discussed below. Process 300 then moves to block 314 and ends.

If the battery is not low (as defined by a user-defined threshold or a dynamically determined threshold), process 300 moves to block 306 where the device predicts whether it will drain the battery before the next charge. For example, if a user typically charges a device at 7pm, and the current time is 1pm, and the battery has 30% charge remaining, the device may determine based on the device's historical usage history that when operating in the current full power usage mode, the device will Drains the battery before the next charge cycle. If this is true, process 300 moves to block 310 where, as discussed above, the device enters an "available but low power" mode, and certain features or processes are disabled based on user preferences or device settings, as will be discussed below. Process 300 then moves to block 314 and may end.

If the device predicts that it will not drain the battery before the next expected charge cycle, process 300 proceeds to block 308 where a determination is made as to whether the user has specified that the time until the next charge cycle will be longer than normal. For example, if the user is traveling and selects a longer time threshold during the configuration phase outlined in process 200 and/or GPS readings from the electronic device indicate that the electronic device is far from a residential area, the device may drain the battery before the next charge cycle . In some embodiments, determining battery level and power consumption may include determining historical power consumption or current power consumption (eg, where the user is taking a large number of pictures in a short time frame). If this is true, process 300 moves to block 310 where, as discussed above, the device enters an "available but low power" mode, and certain features or processes are disabled based on user preferences or device settings, as will be discussed below. Process 300 then moves to block 314 and may end.

If the user has not specified that the time until the next charging cycle will be longer than normal, and the device has not determined that the time until the next charging cycle will be longer than normal, then process 300 moves to block 312, where the device enters or remains at full power model. In full power mode, no limitation of features or delayed post-processing is indicated. Process 300 then moves to block 314 and may end.

In some embodiments, predicting the time until the next charge cycle and determining whether the device will drain the battery before the next charge may be based on location information. For example, the user may be at home (as determined by GPS coordinates), and thus likely to recharge the device shortly. In other embodiments, the predicted time until the next charging cycle may also be based on time and date information. For example, a user may typically charge the device at 7pm every night. In other embodiments, the predicted time to the next charge cycle may be based on other historical device usage information.

operate in low power mode

When the device has alerted that power should be conserved, based on one of the battery level determination steps of process 300 discussed above, the device may run a low power operation process 400 , an example of which is shown in FIG. 4 . Operation in the low power mode of operation desirably consumes less power than operation in the normal power mode of operation. In some embodiments, process 400 may be used to delay post-processing of sensor data until the time the device is plugged in or fully charged. For example, if the user is at a theme park and takes multiple photos and videos, but has no opportunity to recharge the electronic device, the device may use the battery level determination process 300 described above to determine that the device may not have sufficient battery charge to continue operating in full power mode until the user can recharge the device. In this case, the device can switch from a full power mode, where sensor data is post-processed shortly after acquisition, to a low power mode, where minimal post-processing is performed on sensor data to conserve battery power so that the device can continue to operate until The user can recharge the device. In some embodiments, the device may run low power camera operations according to the steps outlined in process 400 . In some examples, process 400 may run on a processor, such as processor 120 (FIG. 1), and on other components illustrated in FIG. 1 stored in memory 140 or incorporated in other hardware or software.

The low power operation process 400 begins at START block 402 and proceeds to block 404 where the device operates in a low power mode. For example, in some embodiments, imaging device 110 and a camera application may operate in a low power mode. In the low power mode of operation, processes that utilize significant power, such as image processing functions including automatic white balance, CFA demosaicing, and storing processed images as JPEGs, can be delayed until the device's battery is fully charged or the device is plugged in until. In some embodiments, operation in the low power mode of operation shuts down or bypasses high power consuming processes of the image signal processor of processor 120 that may run during the normal power mode of operation.

Process 400 then moves to block 406 where sensor data is obtained. For example, sensor data may include still image data or video image data acquired by imaging device 110, audio data acquired by microphone 115, or any other additional sensor data, such as temperature or pressure. In other embodiments, other sensor data may also be obtained. Process 400 then moves to block 408 where the sensor data is stored in memory for later processing. Using a camera application as an example, in some embodiments, a low power application can configure imaging device 110 via an API to capture raw image data and store the data in memory (eg, memory 125 ) for later processing. For example, a low power camera application may not utilize a view finder, run auto white balance, run auto focus, or process raw BGGR Bayer data right away. Unlike a full-powered camera application that can use the camera's image processing pipeline to process raw BGGR data and store the image data as a JPEG file in memory, raw BGGR data acquired by imaging device 110 can be stored in memory and processed later . One example of a general image data processing pipeline 600 that may be bypassed when the device is in a low power mode of operation is shown in FIG. 6 . Typically, during a normal power mode of operation of an imaging system, significant pre- and post-processing is performed on the sensor data acquired by the image sensor. This typical processing operation is illustrated in FIG. 6 .

In a full power or normal power mode of operation, the pipeline 600 receives raw image data 602 from the camera sensor and preferably performs complete post-processing of the image data. Subsequent processing may include white balancing 606 , CFA demosaicing 608 , color conversion 610 , and color correction 612 of the image data before storing the image data in memory 616 . This typical process is indicated in Figure 6 by white arrows. As discussed above, these processing steps consume battery power and can be bypassed (as discussed with respect to process 400) to conserve battery power. For example, during a low power mode of operation, the device may bypass one or more steps of the complete image processing pipeline shown in FIG. 6 . In one embodiment, in one embodiment of the low power mode of operation, the processor may be instructed to perform limited pre-processing 604 of the raw image data 602 and then save the image data to memory 616, as shown by the solid black line in FIG. 620 instructions. In another embodiment, while the device is operating in another embodiment of the low power mode of operation, the processor may be instructed to immediately store the raw image data without performing any processing on the raw image data, as shown by Indicated by dashed line 622 . A command to post-process the image data, such as a direct command from the user or an instruction received by the processor 120 to enter high or normal power operation due to the device reaching or exceeding a battery level threshold, may trigger instructions to the processor 120 to turn the sensor The data is injected back into the Image Signal Processor (ISP) for post-processing. Once fired back into the ISP for post-processing, the sensor data can be fully processed, as indicated by the white arrows in FIG. 6 .

In some embodiments, thumbnail photos can be processed and displayed in the electronic device's photo library, with higher quality processing using more robust demosaicing filters, motion stabilization filters, skin tone correction filters, etc. while the device is charging or when the user manually selects a picture to share. In one example, a photo library of an electronic device such as a mobile phone may display text to the user indicating delayed post-processing of image data based on battery power, as shown in FIG. 5 . For example, in order to generate a camera preview image, the quality of the processed image data may be degraded to generate a temporary image while the photo is being taken. The temporary image may be formed by reducing the resolution of the displayed image using, for example, a nearest neighbor Bayer pattern demosaic filter or another image resolution process. This temporary image can be displayed in the user's photo library, as shown in FIG. 5 . The photo gallery can display instructional text to the user indicating that images will be displayed when the device is plugged in or fully charged. However, in other embodiments, the user may select text or an icon indicating that the image data has not been post-processed to request that the image be fully post-processed so that the image may be mailed, uploaded to a website, etc. This instructs the system to post-process the image and display a thumbnail in the photo gallery instead of an icon or text indicating that the image data has not been fully post-processed.

In another example, a microphone such as microphone 115 may be used to capture raw audio data. If the device is in low power mode, the system may store the raw audio data in a memory storage device (such as memory 125) for later use when the battery is fully charged, when the device is plugged in and charged, or at the user's request. Do postprocessing.

In yet another example, an imaging sensor such as imaging sensor 110 may record raw video data. If the device is in low power mode, the system can store the raw video data in a memory storage device (such as memory 125) for use when the device's battery is fully charged, when the device is plugged in and charged, or at the user's request. Process it afterwards.

Process 400 then moves to block 412, where a determination is made as to whether the battery charge of the electronic device is low. If the battery is low, process 400 moves to block 404 and the process repeats, as discussed above. However, if the device is charged or the device is plugged or charged, then process 400 moves to block 414 where the device may then perform post-processing of the raw image data. Post-processing may include applying a more robust demosaicing filter, applying a motion stabilization filter, applying a skin tone correction filter, among other post-processing filters and functions. Post-processing may include loading image data from memory and providing the image data to an image signal processor (ISP). In one embodiment, the processor may be instructed to load raw image data 630 and insert raw image data 630 into pipeline 600 as indicated by line 631 in FIG. 6 . Process 400 then moves to block 416 and ends.

In some embodiments, post-processing of sensor data may also be initiated if the user manually initiates an action that requires a full quality version of the data even when the battery is low. For example, if a user attempts to email an image, the device may perform higher quality post-processing of the image data.

In some embodiments, if the battery charge falls below a user-defined threshold or the device predicts that it will not have enough charge to last through the next charge cycle, the system can change a look-up table (LUT) that is defined when the user clicks Apply Which application to start when the program icon is displayed. In other embodiments, the operating process may use system attributes to identify whether to operate in low power mode or normal mode. In some embodiments, system properties may include an indication of battery charge. Procedures can contain arguments that can be introduced to detop modes of operation. For example, a low power version of an application such as a camera may be launched while the system is in a low power situation. In other embodiments, a low power version of the gaming application may also be launched while the device is in low power mode. Lower power versions of these applications may contain shorter games and/or less complex graphics. When the battery level is above a defined threshold or the device is charging, the LUT can be changed so that the application launched when the user selects the application icon is a full power operating application.

In another example, a processor of the electronic device (eg, processor 120) may receive instructions to operate the display of the electronic device (eg, electronic display 130) in a low power mode. In some embodiments, during a low power mode of operation, the color format may be changed, or the frames per second (fps) may be reduced to save battery power. For example, changing from the color format RGBX8888 at 60fps to the color format RGB565 at 30fps can reduce the amount of bandwidth consumed by the display processing pipeline. Additionally, this color format change and fps reduction may reduce the clock rate for hardware operations. In some embodiments, with an AMOLED display, the pixels of the electronic display can be dimmed to save battery power. In other embodiments, the screen resolution of the electronic display may be adjusted. For example, a 720p screen may be displayed at a lower resolution such that portions of the viewing surface of the electronic display are black.

Embodiments of this predictive battery life and latency post-processing method can be implemented on mobile devices such as phones, cameras (including plenoptic or light field cameras and stereo cameras), tablets, computers, heart rate monitors, and the like. These examples are intended to be illustrative, not limiting.

Clarification on Terminology

Unless otherwise indicated, any disclosure of the operation of an apparatus having a particular feature is also expressly intended to disclose a method of similar characteristics (and vice versa), and any disclosure of the operation of an apparatus according to a particular configuration is also expressly intended to disclose It is intended to disclose methods according to similar configurations (and vice versa). The term "configuration" may be used with reference to a method, apparatus and/or system as indicated by its particular context. The terms "method," "process," "procedure" and "technique" are used generically and interchangeably unless the specific context dictates otherwise. The terms "apparatus" and "device" are also used generically and interchangeably unless otherwise indicated by the particular context. The terms "element" and "module" are generally used to denote a portion of a larger configuration. Unless expressly limited by its context, the term "system" is used herein to indicate either of its ordinary meanings, including "a group of elements that interact to serve a common purpose." Any incorporation by reference of a section should also be understood to incorporate definitions for terms or variables referenced within that section, where such definitions appear elsewhere in the document, as well as incorporating definitions in the incorporated section. Any figure referenced in .

Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and process steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. Those skilled in the art will recognize that a portion or a portion may comprise less than or equal to the whole. For example, a portion of a set of pixels may refer to a subset of those pixels.

A general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other programmable logic device, discrete gates or transistors designed to perform the functions described herein can be used The various illustrative logical blocks, modules and circuits described in connection with the embodiments disclosed herein may be implemented or performed by logic, discrete hardware components, or any combination thereof. A general-purpose processor can be a microprocessor, but in the alternative, the processor can be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

The steps of methods or processes described in connection with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary computer-readable storage medium is coupled to the processor such that the processor can read information from, and write information to, the computer-readable storage medium. In the alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in the ASIC. The ASIC may reside in a user terminal, camera or other device. In alternative embodiments, the processor and storage medium may reside as discrete components in a user terminal, camera or other device.

This article contains several headings for reference and to aid in locating various sections. These headings are not intended to limit the scope of the concepts they describe. Such concepts may apply throughout the specification.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (30)

1., for postponing a system for the power consumption of electronic installation, described system includes:

Memory assembly, it is configured to store sensor data；

Being coupled to the processor of described memory assembly, described processor is configured to retrieve the institute from described memory assembly State sensing data, and process normal power operator scheme and sensing data process low-power operation comprising sensing data At least two operational mode of pattern performs the process of described sensing data, the merit that described low-power operating mode consumes Rate is less than described normal power operator scheme；And

Being stored in the control module in described memory assembly, described control module comprises some instructions, and described instruction is configured To operate described processor to carry out following operation:

Determine that low power condition, described low power condition are will to enter described low-power when less than it based on described electronic installation The threshold battery power of operator scheme or described device are after which by the threshold time entering described low-power operating mode At least one；And

Based on whether described low power condition occurs and operates described device in described low-power operating mode.

System the most according to claim 1, wherein operates described device in described low-power operating mode and includes passing Sensor data are stored in described memory assembly, and with in described normal power operator scheme, operate described device phase Ratio, performs the less process of described sensing data.

System the most according to claim 1, it farther includes imaging device, itself and described processor communication, and through joining Putting to produce view data, described imaging device includes at least one imaging sensor.

System the most according to claim 1, wherein said control module is further configured to accept for described electronics The user preference of the low-power operating mode of device.

System the most according to claim 1, wherein said control module is further configured be in described low-power During operator scheme, described sensing data is stored in described memory assembly for processing a little later.

System the most according to claim 5, wherein said control module is further configured to store described sensor number According to this for carrying out the high-quality post processing postponed during the charging cycle a little later of described electronic installation.

System the most according to claim 1, wherein said low-power operating mode comprises operation for light field or full light phase The camera application program of machine.

System the most according to claim 1, wherein said low-power operating mode comprises the operation phase for stereoscopic camera Machine application program.

System the most according to claim 1, wherein said low-power operating mode comprises operation heart rate monitor application journey Sequence.

System the most according to claim 1, wherein said low-power operating mode comprises operation audio application.

11. systems according to claim 1, wherein said low-power operating mode comprises the operation phase for mobile device Machine application program.

12. 1 kinds are used for the method postponing the power consumption of electronic installation, and described method includes:

Sensing data is stored in the memory assembly of described electronic installation；

Retrieve the described sensing data from described memory assembly, and process normal power operation comprising sensing data At least two operational mode of pattern and sensing data process low-power operating mode performs described sensing data Processing, the power that described low-power operating mode consumes operates less than described normal power；

Determine that low power condition, described low power condition are will to enter described low-power when less than it based on described electronic installation The threshold battery power of operator scheme or described device are after which by the threshold time entering described low-power operating mode At least one；And

Based on whether described low power condition occurs and operates described device in described low-power operating mode.

13. methods according to claim 12, wherein operate in described low-power operating mode described device include by Described sensing data is stored in described memory assembly, and operates described device with in described normal power operator scheme Time compare, perform the less process of described sensing data.

14. methods according to claim 12, it farther includes to accept the low-power operation for described electronic installation The user preference of pattern.

15. methods according to claim 12, it farther includes described sensing data is stored in described memorizer For carrying out post processing a little later during the low-power operation of described electronic installation in assembly.

16. methods according to claim 12, it farther includes described sensing data is stored in described electronics dress For carrying out the post processing that postpones during the charging cycle a little later of described electronic installation in the described memory assembly put.

17. methods according to claim 12, wherein said low-power operating mode comprises operation for light field or full light The camera application program of camera.

18. methods according to claim 12, wherein said low-power operating mode comprises operation for stereoscopic camera Camera application program.

19. methods according to claim 12, wherein said low-power operating mode comprises operation heart rate monitor application Program.

20. methods according to claim 12, wherein said low-power operating mode comprises operation audio application.

21. methods according to claim 12, wherein said low-power operating mode comprises operation for mobile device Camera application program.

22. 1 kinds of equipment being used for postponing the power consumption of electronic installation, described equipment includes:

Device for store sensor data；

For retrieving the described sensing data from described memory assembly, and process normal power comprising sensing data At least two operational mode of operator scheme and sensing data process low-power operating mode performs described sensor number According to the device of process, the power that described low-power operating mode consumes operates less than described normal power；

For determining the device of low power condition, described low power condition is will to enter when less than it based on described electronic installation The threshold battery power of described low-power operating mode or described device will enter described low-power operating mode after which At least one in threshold time；And

For based on whether described low power condition occurs and operates the device of described device in described low-power operating mode.

23. 1 kinds of non-transitory computer-readable medias, it stores some instructions, and described instruction makes at least one upon execution Physical computer processor performs the method postponing the power operation of electronic installation, and described method includes:

Sensing data is stored in the memory assembly of described electronic installation；

Retrieve the described sensing data from described memory assembly, and process normal power operation comprising sensing data At least two operational mode of pattern and sensing data process low-power operating mode performs described sensing data Processing, the power that described low-power operating mode consumes operates less than described normal power；

Determine that low power condition, described low power condition are will to enter described low-power when less than it based on described electronic installation The threshold battery power of operator scheme or described device are after which by the threshold time entering described low-power operating mode At least one；And

Based on whether described low power condition occurs and operates described device in described low-power operating mode.

24. non-transitory computer-readable medias according to claim 23, wherein in described low-power operating mode Operate described device and include that sensing data is stored in described memory assembly to be neutralized and in described normal power operator scheme Compare during the described device of middle operation, perform the less process of described sensing data.

25. non-transitory computer-readable medias according to claim 23, it farther includes to accept for described electricity The user preference of the described low-power operating mode of sub-device.

26. non-transitory computer-readable medias according to claim 23, it farther includes described sensor number High-quality for carry out postponing during the charging cycle a little later of described electronic installation according to being stored in described memory assembly Amount post processing.

27. non-transitory computer-readable medias according to claim 23, wherein said low-power operating mode comprises Operation is for light field or the camera application program of full light camera.

28. non-transitory computer-readable medias according to claim 23, wherein said low-power operating mode comprises Operation is for the camera application program of stereoscopic camera.

29. non-transitory computer-readable medias according to claim 23, wherein said low-power operating mode comprises Operation heart rate monitor application program.

30. non-transitory computer-readable medias according to claim 23, wherein said low-power operating mode comprises Operation audio application.