US20130329092A1 - Intelligent Auto-Exposure Bracketing - Google Patents
Intelligent Auto-Exposure Bracketing Download PDFInfo
- Publication number
- US20130329092A1 US20130329092A1 US13/490,099 US201213490099A US2013329092A1 US 20130329092 A1 US20130329092 A1 US 20130329092A1 US 201213490099 A US201213490099 A US 201213490099A US 2013329092 A1 US2013329092 A1 US 2013329092A1
- Authority
- US
- United States
- Prior art keywords
- image
- low
- processor
- cause
- bracket setting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/743—Bracketing, i.e. taking a series of images with varying exposure conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
Definitions
- This disclosure relates generally to the field of digital image processing. More particularly, this disclosure relates to auto-exposure bracketing operations.
- Many digital cameras provide a user with an auto-exposure bracketing (AEB) option. Under this option, a number of pictures (generally 3, 5 or 7) are captured at different exposures settings, which are typically uniformly spaced. Under extremely challenging imaging/auto-exposure conditions, users can employ the AEB option to increase the probability of acquiring an optimally exposed picture.
- AEB auto-exposure bracketing
- AEB high dynamic range
- HDR high dynamic range
- the full dynamic range of a scene needs to be captured (specifically, highlight and shadow information).
- the dynamic range of a scene often exceeds the dynamic range of the camera's imaging sensor, which are typically limited to capturing 256 (8 bits) to 1024 (10 bits) levels of brightness.
- AEB may be employed.
- the most commonly employed brackets are: 2EV ⁇ , EV 0 , 2EV+ and 3EV ⁇ , EV 0 , 3EV+.
- EV stands for exposure value and refers to a combination of the camera's shutter speed and aperture setting.
- the EV 0 image refers to an image captured using an exposure value as determined by the camera's auto-exposure (AE) algorithm
- EV ⁇ refers to an image captured at a lower stop (e.g., 2 or 3)
- EV+ refers to an image captured at a higher stop (e.g., 2 or 3).
- fixed bracketing schemes such as the two listed above
- the brackets are set too large, the likelihood of excessive noise and motion blur in the lower and upper bracketed images increases. If the brackets are set too small, the likelihood of clipped information in the lower and upper bracketed images increases.
- the inventive concept provides a method to adaptively change the lower and upper bracket settings during AEB operations.
- the method begins by capturing first neutral (EV 0 ), low and high bracketed images.
- a neutral image is one that is captured using an exposure value provided by an image capture device's auto-exposure process.
- a second neutral image may be obtained and evaluated against the first neutral image. If the two primary images are determined to be consistent, quality metrics may be determined for one or both of the high and low exposure bracket settings and used to establish new exposure bracket settings. The new exposure bracket settings may then be used to capture new neutral, low and high bracketed images.
- the collection of images so captured may be fused to generate a composite image (either immediately or later).
- a computer executable program to implement the method may be stored in any media that is readable and executable by a programmable control device and may be in any desired computer programming language.
- the disclosed capabilities may also be implemented in an electronic device having the ability to capture digital images.
- lower and upper bracket settings may be adaptively changed by obtaining first low and high images using first low and high exposure bracket settings.
- a histogram may be obtained for either or both of these low and high images based, for example, on luminescence or the R, G, B color channels.
- quality metrics may be determined for one or both of the first low and high images (e.g., based on their histograms). If the quality metrics so indicate, one or both of the low and high exposure bracket settings may be adjusted. For example, if either of the first low or high images have an excessive number of blown-out pixels, the respective exposure bracket setting may be adjusted to compensate for this condition.
- a new set of images based on the adjusted low and high exposure bracket settings (as well as a corresponding neutral image) may be captured and fused (immediately or later) into a composite image.
- Computer executable programs to implement the disclosed methods may be stored in any media that is readable and executable by a programmable control device. Such programs may be implemented in any desired computer programming language. In addition, the disclosed capabilities may also be implemented in an electronic device having the ability to capture digital images.
- FIGS. 1A and 1B show, in flowchart form, an intelligent auto-exposure bracketing operation in accordance with one embodiment.
- FIG. 2 shows an illustrative intelligent auto-exposure bracketing timeline in accordance with FIGS. 1A and 1B .
- FIG. 3 shows, in flowchart form, an intelligent auto-exposure bracketing operation in accordance with another embodiment.
- FIG. 4 shows an illustrative intelligent auto-exposure bracketing timeline in accordance with FIG. 3 .
- FIG. 5 shows, in block diagram form, an image processing pipeline in accordance with one embodiment.
- FIG. 6 shows, in block diagram form, an electronic device in accordance with one embodiment.
- This disclosure pertains to systems, methods, and computer readable media for implementing methods to adaptively select bracket settings during AEB operations.
- techniques are disclosed for adjusting AEB settings for a current image based on the evaluation of prior bracketed images.
- the current EV 0 image may be compared with a prior EV 0 image. If the two are determined to be consistent, the prior image's EV ⁇ and/or EV+ images may be used to adaptively change the bracket setting for the current image.
- the EV 0 image refers to the image captured using the exposure value as determined by the camera's auto-exposure (AE) mechanism.
- AE auto-exposure
- the EV ⁇ image refers to that image captured by the lower f-stop setting in an AEB capture operation
- the EV+ image refers to that image captured by the higher f-stop setting in an AEB capture operation.
- the term “consistent” refers to the concept of two images being similar. Information or parameters that may be used to determine if two images are consistent can include, but are not limited to: timestamp; camera location (e.g., via GPS data); camera directional information; camera exposure; and color histogram data. More particularly, techniques disclosed herein may analyze a prior image's EV ⁇ and/or EV+ histograms.
- EV ⁇ bracket exposure settings may be adjusted so as to reduce the number pixels that are too bright and/or too dark.
- the resulting image may have an improved dynamic range over that obtained using conventional AEB techniques.
- preview-mode In many modern digital capture devices, there is a state known as “preview-mode” during which the device repeatedly obtains images from it's sensor and displays those for a user to see. It will be recognized that the displayed image is often smaller than the sensor's capture capacity (e.g., 1 or 2 MPix versus 5 or 8 MPix). While the specific rate at which the preview image is obtained and displayed may be dependent upon the particular type of sensor element (e.g., charged-coupled device or complementary metal-oxide semiconductor) and supporting circuitry, typical display rates are currently between 15 and 30 frames-per-second.
- sensor element e.g., charged-coupled device or complementary metal-oxide semiconductor
- illustrative iAEB operation 100 begins when an image capture device is placed into HDR preview mode 200 (block 105 ). At that time, the device begins to obtain and display preview images 205 taken at some rate (e.g., 15 to 30 frames-per-second) and some exposure as determined by the device's AE mechanism.
- some rate e.g. 15 to 30 frames-per-second
- the term “obtained” means that the image capture device's sensor has acquired an image without an intent to store the image in long-term memory/storage.
- each preview image may also include metadata of the sort that may be used to determine its consistency with other images.
- metadata may be incorporated within the image's file structure or be retained in a separate data store with a reference between the two (e.g., using one- or two-way pointers).
- Consistency data may include, but is not limited to: timestamp; camera location from, for example, GPS data; camera directional information such as that available from accelerometer and/or gyroscopic sensors; camera exposure; and color histogram data (which is frequently used in QBIC and CBIR systems to determine similarity/dis-similarity between image pairs).
- first HDR capture event 210 occurs (block 110 ).
- first HDR capture event 210 causes three images to be acquired: first EV ⁇ image 215 , first EV 0 image 220 , and first EV+ image 225 .
- the device may return to HDR preview mode 200 (block 115 ).
- HDR images 215 , 220 and 225 may be fused into a single HDR image (shown in dashed line fashion).
- second HDR capture event 230 may occur that begins an intelligent bracket adjustment operation in accordance with this disclosure (block 120 ).
- most recently obtained EV 0 image 235 may be evaluated to determine if it is consistent to prior captured EV 0 image 220 (block 125 ).
- two images may be considered consistent if they are taken within 10 seconds of one another and have exposure values differing by no more than 1 ⁇ 2 stop.
- two images may be considered consistent if the image capture device's orientation changed less than ⁇ 5° between the time the two images were captured.
- two images may be considered consistent if they were taken at the same location.
- combinations of these characteristics may be used; the precise selection of which will depend upon the developer's specific goals.
- prior captured EV ⁇ and/or EV+ images 215 and 225 may be evaluated (block 130 ).
- evaluation in accordance with block 130 may include determining how many blown-out pixels are in prior captured EV ⁇ and/or EV+ images 215 and 225 .
- this information in the form of histograms, may be obtained. Histogram information may come in a variety of forms such as, for example, luminance (e.g., when operating in the YUV/YCbCr colorspace) or color (e.g., red, green, or blue when operating in the RGB colorspace).
- any pixel having a value greater than X (e.g., in EV ⁇ image 215 ) or less than Y (e.g., in EV+ image 225 ) may be counted as “blown-out.”
- Values for X and Y may be selected by a designer to meet the particular needs and constraints of their implementation (e.g., 251 or 255 for X and 0 or 6 for Y).
- the precise number of blown-out pixels needed to be considered “bad,” may be influenced by a number of factors. Illustrative factors include, but are not limited to, the size of image being captured (e.g., 3 MPix versus 10 MPix), where during image processing histogram data is obtained (see discussion below), the image's exposure value, and the particular sensor and image processing pipeline within the device being used to generate the images.
- the number of blown-out pixels may be compared to a threshold and if that threshold is exceeded (the “YES” prong of block 135 ), the brackets may be adjusted (block 140 ).
- a threshold exceeded (the “YES” prong of block 135 )
- the brackets may be adjusted (block 140 ).
- an image sensor that returns 8-bit quantities (0 to 255). If the EV ⁇ image has a large number of blown-out high pixels (e.g., pixels having a value equal to or greater than a first specified value), this may be at least partially compensated for by stepping the bracket for the EV ⁇ image down (e.g., from ⁇ 1 EV to ⁇ 1.5 EV, or from ⁇ 2 EV to ⁇ 3 EV).
- the EV+ image has a large number of blown-out dark pixels (e.g., pixels having a value less than or equal to a second specified value)
- this may be at least partially compensated for by stepping the bracket for the EV+ image up (e.g., from +1 EV to +1.5 EV, or from +2 EV to +3 EV).
- upper and lower bracket settings may be adjusted independently: e.g., ⁇ 1 EV to ⁇ 2 EV for the EV ⁇ image and +2 EV to +2.5 EV for the EV+ image.
- only one side of an AEB bracket need be adjusted.
- second HDR image may be obtained (block 145 ), resulting in second EV ⁇ image 240 , second EV 0 image 245 , and second EV+ image 250 .
- AEB brackets may be set to their default value (block 150 ), where after iAEB operation 100 continues at block 145 . If evaluation in accordance with block 135 determines the current exposure brackets are good (the “NO” prong of block 135 ), the AEB brackets may be left at their current value (block 155 ), where after iAEB operation 100 continues at block 145 . In one embodiment, after each HDR image capture the EV ⁇ and EV+ images may be returned to their default setting (e.g., ⁇ 2 EV, or ⁇ 3 EV).
- the AEB bracket settings may be left at whatever value they were last set to.
- AEB brackets may initially be left at their current values, reverting to their default values after a stipulated event (e.g., the elapse of a specified time, a change in image capture device orientation or location).
- HDR images 240 , 245 and 250 may be fused into a single HDR image (shown in dashed line fashion). If it is determined that HDR capture activities are complete (the “YES” prong of block 160 ), HDR preview mode 200 may be left (block 165 ).
- the second HDR image set (images 240 , 245 and 250 ) may be made the first HDR image set (block 170 ), where after a wait state may be entered until another HDR capture event is detected (block 175 ).
- iAEB operation 100 continues at block 120 .
- the HDR preview mode may obtain EV ⁇ , EV 0 , and EV+ images in a cyclic fashion.
- each HDR image capture may evaluate the most recent EV ⁇ and EV+ images rather than looking back to the last captured HDR image, thereby allowing the HDR capture process to use the most up-to-date exposure information.
- Another advantage of this approach is that consistency checks between current and prior EV 0 images may be eliminated. While most any sequence of EV ⁇ , EV 0 , and EV+ images may be employed, only the EV 0 image is generally shown on an image capture device's display element.
- illustrative iAEB operation 300 cyclically obtains EV ⁇ , EV 0 , and EV+ images.
- example iAEB operation 300 obtains an EV ⁇ image followed by an EV 0 image followed by an EV+ image.
- a wait state may be entered (block 310 ).
- the image capture device obtains EV ⁇ , EV 0 , and EV+ images at some rate (e.g., 20 frames-per-second), with EV 0 images being captured using the exposure setting determined by the device's AE mechanism.
- the most recently obtained EV ⁇ and EV+ images 415 may be evaluated (block 315 ).
- Evaluation in accordance with block 315 may include determining how many blown-out pixels are in prior obtained EV ⁇ and/or EV+ images 415 . In practice this information may be generated by camera firmware and attached to each image as it moves along the image processing pipeline (see discussion below with respect to FIG. 5 ). Once this is done, a determination may be made as to whether the current brackets are “good” (block 320 ). If evaluation of EV ⁇ and/or EV+ images 415 determines that one or both bracket settings should to be adjusted (the “YES” prong of block 320 ), such adjustment may be made (block 325 ).
- first EV ⁇ image 420 , first EV 0 image 425 and first EV+ image 430 may be captured (block 330 ). If, on the other hand, the bracket settings used to capture prior EV ⁇ and EV+ images 415 are determined to be “good” (the “NO” prong of block 320 ), EV ⁇ , EV 0 , and EV+ images 420 - 430 may be captured immediately (block 330 ). In one embodiment, following the capture of HDR image set 420 - 430 in accordance with block 330 , the bracket settings used to obtain subsequent images 435 may be default bracket settings.
- the bracket settings used to obtain subsequent images 435 may be the “current” bracket settings. In this latter embodiment, if block 330 is arrived at via the “NO” prong of block 320 , the bracket settings used to obtain subsequent images 435 will be the same bracket settings as was used to obtain prior images 415 . However, if block 330 was arrived at via the “YES” prong of block 320 , subsequent images 435 may be obtained using the bracket settings set in accordance with block 325 .
- HDR mode may be terminated by an explicit act of a user (e.g., the user actively turns HDR mode off). In another embodiment, HDR mode may be terminated automatically in accordance with a specified event (e.g., the passage of time). If it is determined that the HDR mode should be terminated (the “YES” prong of 335 ), it is so terminated (block 340 ). If it is determined that the HDR mode should not be terminated (the “NO” prong of 335 ), iAEB operation 300 continues at block 310 .
- second HDR image capture event 440 may occur. Again, the most recent EV ⁇ and EV+ images 445 may be evaluated (block 315 ) for purposes of determining what bracket setting should be used to capture second HDR image set 450 (blocks 320 - 330 ). Following capture of HDR image set 450 , iAEB operation 300 continues at block 335 as described above.
- illustrative image capture subsystem 500 such as may be found in a mobile device having an embedded camera—includes sensor 505 and image processing pipeline 510 , itself composed of early and late stage processing units 515 and 520 .
- image subsystem 500 may be composed of hardware (e.g., special and/or general purpose processing units), software (e.g., firmware), or a combination hardware and software.
- Raw sensor data 525 may be initially operated upon by early stage processing unit 515 .
- unit 515 may provide dead pixel correction (and other sensor and/or lens accommodation operations), and white balancing functions.
- image capture subsystem 500 may provide clipped pixel information 530 ; typically in the form of a histogram. This information may be used to determine the number of pixels in EV ⁇ and/or EV+ that are blown-out (see discussion above). Further processing by unit 520 may provide image sharpening, gamma correction, and noise reduction. Output from image subsystem 500 may include multiple images 535 (e.g., EV ⁇ , EV 0 , and EV+ images). Once obtained in accordance with this disclosure, image set 535 may be fused to generate a single HDR output image or saved to memory/storage for later fusion or other processing. It will be recognized that clipped pixel information may also be obtained from image processing pipeline 510 output (e.g., from images 535 ).
- the threshold used to determine whether the brackets used to capture prior EV ⁇ and EV+ images (e.g., 215 , 225 , 240 and 250 in FIGS. 2 or 415 and 445 in FIG. 4 ) are good or bad in accordance with blocks 130 - 135 (see FIG. 1 ) or 315 - 320 (see FIG. 3 ) may be different.
- the thresholds may be a function (e.g., a percentage) of the number of pixels provided by sensor 505 .
- the threshold applied to an EV ⁇ image may be different from that applied to an EV+ image.
- Electronic device 600 may include processor 605 , display 610 , user interface 615 , graphics hardware 620 , device sensors 625 (e.g., proximity sensor/ambient light sensor, accelerometer and/or gyroscope), microphone 630 , audio codec(s) 635 , speaker(s) 640 , communications circuitry 645 , digital image capture unit 650 , video codec(s) 655 , memory 660 , storage 665 , and communications bus 670 .
- Electronic device 600 may be, for example, a personal digital assistant (PDA), personal music player, a mobile telephone, or a notebook, laptop or tablet computer system.
- PDA personal digital assistant
- Processor 605 may execute instructions necessary to carry out or control the operation of many functions performed by device 600 (e.g., such as the generation and/or processing of images in accordance with operations 100 and 300 ).
- Processor 605 may, for instance, drive display 610 and receive user input from user interface 615 .
- User interface 615 can take a variety of forms, such as a button, keypad, dial, a click wheel, keyboard, display screen and/or a touch screen.
- Processor 605 may be a system-on-chip such as those found in mobile devices and include a dedicated graphics processing unit (GPU).
- GPU graphics processing unit
- Processor 605 may be based on reduced instruction-set computer (RISC) or complex instruction-set computer (CISC) architectures or any other suitable architecture and may include one or more processing cores.
- Graphics hardware 620 may be special purpose computational hardware for processing graphics and/or assisting processor 605 process graphics information.
- graphics hardware 620 may include a programmable graphics processing unit (GPU).
- Sensor and camera circuitry 650 may capture still and video images that may be processed to generate images in accordance with this disclosure and may, for example, incorporate image capture subsystem 500 .
- Output from camera circuitry 650 may be processed, at least in part, by video codec(s) 655 and/or processor 605 and/or graphics hardware 620 , and/or a dedicated image processing unit incorporated within circuitry 650 .
- Images so captured may be stored in memory 660 and/or storage 665 .
- Memory 660 may include one or more different types of media used by processor 605 , graphics hardware 620 , and image capture circuitry 650 to perform device functions.
- memory 660 may include memory cache, read-only memory (ROM), and/or random access memory (RAM).
- Storage 665 may store media (e.g., audio, image and video files), computer program instructions or software, preference information, device profile information, and any other suitable data.
- Storage 665 may include one more non-transitory storage mediums including, for example, magnetic disks (fixed, floppy, and removable) and tape, optical media such as CD-ROMs and digital video disks (DVDs), and semiconductor memory devices such as Electrically Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM).
- Memory 660 and storage 665 may be used to retain computer program instructions or code organized into one or more modules and written in any desired computer programming language. When executed by, for example, processor 605 such computer program code may implement one or more of the methods described herein.
- the capture of EV ⁇ and EV+ images are not to be limited to one each.
- 2 or more EV ⁇ images (using different exposure values) and/or two or more EV+ images (using different exposure values) may be used.
- many different image capture orders may be used.
- the EV 0 image may be captured first followed by the EV ⁇ image followed by the EV+ image.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
- Exposure Control For Cameras (AREA)
Abstract
Description
- This disclosure relates generally to the field of digital image processing. More particularly, this disclosure relates to auto-exposure bracketing operations. Many digital cameras provide a user with an auto-exposure bracketing (AEB) option. Under this option, a number of pictures (generally 3, 5 or 7) are captured at different exposures settings, which are typically uniformly spaced. Under extremely challenging imaging/auto-exposure conditions, users can employ the AEB option to increase the probability of acquiring an optimally exposed picture.
- Another use of the AEB option is during high dynamic range (HDR) imaging. In order to create a high quality HDR image, the full dynamic range of a scene needs to be captured (specifically, highlight and shadow information). Unfortunately, the dynamic range of a scene often exceeds the dynamic range of the camera's imaging sensor, which are typically limited to capturing 256 (8 bits) to 1024 (10 bits) levels of brightness. In order to address this shortcoming, AEB may be employed. The most commonly employed brackets are: 2EV−, EV0, 2EV+ and 3EV−, EV0, 3EV+. Here, “EV” stands for exposure value and refers to a combination of the camera's shutter speed and aperture setting. The EV0 image refers to an image captured using an exposure value as determined by the camera's auto-exposure (AE) algorithm, EV− refers to an image captured at a lower stop (e.g., 2 or 3), and EV+ refers to an image captured at a higher stop (e.g., 2 or 3). In practice, fixed bracketing schemes (such as the two listed above), are not able to capture quality images for all possible illumination conditions. If the brackets are set too large, the likelihood of excessive noise and motion blur in the lower and upper bracketed images increases. If the brackets are set too small, the likelihood of clipped information in the lower and upper bracketed images increases.
- In one embodiment the inventive concept provides a method to adaptively change the lower and upper bracket settings during AEB operations. The method begins by capturing first neutral (EV0), low and high bracketed images. (As used herein, a neutral image is one that is captured using an exposure value provided by an image capture device's auto-exposure process.) At some later time, a second neutral image may be obtained and evaluated against the first neutral image. If the two primary images are determined to be consistent, quality metrics may be determined for one or both of the high and low exposure bracket settings and used to establish new exposure bracket settings. The new exposure bracket settings may then be used to capture new neutral, low and high bracketed images. The collection of images so captured may be fused to generate a composite image (either immediately or later). A computer executable program to implement the method may be stored in any media that is readable and executable by a programmable control device and may be in any desired computer programming language. The disclosed capabilities may also be implemented in an electronic device having the ability to capture digital images.
- In another embodiment, lower and upper bracket settings may be adaptively changed by obtaining first low and high images using first low and high exposure bracket settings. A histogram may be obtained for either or both of these low and high images based, for example, on luminescence or the R, G, B color channels. When an indication is received to capture an image afterwards, quality metrics may be determined for one or both of the first low and high images (e.g., based on their histograms). If the quality metrics so indicate, one or both of the low and high exposure bracket settings may be adjusted. For example, if either of the first low or high images have an excessive number of blown-out pixels, the respective exposure bracket setting may be adjusted to compensate for this condition. A new set of images based on the adjusted low and high exposure bracket settings (as well as a corresponding neutral image) may be captured and fused (immediately or later) into a composite image.
- Computer executable programs to implement the disclosed methods may be stored in any media that is readable and executable by a programmable control device. Such programs may be implemented in any desired computer programming language. In addition, the disclosed capabilities may also be implemented in an electronic device having the ability to capture digital images.
-
FIGS. 1A and 1B show, in flowchart form, an intelligent auto-exposure bracketing operation in accordance with one embodiment. -
FIG. 2 shows an illustrative intelligent auto-exposure bracketing timeline in accordance withFIGS. 1A and 1B . -
FIG. 3 shows, in flowchart form, an intelligent auto-exposure bracketing operation in accordance with another embodiment. -
FIG. 4 shows an illustrative intelligent auto-exposure bracketing timeline in accordance withFIG. 3 . -
FIG. 5 shows, in block diagram form, an image processing pipeline in accordance with one embodiment. -
FIG. 6 shows, in block diagram form, an electronic device in accordance with one embodiment. - This disclosure pertains to systems, methods, and computer readable media for implementing methods to adaptively select bracket settings during AEB operations. In general, techniques are disclosed for adjusting AEB settings for a current image based on the evaluation of prior bracketed images. In some embodiments, the current EV0 image may be compared with a prior EV0 image. If the two are determined to be consistent, the prior image's EV− and/or EV+ images may be used to adaptively change the bracket setting for the current image. As previously noted, the EV0 image refers to the image captured using the exposure value as determined by the camera's auto-exposure (AE) mechanism. The EV− image refers to that image captured by the lower f-stop setting in an AEB capture operation, and the EV+ image refers to that image captured by the higher f-stop setting in an AEB capture operation. As used here, the term “consistent” refers to the concept of two images being similar. Information or parameters that may be used to determine if two images are consistent can include, but are not limited to: timestamp; camera location (e.g., via GPS data); camera directional information; camera exposure; and color histogram data. More particularly, techniques disclosed herein may analyze a prior image's EV− and/or EV+ histograms. Those that contain a significant number of pixels that are either too bright or too dark (Le., are “blown-out”) may be taken as indicative that a change in bracket exposures are needed. In such cases, either or both EV− and EV+ bracket exposure settings may be adjusted so as to reduce the number pixels that are too bright and/or too dark. When the EV−, EV0, and EV+ images are fused (such as during HDR operations), the resulting image may have an improved dynamic range over that obtained using conventional AEB techniques.
- In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the inventive concept. As part of this description, some of this disclosure's drawings represent structures and devices in block diagram form in order to avoid obscuring the invention. In the interest of clarity, not all features of an actual implementation are described in this specification. Moreover, the language used in this disclosure has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such inventive subject matter. Reference in this disclosure to “one embodiment” or to “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention, and multiple references to “one embodiment” or “an embodiment” should not be understood as necessarily all referring to the same embodiment.
- It will be appreciated that in the development of any actual implementation (as in any development project), numerous decisions must be made to achieve the developers' specific goals (e.g., compliance with system- and business-related constraints), and that these goals may vary from one implementation to another. It will also be appreciated that such development efforts might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the design an implementation of image processing systems having the benefit of this disclosure.
- In many modern digital capture devices, there is a state known as “preview-mode” during which the device repeatedly obtains images from it's sensor and displays those for a user to see. It will be recognized that the displayed image is often smaller than the sensor's capture capacity (e.g., 1 or 2 MPix versus 5 or 8 MPix). While the specific rate at which the preview image is obtained and displayed may be dependent upon the particular type of sensor element (e.g., charged-coupled device or complementary metal-oxide semiconductor) and supporting circuitry, typical display rates are currently between 15 and 30 frames-per-second.
- One application of an intelligent auto-exposure bracketing (iAEB) operation in accordance with this disclosure is during HDR acquisition. Referring to
FIGS. 1A , 1B, and 2,illustrative iAEB operation 100 begins when an image capture device is placed into HDR preview mode 200 (block 105). At that time, the device begins to obtain anddisplay preview images 205 taken at some rate (e.g., 15 to 30 frames-per-second) and some exposure as determined by the device's AE mechanism. As used herein, the term “obtained” means that the image capture device's sensor has acquired an image without an intent to store the image in long-term memory/storage. In contrast, the term “captured” means that the image capture device's sensor has acquired an image with an intent of storing it in long-term memory/storage. Along with the image data itself, each preview image may also include metadata of the sort that may be used to determine its consistency with other images. Such metadata may be incorporated within the image's file structure or be retained in a separate data store with a reference between the two (e.g., using one- or two-way pointers). Consistency data may include, but is not limited to: timestamp; camera location from, for example, GPS data; camera directional information such as that available from accelerometer and/or gyroscopic sensors; camera exposure; and color histogram data (which is frequently used in QBIC and CBIR systems to determine similarity/dis-similarity between image pairs). - At some later point in time, first
HDR capture event 210 occurs (block 110). In the illustrated embodiment, firstHDR capture event 210 causes three images to be acquired: first EV−image 215,first EV0 image 220, andfirst EV+ image 225. After capturing this first set of HDR images, the device may return to HDR preview mode 200 (block 115). In addition, 215, 220 and 225 may be fused into a single HDR image (shown in dashed line fashion). At yet a later time, secondHDR images HDR capture event 230 may occur that begins an intelligent bracket adjustment operation in accordance with this disclosure (block 120). Onevent 230, most recently obtainedEV0 image 235 may be evaluated to determine if it is consistent to prior captured EV0 image 220 (block 125). In one embodiment, two images may be considered consistent if they are taken within 10 seconds of one another and have exposure values differing by no more than ½ stop. In another embodiment, two images may be considered consistent if the image capture device's orientation changed less than ±5° between the time the two images were captured. In still another embodiment, two images may be considered consistent if they were taken at the same location. In yet other embodiments, combinations of these characteristics may be used; the precise selection of which will depend upon the developer's specific goals. If most currently obtainedEV0 image 235 is consistent with prior captured EV0 image 220 (the “YES” prong of block 125), prior captured EV− and/or 215 and 225 may be evaluated (block 130). In one embodiment, evaluation in accordance withEV+ images block 130 may include determining how many blown-out pixels are in prior captured EV− and/or 215 and 225. In many image capture devices this information, in the form of histograms, may be obtained. Histogram information may come in a variety of forms such as, for example, luminance (e.g., when operating in the YUV/YCbCr colorspace) or color (e.g., red, green, or blue when operating in the RGB colorspace). In one embodiment, any pixel having a value greater than X (e.g., in EV− image 215) or less than Y (e.g., in EV+ image 225) may be counted as “blown-out.” Values for X and Y may be selected by a designer to meet the particular needs and constraints of their implementation (e.g., 251 or 255 for X and 0 or 6 for Y).EV+ images - Once the number of blown-out pixels in prior captured EV− and/or
215 and 225 are known, a determination may be made as to whether the AEB brackets used to obtain them were “good” (block 135). The precise number of blown-out pixels needed to be considered “bad,” may be influenced by a number of factors. Illustrative factors include, but are not limited to, the size of image being captured (e.g., 3 MPix versus 10 MPix), where during image processing histogram data is obtained (see discussion below), the image's exposure value, and the particular sensor and image processing pipeline within the device being used to generate the images. Accordingly, the number of blown-out pixels may be compared to a threshold and if that threshold is exceeded (the “YES” prong of block 135), the brackets may be adjusted (block 140). Consider, for example, an image sensor that returns 8-bit quantities (0 to 255). If the EV− image has a large number of blown-out high pixels (e.g., pixels having a value equal to or greater than a first specified value), this may be at least partially compensated for by stepping the bracket for the EV− image down (e.g., from −1 EV to −1.5 EV, or from −2 EV to −3 EV). Similarly, if the EV+ image has a large number of blown-out dark pixels (e.g., pixels having a value less than or equal to a second specified value), this may be at least partially compensated for by stepping the bracket for the EV+ image up (e.g., from +1 EV to +1.5 EV, or from +2 EV to +3 EV). In another embodiment, upper and lower bracket settings may be adjusted independently: e.g., −1 EV to −2 EV for the EV− image and +2 EV to +2.5 EV for the EV+ image. In yet another embodiment, only one side of an AEB bracket need be adjusted. For example, if a particular embodiment experiences most bracketing problems due to blown-out bright pixels, only the EV− image need be evaluated or changed. Similarly, if a particular embodiment experiences most bracketing problems due to blown-out dark pixels, only the EV+ image need be evaluated or changed. Once the bracket(s) are adjusted in accordance withEV+ images block 140, second HDR image may be obtained (block 145), resulting in second EV−image 240,second EV0 image 245, andsecond EV+ image 250. - If most currently obtained
EV0 image 235 is not consistent with prior captured EV0 image 220 (the “NO” prong of block 125), AEB brackets may be set to their default value (block 150), where afteriAEB operation 100 continues atblock 145. If evaluation in accordance withblock 135 determines the current exposure brackets are good (the “NO” prong of block 135), the AEB brackets may be left at their current value (block 155), where afteriAEB operation 100 continues atblock 145. In one embodiment, after each HDR image capture the EV− and EV+ images may be returned to their default setting (e.g., ±2 EV, or ±3 EV). In another embodiment, the AEB bracket settings may be left at whatever value they were last set to. In yet another embodiment, AEB brackets may initially be left at their current values, reverting to their default values after a stipulated event (e.g., the elapse of a specified time, a change in image capture device orientation or location). - Once second HDR image set 240, 245 and 250 have been obtained in accordance with
block 145, a check may be made to determine if HDR capture activities are complete (block 160). In addition, 240, 245 and 250 may be fused into a single HDR image (shown in dashed line fashion). If it is determined that HDR capture activities are complete (the “YES” prong of block 160),HDR images HDR preview mode 200 may be left (block 165). On the other hand, if it is determined that HDR capture activities are not complete (the “NO” prong of block 160), the second HDR image set ( 240, 245 and 250) may be made the first HDR image set (block 170), where after a wait state may be entered until another HDR capture event is detected (block 175). When such an event occurs,images iAEB operation 100 continues atblock 120. - In another embodiment, the HDR preview mode may obtain EV−, EV0, and EV+ images in a cyclic fashion. One benefit of this approach is that each HDR image capture may evaluate the most recent EV− and EV+ images rather than looking back to the last captured HDR image, thereby allowing the HDR capture process to use the most up-to-date exposure information. Another advantage of this approach is that consistency checks between current and prior EV0 images may be eliminated. While most any sequence of EV−, EV0, and EV+ images may be employed, only the EV0 image is generally shown on an image capture device's display element.
- Referring to
FIGS. 3 and 4 ,illustrative iAEB operation 300 cyclically obtains EV−, EV0, and EV+ images. As shown intimeline 400,example iAEB operation 300 obtains an EV− image followed by an EV0 image followed by an EV+ image. In this embodiment, once the image capture device is placed into HDR preview mode 405 (block 305), a wait state may be entered (block 310). During this time, the image capture device obtains EV−, EV0, and EV+ images at some rate (e.g., 20 frames-per-second), with EV0 images being captured using the exposure setting determined by the device's AE mechanism. - At some later time, when first HDR capture event 410 occurs, the most recently obtained EV− and EV+ images 415 may be evaluated (block 315). Evaluation in accordance with
block 315 may include determining how many blown-out pixels are in prior obtained EV− and/or EV+ images 415. In practice this information may be generated by camera firmware and attached to each image as it moves along the image processing pipeline (see discussion below with respect toFIG. 5 ). Once this is done, a determination may be made as to whether the current brackets are “good” (block 320). If evaluation of EV− and/or EV+ images 415 determines that one or both bracket settings should to be adjusted (the “YES” prong of block 320), such adjustment may be made (block 325). Evaluation and adjustment in accordance with 320 and 325 may be performed as described above with respect toblocks 130 and 135 inblocks FIG. 1 . With proper bracket settings, first EV−image 420,first EV0 image 425 andfirst EV+ image 430 may be captured (block 330). If, on the other hand, the bracket settings used to capture prior EV− and EV+ images 415 are determined to be “good” (the “NO” prong of block 320), EV−, EV0, and EV+ images 420-430 may be captured immediately (block 330). In one embodiment, following the capture of HDR image set 420-430 in accordance withblock 330, the bracket settings used to obtainsubsequent images 435 may be default bracket settings. In another embodiment, the bracket settings used to obtainsubsequent images 435 may be the “current” bracket settings. In this latter embodiment, ifblock 330 is arrived at via the “NO” prong ofblock 320, the bracket settings used to obtainsubsequent images 435 will be the same bracket settings as was used to obtain prior images 415. However, ifblock 330 was arrived at via the “YES” prong ofblock 320,subsequent images 435 may be obtained using the bracket settings set in accordance withblock 325. - Following capture of the first HDR image set (images 420-430), a test may be made to determine if the HDR mode should be terminated (block 335). In one embodiment, HDR mode may be terminated by an explicit act of a user (e.g., the user actively turns HDR mode off). In another embodiment, HDR mode may be terminated automatically in accordance with a specified event (e.g., the passage of time). If it is determined that the HDR mode should be terminated (the “YES” prong of 335), it is so terminated (block 340). If it is determined that the HDR mode should not be terminated (the “NO” prong of 335),
iAEB operation 300 continues atblock 310. At some time after the capture of HDR images 420-430, second HDRimage capture event 440 may occur. Again, the most recent EV− and EV+ images 445 may be evaluated (block 315) for purposes of determining what bracket setting should be used to capture second HDR image set 450 (blocks 320-330). Following capture of HDR image set 450,iAEB operation 300 continues atblock 335 as described above. - Referring to
FIG. 5 , illustrativeimage capture subsystem 500—such as may be found in a mobile device having an embedded camera—includessensor 505 andimage processing pipeline 510, itself composed of early and late 515 and 520. In practice,stage processing units image subsystem 500 may be composed of hardware (e.g., special and/or general purpose processing units), software (e.g., firmware), or a combination hardware and software.Raw sensor data 525 may be initially operated upon by earlystage processing unit 515. In one embodiment,unit 515 may provide dead pixel correction (and other sensor and/or lens accommodation operations), and white balancing functions. As shown, once early-stage processing has completed,image capture subsystem 500 may provide clippedpixel information 530; typically in the form of a histogram. This information may be used to determine the number of pixels in EV− and/or EV+ that are blown-out (see discussion above). Further processing byunit 520 may provide image sharpening, gamma correction, and noise reduction. Output fromimage subsystem 500 may include multiple images 535 (e.g., EV−, EV0, and EV+ images). Once obtained in accordance with this disclosure, image set 535 may be fused to generate a single HDR output image or saved to memory/storage for later fusion or other processing. It will be recognized that clipped pixel information may also be obtained fromimage processing pipeline 510 output (e.g., from images 535). - Due to the additional treatment provided by late-
stage processing unit 520, the threshold used to determine whether the brackets used to capture prior EV− and EV+ images (e.g., 215, 225, 240 and 250 inFIGS. 2 or 415 and 445 inFIG. 4 ) are good or bad in accordance with blocks 130-135 (seeFIG. 1 ) or 315-320 (seeFIG. 3 ) may be different. In one embodiment, ifsensor 505 provides 8 Mpix images the blown-out pixel threshold using clippedpixel information 530 may be 15,000 pixels while the blown-out pixelthreshold using pipeline 510's output (e.g., from images 535) may be 10,000 pixels. In another embodiment, the thresholds may be a function (e.g., a percentage) of the number of pixels provided bysensor 505. In yet another embodiment, the threshold applied to an EV− image may be different from that applied to an EV+ image. - Referring to
FIG. 6 , a simplified functional block diagram of illustrativeelectronic device 600 is shown according to one embodiment.Electronic device 600 may includeprocessor 605,display 610,user interface 615,graphics hardware 620, device sensors 625 (e.g., proximity sensor/ambient light sensor, accelerometer and/or gyroscope),microphone 630, audio codec(s) 635, speaker(s) 640,communications circuitry 645, digitalimage capture unit 650, video codec(s) 655,memory 660,storage 665, andcommunications bus 670.Electronic device 600 may be, for example, a personal digital assistant (PDA), personal music player, a mobile telephone, or a notebook, laptop or tablet computer system. -
Processor 605 may execute instructions necessary to carry out or control the operation of many functions performed by device 600 (e.g., such as the generation and/or processing of images in accordance withoperations 100 and 300).Processor 605 may, for instance,drive display 610 and receive user input fromuser interface 615.User interface 615 can take a variety of forms, such as a button, keypad, dial, a click wheel, keyboard, display screen and/or a touch screen.Processor 605 may be a system-on-chip such as those found in mobile devices and include a dedicated graphics processing unit (GPU).Processor 605 may be based on reduced instruction-set computer (RISC) or complex instruction-set computer (CISC) architectures or any other suitable architecture and may include one or more processing cores.Graphics hardware 620 may be special purpose computational hardware for processing graphics and/or assistingprocessor 605 process graphics information. In one embodiment,graphics hardware 620 may include a programmable graphics processing unit (GPU). - Sensor and
camera circuitry 650 may capture still and video images that may be processed to generate images in accordance with this disclosure and may, for example, incorporateimage capture subsystem 500. Output fromcamera circuitry 650 may be processed, at least in part, by video codec(s) 655 and/orprocessor 605 and/orgraphics hardware 620, and/or a dedicated image processing unit incorporated withincircuitry 650. Images so captured may be stored inmemory 660 and/orstorage 665.Memory 660 may include one or more different types of media used byprocessor 605,graphics hardware 620, andimage capture circuitry 650 to perform device functions. For example,memory 660 may include memory cache, read-only memory (ROM), and/or random access memory (RAM).Storage 665 may store media (e.g., audio, image and video files), computer program instructions or software, preference information, device profile information, and any other suitable data.Storage 665 may include one more non-transitory storage mediums including, for example, magnetic disks (fixed, floppy, and removable) and tape, optical media such as CD-ROMs and digital video disks (DVDs), and semiconductor memory devices such as Electrically Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM).Memory 660 andstorage 665 may be used to retain computer program instructions or code organized into one or more modules and written in any desired computer programming language. When executed by, for example,processor 605 such computer program code may implement one or more of the methods described herein. - It is to be understood that the above description is intended to be illustrative, and not restrictive. For example, the capture of EV− and EV+ images are not to be limited to one each. In some embodiments, 2 or more EV− images (using different exposure values) and/or two or more EV+ images (using different exposure values) may be used. In addition, many different image capture orders may be used. For example, the EV0 image may be captured first followed by the EV− image followed by the EV+ image. The material has been presented to enable any person skilled in the art to make and use the invention as claimed and is provided in the context of particular embodiments, variations of which will be readily apparent to those skilled in the art (e.g., some of the disclosed embodiments may be used in combination with each other). The scope of the invention therefore should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.”
Claims (30)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US13/490,099 US9137456B2 (en) | 2012-06-06 | 2012-06-06 | Intelligent auto-exposure bracketing |
| US14/841,055 US9407831B2 (en) | 2012-06-06 | 2015-08-31 | Intelligent auto-exposure bracketing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US13/490,099 US9137456B2 (en) | 2012-06-06 | 2012-06-06 | Intelligent auto-exposure bracketing |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US14/841,055 Continuation US9407831B2 (en) | 2012-06-06 | 2015-08-31 | Intelligent auto-exposure bracketing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20130329092A1 true US20130329092A1 (en) | 2013-12-12 |
| US9137456B2 US9137456B2 (en) | 2015-09-15 |
Family
ID=49715018
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US13/490,099 Expired - Fee Related US9137456B2 (en) | 2012-06-06 | 2012-06-06 | Intelligent auto-exposure bracketing |
| US14/841,055 Expired - Fee Related US9407831B2 (en) | 2012-06-06 | 2015-08-31 | Intelligent auto-exposure bracketing |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US14/841,055 Expired - Fee Related US9407831B2 (en) | 2012-06-06 | 2015-08-31 | Intelligent auto-exposure bracketing |
Country Status (1)
| Country | Link |
|---|---|
| US (2) | US9137456B2 (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140168474A1 (en) * | 2012-12-13 | 2014-06-19 | Google Inc. | Determining an Image Capture Payload Burst Structure |
| US20140267883A1 (en) * | 2013-03-14 | 2014-09-18 | Konica Minolta Laboratory U.S.A., Inc. | Method of selecting a subset from an image set for generating high dynamic range image |
| US8964060B2 (en) | 2012-12-13 | 2015-02-24 | Google Inc. | Determining an image capture payload burst structure based on a metering image capture sweep |
| US9066017B2 (en) | 2013-03-25 | 2015-06-23 | Google Inc. | Viewfinder display based on metering images |
| US9077913B2 (en) | 2013-05-24 | 2015-07-07 | Google Inc. | Simulating high dynamic range imaging with virtual long-exposure images |
| US9117134B1 (en) | 2013-03-19 | 2015-08-25 | Google Inc. | Image merging with blending |
| US9131201B1 (en) | 2013-05-24 | 2015-09-08 | Google Inc. | Color correcting virtual long exposures with true long exposures |
| US9172888B2 (en) | 2012-12-18 | 2015-10-27 | Google Inc. | Determining exposure times using split paxels |
| US9247152B2 (en) | 2012-12-20 | 2016-01-26 | Google Inc. | Determining image alignment failure |
| US9390487B2 (en) | 2014-10-20 | 2016-07-12 | Microsoft Technology Licensing, Llc | Scene exposure auto-compensation for differential image comparisons |
| JP2017046151A (en) * | 2015-08-26 | 2017-03-02 | カシオ計算機株式会社 | Imaging apparatus, imaging setting method and program |
| US9615012B2 (en) | 2013-09-30 | 2017-04-04 | Google Inc. | Using a second camera to adjust settings of first camera |
| US9686537B2 (en) | 2013-02-05 | 2017-06-20 | Google Inc. | Noise models for image processing |
| US20180084181A1 (en) * | 2016-09-22 | 2018-03-22 | Apple Inc. | Adaptive bracketing techniques |
| US20180260941A1 (en) * | 2017-03-07 | 2018-09-13 | Adobe Systems Incorporated | Preserving color in image brightness adjustment for exposure fusion |
| WO2019125074A1 (en) * | 2017-12-21 | 2019-06-27 | Samsung Electronics Co., Ltd. | Method of generating composite image using plurality of images with different exposure values and electronic device supporting the same |
| US10542204B2 (en) * | 2015-08-05 | 2020-01-21 | Microsoft Technology Licensing, Llc | Methods and apparatuses for capturing multiple digital image frames |
| CN115037884A (en) * | 2018-06-01 | 2022-09-09 | 苹果公司 | Unified Bracketing Method for Imaging |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9137456B2 (en) * | 2012-06-06 | 2015-09-15 | Apple Inc. | Intelligent auto-exposure bracketing |
| CN108111741A (en) * | 2016-11-24 | 2018-06-01 | 努比亚技术有限公司 | A kind of method and terminal for realizing image taking |
| US10498971B2 (en) | 2017-11-03 | 2019-12-03 | Qualcomm Incorporated | Systems and methods for high-dynamic range imaging |
| CN110139066B (en) * | 2019-03-24 | 2021-02-02 | 初速度(苏州)科技有限公司 | Sensor data transmission system, method and device |
| CN110012227B (en) | 2019-04-09 | 2021-06-29 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic device |
| US11363213B1 (en) * | 2021-02-26 | 2022-06-14 | Qualcomm Incorporated | Minimizing ghosting in high dynamic range image processing |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7239757B2 (en) * | 2003-04-29 | 2007-07-03 | Microsoft Corporation | System and process for generating high dynamic range video |
| US20080187235A1 (en) * | 2006-10-19 | 2008-08-07 | Sony Corporation | Image processing apparatus, imaging apparatus, imaging processing method, and computer program |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9137456B2 (en) * | 2012-06-06 | 2015-09-15 | Apple Inc. | Intelligent auto-exposure bracketing |
-
2012
- 2012-06-06 US US13/490,099 patent/US9137456B2/en not_active Expired - Fee Related
-
2015
- 2015-08-31 US US14/841,055 patent/US9407831B2/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7239757B2 (en) * | 2003-04-29 | 2007-07-03 | Microsoft Corporation | System and process for generating high dynamic range video |
| US20080187235A1 (en) * | 2006-10-19 | 2008-08-07 | Sony Corporation | Image processing apparatus, imaging apparatus, imaging processing method, and computer program |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9087391B2 (en) * | 2012-12-13 | 2015-07-21 | Google Inc. | Determining an image capture payload burst structure |
| US9118841B2 (en) | 2012-12-13 | 2015-08-25 | Google Inc. | Determining an image capture payload burst structure based on a metering image capture sweep |
| US8964060B2 (en) | 2012-12-13 | 2015-02-24 | Google Inc. | Determining an image capture payload burst structure based on a metering image capture sweep |
| US20140168474A1 (en) * | 2012-12-13 | 2014-06-19 | Google Inc. | Determining an Image Capture Payload Burst Structure |
| US9172888B2 (en) | 2012-12-18 | 2015-10-27 | Google Inc. | Determining exposure times using split paxels |
| US9247152B2 (en) | 2012-12-20 | 2016-01-26 | Google Inc. | Determining image alignment failure |
| US9749551B2 (en) | 2013-02-05 | 2017-08-29 | Google Inc. | Noise models for image processing |
| US9686537B2 (en) | 2013-02-05 | 2017-06-20 | Google Inc. | Noise models for image processing |
| US20140267883A1 (en) * | 2013-03-14 | 2014-09-18 | Konica Minolta Laboratory U.S.A., Inc. | Method of selecting a subset from an image set for generating high dynamic range image |
| US8902328B2 (en) * | 2013-03-14 | 2014-12-02 | Konica Minolta Laboratory U.S.A., Inc. | Method of selecting a subset from an image set for generating high dynamic range image |
| US9117134B1 (en) | 2013-03-19 | 2015-08-25 | Google Inc. | Image merging with blending |
| US9066017B2 (en) | 2013-03-25 | 2015-06-23 | Google Inc. | Viewfinder display based on metering images |
| US9077913B2 (en) | 2013-05-24 | 2015-07-07 | Google Inc. | Simulating high dynamic range imaging with virtual long-exposure images |
| US9131201B1 (en) | 2013-05-24 | 2015-09-08 | Google Inc. | Color correcting virtual long exposures with true long exposures |
| US9615012B2 (en) | 2013-09-30 | 2017-04-04 | Google Inc. | Using a second camera to adjust settings of first camera |
| US9390487B2 (en) | 2014-10-20 | 2016-07-12 | Microsoft Technology Licensing, Llc | Scene exposure auto-compensation for differential image comparisons |
| US10542204B2 (en) * | 2015-08-05 | 2020-01-21 | Microsoft Technology Licensing, Llc | Methods and apparatuses for capturing multiple digital image frames |
| JP2017046151A (en) * | 2015-08-26 | 2017-03-02 | カシオ計算機株式会社 | Imaging apparatus, imaging setting method and program |
| US20180084181A1 (en) * | 2016-09-22 | 2018-03-22 | Apple Inc. | Adaptive bracketing techniques |
| US9955085B2 (en) * | 2016-09-22 | 2018-04-24 | Apple Inc. | Adaptive bracketing techniques |
| US10706512B2 (en) * | 2017-03-07 | 2020-07-07 | Adobe Inc. | Preserving color in image brightness adjustment for exposure fusion |
| US20180260941A1 (en) * | 2017-03-07 | 2018-09-13 | Adobe Systems Incorporated | Preserving color in image brightness adjustment for exposure fusion |
| US20190199905A1 (en) * | 2017-12-21 | 2019-06-27 | Samsung Electronics Co., Ltd. | Method of generating composite image using plurality of images with different exposure values and electronic device supporting the same |
| KR20190075292A (en) * | 2017-12-21 | 2019-07-01 | 삼성전자주식회사 | Method of generating composite image using a plurality of images with different exposure values and electronic device supporting the same |
| WO2019125074A1 (en) * | 2017-12-21 | 2019-06-27 | Samsung Electronics Co., Ltd. | Method of generating composite image using plurality of images with different exposure values and electronic device supporting the same |
| US10904452B2 (en) | 2017-12-21 | 2021-01-26 | Samsung Electronics Co., Ltd. | Method of generating composite image using plurality of images with different exposure values and electronic device supporting the same |
| KR102412591B1 (en) * | 2017-12-21 | 2022-06-24 | 삼성전자주식회사 | Method of generating composite image using a plurality of images with different exposure values and electronic device supporting the same |
| CN115037884A (en) * | 2018-06-01 | 2022-09-09 | 苹果公司 | Unified Bracketing Method for Imaging |
Also Published As
| Publication number | Publication date |
|---|---|
| US9137456B2 (en) | 2015-09-15 |
| US20150373248A1 (en) | 2015-12-24 |
| US9407831B2 (en) | 2016-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9407831B2 (en) | Intelligent auto-exposure bracketing | |
| US11562470B2 (en) | Unified bracketing approach for imaging | |
| CN109040609B (en) | Exposure control method, exposure control device, electronic equipment and computer-readable storage medium | |
| US8339506B2 (en) | Image capture parameter adjustment using face brightness information | |
| US9077913B2 (en) | Simulating high dynamic range imaging with virtual long-exposure images | |
| US9131201B1 (en) | Color correcting virtual long exposures with true long exposures | |
| EP4033750B1 (en) | Method and device for processing image, and storage medium | |
| JP6272486B2 (en) | Image processing apparatus, imaging apparatus, image processing method, and program | |
| JP6720881B2 (en) | Image processing apparatus and image processing method | |
| WO2019183813A1 (en) | Image capture method and device | |
| US9380218B2 (en) | Highlight exposure metric and its applications | |
| TW201233159A (en) | High dynamic range transition | |
| JP6460721B2 (en) | Image processing apparatus, image processing method, and program | |
| CN108337447A (en) | High dynamic range image exposure compensation value acquisition method, device, equipment and medium | |
| JP2016516378A (en) | Generation of high dynamic range images without ghosting | |
| JP2012199675A (en) | Image processing apparatus, image processing method, and program | |
| US10769416B2 (en) | Image processing method, electronic device and storage medium | |
| CN116055855B (en) | Image processing method and related device | |
| EP3179716B1 (en) | Image processing method, computer storage medium, device, and terminal | |
| JP5370555B2 (en) | Imaging apparatus, imaging method, and program | |
| US8675964B2 (en) | Image fusion using intensity mapping functions | |
| CN105657236A (en) | Image processing method and device | |
| JP5355124B2 (en) | Imaging apparatus and scene discrimination method thereof | |
| JP5966899B2 (en) | Imaging apparatus, shooting mode determination method, and shooting mode determination program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment |
Owner name: APPLE INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WONG, EARL Q.;REEL/FRAME:028330/0231 Effective date: 20120604 |
|
| FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
| STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
| CC | Certificate of correction | ||
| MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
| FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
| LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
| STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
| FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20230915 |