[go: up one dir, main page]

GB2531299A - Device for illuminating a scene and control method thereof - Google Patents

Device for illuminating a scene and control method thereof Download PDF

Info

Publication number
GB2531299A
GB2531299A GB1418279.4A GB201418279A GB2531299A GB 2531299 A GB2531299 A GB 2531299A GB 201418279 A GB201418279 A GB 201418279A GB 2531299 A GB2531299 A GB 2531299A
Authority
GB
United Kingdom
Prior art keywords
scene
illumination
light
illumination source
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB1418279.4A
Other versions
GB2531299B (en
GB201418279D0 (en
Inventor
Carl Christian Gressum
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Priority to GB1418279.4A priority Critical patent/GB2531299B/en
Publication of GB201418279D0 publication Critical patent/GB201418279D0/en
Priority to KR1020150094930A priority patent/KR20160044405A/en
Priority to US14/870,969 priority patent/US10154202B2/en
Priority to EP15189532.3A priority patent/EP3010223B1/en
Publication of GB2531299A publication Critical patent/GB2531299A/en
Application granted granted Critical
Publication of GB2531299B publication Critical patent/GB2531299B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00Special procedures for taking photographs; Apparatus therefor
    • G03B15/02Illuminating scene
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00Special procedures for taking photographs; Apparatus therefor
    • G03B15/02Illuminating scene
    • G03B15/03Combinations of cameras with lighting apparatus; Flash units
    • G03B15/05Combinations of cameras with electronic flash apparatus; Electronic flash units
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B2215/00Special procedures for taking photographs; Apparatus therefor
    • G03B2215/05Combinations of cameras with electronic flash units
    • G03B2215/0589Diffusors, filters or refraction means
    • G03B2215/0592Diffusors, filters or refraction means installed in front of light emitter

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Studio Devices (AREA)

Abstract

A device for illuminating a scene includes an adaptive optical element 110 which is controllable to change an illumination pattern provided by an illumination source 106. The adaptive optical element can be controlled to provide a suitable illumination pattern for the scene to ensure a more uniform exposure across the scene. The optical element may be a lens, a mirror, a micromirror array or a microlens array. The device may be part of a smartphone.

Description

Device for illuminating a scene and control method thereof
Technical Field
The present invention relates to a device for illuminating a scene and control method thereof. More paiticulafly, the present invention relates to a device comprising an adaptive optical element which can be controfled to change an illumination pattern provided by an illumination source, such as a camera flash.
Background of the Invention
io When capturing an image in a low-light setting, an illumination source such as a flash can be used to temporarily increase the amount of light in the scene being captured.
The illumination source is typically not capable of illuminating the scene correctly, since illumination sources found in many portable devices are not designed to reproduce sufficient light to illuminate a wh&e scene. Consequently with a i conventional camera and flash system a foreground subject can be overexposed, whilst the background section is typically underexposed due to the limited dynamic range of the image sensor and the use of flash that is not powerful enough to correctly illuminate the whole scene. This is particulariy problematic with integrated mobile phone cameras, as design constraints necessarily impose size restrictions on lenses and image sensors that can be used, limiting the amount of light that can reach the image sensor.
As a result, the image maybe overly dark, elements in the foreground can be overexposed, and the image may appear noisy. In some cases the image quafity can be improved during post-processing, however, this requires additional processing time and resources, and can be inconvenient for a user.
The invention is made in this context.
Summary of the Invention
According to the present invention, there is provided a device for illuminating a scene comprising: an illumination source arranged to illuminate the scene; an adaptive optical element controllable to change how the scene is illuminated by the illumination source; and a controller arranged to determine an illumination pattern for the scene, and to control the adaptive optical element to provide the illumination pattern when the scene is illuminated by the illumination source.
The controller can be further arranged to obtain pre-capture data relating to the scene to be captured, and to determine the illumination pattern based on the pre-capture data.
The pre-capture data can include data r&ating to light levels within the scene, and the controller can be arranged to set a higher illumination levál for a darker area within the scene and set a lower illumination evel for a lighter area within the scene when determining the illumination pattern.
jo The controller can be arranged to use the sensor to obtain the data relating to light levels from an image sensor.
The pre-capture data can include co'our information relating to a plurality of elements within the scene, and when determining the illumination pattern the controller can be arranged to determine an amount of light from the illumination source that will be reflected by each one of the plurality of elements, and to set a higher illumination level for an element that will reflect less light from the illumination source and a lower illumination eve1 for an element that will reflect more light from the illumination source.
The pre-capture data can include image data captured from the scene using an image sensor, and the device can further comprise: a foreground separation module arranged to analyse the image data to identify a foreground element in the scene, wherein the controller is arranged to set different illumination evels for the foreground element and for the background when determining the illumination pattern.
The controller can be further arranged to control an exposure time and/or an exposure intensity of the illumination source.
The controller can be further arranged to obtain information about the dynamic range of an image sensor to be used to capture an image of the scene, and to determine the illumination pattern in accordance with the dynamic range of the image sensor.
The adaptive optical element can comprise an adaptive microlens array and/or a shape controlled lens.
The adaptive optical element can comprise one or more lenses configured to be moveable into and out of an optical path between the illumination source and the scene.
The device can be included in a mobile communication device.
The device can further comprise an image sensor arranged to capture an image of the scene.
jo The illumination source may comprise a plurality of fight emitting elements.
According to the present invention, there is also provided a control method of a device for illuminating a scene, the device comprising an illumination source arranged to illuminate the scene and an adaptive optical element controllable to change how the scene is illuminated by the illumination source, the control method comprising: determining an illumination pattern for the scene; and controlling the adaptive optical dement to provide the iflumination pattern when the scene is illuminated by the illumination source.
The control method can further comprise: obtaining pre-capture data relating to the scene to be captured, and the illumination pattern can be determined based on the pre-capture data.
The pre-capture data can include data relating to light levels within the scene, and determining the illumination pattern can comprise setting a higher illumination level for a darker area within the scene and setting a lower illumination level for a lighter area within the scene.
Thc data rclating to light lcvcls can bc obtaincd from an imagc scnsor.
The pre-capture data can include colour information relating to a plurality of elements within the scene, and determining the illumination pattern can comprise: determining an amount of light from the illumination source that will be reflected by each one of the plurality of elements; and setting a higher illumination level for an element that will reflect less light from the illumination source and a lower illumination level for an element that will reflect more light from the illumination source.
The pre-capture data can include image data captured from the scene using an image sensor, and the method can further comprise: analysing the image data to identify a foreground element in the scene, wherein the illumination pattern can be determined by setting different illumination lev&s for the foreground &ement and for the background when determining the illumination pattern.
The control method can further comprise controlling an exposure time and/or an exposure intensity of the illumination source.
The control method can further comprise obtaining information about the dynamic range of an image sensor to be used to capture an image of the scene, and the illumination pattern can be determined in accordance with the dynamic range of the image sensor.
According to the present invention, there is also provided a computer-readable storage medium storing computer program instructions which, when executed, perform any of the control methods described herein.
Brief Description of the Drawings
Embodiments of the present invention wifl now be described, byway of example on'y, with reference to the accompanying drawings, in which: Figure 1 schematically illustrates an image capture device, according to an embodiment of the present invention; Figures 2A to 2C schematically illustrate different illumination patterns obtained using a shape-controlled lens as an adaptive optical element, according to an embodiment of the present invention; Figure 3 illustrates an adaptive optical element comprising a micromirror array, according to an cmbodimcnt of thc prcscnt invcntion; Figure 4 illustrates an adaptive optical element comprising a microlens array, according to an embodiment of the present invention; Figures illustrates an adaptive optical element comprising a moveable lens, according to an embodiment of the present invention; Figure 6 is a flowchart showing a control method of an image capture device, according to an embodiment of the present invention; Figure 7 is a flowchart showing a method of determining an illumination pattern based on pre-capture image data, according to an embodiment of the present invention; Figure 8 is a flowchart showing a method of determining an illumination pattern based on pre-capture fight meter data, according to an embodiment of the present invention; Figure 9 is a flowchart showing a method of determining an illumination pattern based on colour information, according to an embodiment of the present invention; and Figure 10 schematically illustrates the controller of Fig. 1.
Detailed Description
jo Referring now to Fig. 1, an image capture device wiiI be described according to an embodiment of the present invention. Tn the present embodiment the image capture device is included in a mobile communication device, specifically, a smartphone.
However, aspects of the invention are not limited to use in mobile communication devices, and can be applied to image capture systems in any type of device, including but not limited to stand-alone digital cameras, camcorders, webcams, tablet computers, laptops and desktop computers, wearable cameras, surveillance cameras and speed cameras.
The image capture device 100 comprises a sensor 102 arranged to capture an image of a scene 120, such as a Charge Coupled Device (CCD) or Complementary Metal-Oxide-Semiconductor (CMOS) sensor, one or more lenses 104 arranged to focus light from the scene 120 onto the sensor 102, an illumination source io6 arranged to illuminate the scene 120, and a light meter 108 for obtaining information about light levels within the scene 120. In the present embodiment the illumination source 106 comprises a Light Emitting Diode (LED) based flash, which can be triggered immediately before capturing an image to provide additional illumination in the scene 120. However, in other embodiments a different type of illumination source may be used, such as a Xenon flash.
Also, the image capture device 100 further comprises an adaptive optical element 110 controllable to change how the scene 120 is illuminated by the illumination source 106, and a controller 112. The controller 112 is arranged to determine an illumination pattern for the scene 120, and to control the adaptive optical element 110 to provide the illumination pattern when an image is captured by the sensor 102.
In some embodiments the illumination source may include a plurality of discrete light emitting elements, for example a plurality of LEDs. When a plurality of discrete light emitting elements are used, each light emitting element may provided with a separate adaptive optical element. Alternatively, some or all of the light emitting elements may share the same adaptive optical element, such that a given adaptive optical element receives light from two or more of the light emitting elements.
Various types of adaptive optical elements can be used alone or in combination in embodiments of the present invention. The adaptive optical element may be o configured to have one or more variable optical properties, such as reflectivity or refractive index. Examp'es of adaptive optical elements with variable optical properties include, but are not limited to, shape-controlled lenses such as liquid lenses and sound-controlled lenses, adaptive microlens arrays, and micromirror arrays. In a liquid lens, the refractive index of the liquid can be electrically tuned using the Kerr effect, in order to control the focal length of the tens. Alternatively, the refractive index could be controlled by changing the dielectric permittivity or magnetic permeability of a material included in the adaptive optical element, for examp'e by applying an external electric
field to the adaptive optical element.
Instead of, or in addition to, an optical element with variable optical properties, in some embodiments the adaptive optical element may include one or more elements with fixed optical properties, such as a fixed-focus lens arranged to be mechanically moveable from one position to another.
In more detail, the controller 112 is arranged to control the adaptive optical element 110 so that the scene 120 will be correctly exposed when illuminated by the illumination source io6. The controller 112 can make use of various types of information to decide on a suitable illumination pattern for the scene 120. In the present embodiment, the controllcr 112 is arrangcd to obtain prc-capturc data from thc scnsor 102 and thc light meter 108. The pre-capture data provides information relating to the scene 120 to be captured, and can be used to determine the illumination pattern. Tn addition to, or instead of, obtaining pre-capture data, the controller 112 may also obtain information about a dynamic range of the sensor 102, and take the dynamic range into account when determining the illumination pattern.
In another embodiment of the invention, in addition to or instead of obtaining pre-capture data, the controller 112 may a'so obtain information about a dynamic range of the sensor 102 and/or the therma' dark current or the noise associated therewith, and determine the illumination pattern based on the obtained information.
The illumination pattern can be defined in various ways. tn some embodiments, the illumination pattern can simply be expressed in terms of control parameters which are used to control the adaptive optical element. In other embodiments, the illumination pattern may be stored as an array of values defining different illumination leveth at Jo different x, y coordinates within the scene. Depending on the type of adaptive optical dement used, suitable control parameters for the adaptive optical element can then be determined based on the illumination pattern.
The image data can be analysed using a foreground separation algorithm, to separate a foreground element 122 from the background 124. When a foreground element 122 is identified, the controller 112 can set different illumination levels for the foreground element 122 and for the background 124 when determining the illumination pattern. Tn many cases a higher illumination level will be required for the background 124, in order to achieve an even exposure across the scene 120. However, in some cases a higher illumination level may be set for the foreground element 122, for example when the foreground element 122 is darker in c&our than the background 124, and/or in shadow
in comparison to the background 124.
Also, as described above, in the present embodiment the controller 112 makes use of light meter data received from the light meter io8. The light meter data relates to light levels within the scene 120. Depending on the type of light meter used, the light meter data may be expressed as a single average value relating to the ambient light levels across the scene 120 as a whole, or may include a plurality of values each associated with a diffcrcnt arca within thc sccnc 120. Whcn granular information about light evels within the scene 120 is available, the controller 112 can set a higher illumination level for a darker area within the scene 120 and set a lower iflumination level for a lighter area within the scene 120 when determining the illumination pattern. Here, the terms darker' and lighter' areas are used to refer to areas with lower and higher Lux values, respectively.
In addition to controlling the adaptive optical e'ement 110, in some embodiments the controller 112 can a'so contr& the exposure time and/or the exposure intensity of the illumination source 106 in order to achieve the desired illumination pattern.
Although in the present embodiment light meter data relating to light levels within the scene is obtained from a separate light meter 108, in other embodiments the light meter data may not be used, or may be captured using the camera sensor 102. For example, the camera sensor can be used to pre-capture an image before capturing a final image with the scene illuminated by the flash. The pre-capture image can be jo analysed to extract information about light levels within the scene, which may be expressed as lux values for individual pixels or groups of pixels. By using the camera sensor 102 to obtain light meter data, the separate light meter can be omitted, and higher-resolution information about light levels within the scene can be obtained.
Tn the present embodiment, the iflumination pattern is determined based on pre-capture data. Tn some embodiments, the pre-capture data can inchide colour information relating to a plurality of elements within the scene, in addition to or instead of the other types of pre-capture data described above. When colour information is available, the controller can estimate the amount of light from the illumination source that will be reflected by each element in the scene, and set the illumination pattern according'y. The controller can set a higher illumination levál can be set for an element that will reflect less light from the illumination source, and set a lower illumination evel for an element that will reflect more light. For example, a predominately black object will reflect less light than a predominately white object. To avoid a pale object appearing over-exposed in the final image, a lower illumination level can be used. Similarly, a higher illumination leve' can be used for a darker-coloured object, to avoid the object appearing under-exposed. This approach can ensure that the levol of reflected light reaching the image sensor when capturing the image remains substantially uniform across thc sccnc.
Tn the embodiment shown in Fig. 1, the adaptive optical element 110 and controller 112 are included in the same physical device as the other elements of the image capture device 100, including the image sensor 102 and illumination source 106. However, in other embodiments the adaptive optical olement 110, controller 112 and/or illumination source 108 may be embodied in a separate device to the image sensor 102. For example, the adaptive optical olement 110 and controller 112 may be provided in a separate device configured to attach mechanically to conventional smartphone device, and to communicate wirelessly with a camera controller in the smartphone. The illumination source could be included in either the smartphone or in the separate device.
Figures 2A to 2C schematically illustrate different illumination patterns obtained using a shape-controlled lens as an adaptive optical element, according to an embodiment of the present invention. Various types of shape-controlled lens are known in the art, such as liquid lenses and sound-controlled lenses, and a detailed description will not be jo provided here so as to avoid obscuring the present inventive concept. In Fig. 2A, the shape-controfled lens 210 is controlled to adopt a piano-convex tens shape, in order to focus light emitted from the illumination source 206 onto the scene 220. Tn Fig. 2B, the shape-controlled lens 210 is controlled to adopt a neutral lens shape, so that the path of light from the illumination source 206 is unaffected by the shape-controlled lens 210. In Fig. 2C, the shape-controfled lens 210 is controfled to adopt a pIano-convex lens shape, in order to disperse light emitted from the iflumination source 206 widely over the scene 220.
As described above, embodiments of the invention are not limited to shape-controlled lenses as the adaptive optical element. Other types of adaptive optical elements are schematically illustrated in Figs. 3 to 5. In Fig. 3, an embodiment is illustrated in which the adaptive optical element 310 comprises a micromirror array, similar to a Digital Micromirror Device (DMD) used in a Digital Light Processing (DLP) projector. The micromirror array includes a plurality of independently controllable micromirrors which can be individually switched to reflect light from the illumination source 306 towards or away from the scene 320. In the example shown in Fig. 3 the micromirror array 310 is set to direct light towards the periphery of the scene 320, but in general the micromirror array 310 maybe controlled to provide any desired illumination pattern.
Tn Fig. 4, an embodiment is illustrated in which the adaptive optica' element 410 comprises an adaptive microlens array disposed between the illumination source 406 and the scene 420. The microens array 410 includes an array of adaptive microlenses, which may for example be miniature liquid tenses. The shape of each lens can be independently controlled so as to achieve a desired illumination pattern for the scene 420. -10-
In Fig. 5, an embodiment is illustrated in which the adaptive optical element 510 comprises a lens configured to be moveable into and out of an optical path between the illumination source o6 and the scene 520. Alihough in the present embodiment a single lens io is illustrated, in other embodiments a plurality oflens may be provided.
When a p'urality of moveable lenses are used, different combinations of lenses could be moved into the optical path in order to achieve different illumination patterns.
Various different types of adaptive optical element have been described above with reference to Figs. 2Ato 5. In embodiments of the present invention, any of these types o of adaptive optica' element maybe used alone or in combination with any of the other types, in order to achieve the desired illumination pattern.
Referring now to Fig. 6, a flowchart showing a control method of an image capture device is illustrated, according to an embodiment of the present invention. The method can be performed by the controller 112 of Fig. 1 in order to achieve an even exposure across a scene being imaged. Depending on the embodiment, some or all of the steps in Fig. 6 maybe performed by dedicated hardware such as an Application-Specific Integrated Circuit (ASIC) or Fie'd Programmable Gate Array (FPGA), or may be performed in software by computer program instructions stored in memory and executed on one or more processors.
First, in step S6o1 pre-capture data relating to the scene is obtained. The pre-capture data may include various types of information, for example light meter data, image data and/or colour data. In the present embodiment information about the dynamic range of the image sensor is also obtained in step S6oi.
Next, in step S6o2 an illumination pattern is determined for the scene. In the present embodiment the illumination pattern is determined based on the pre-capture data and dynamic range information obtaincd in stcp S6oi. Howcvcr, as cxplaincd abovc, in some embodiments the pre-capture data and/or dynamic range information maybe omitted, for examp'e when pre-programmed illumination patterns are stored for different camera modes, in which case step S6o1 may be modified as appropriate or omitted entirdy.
Then, in step S6o3 the adaptive optica' álement is controlled to provide the illumination pattern when the image is captured by the sensor. In some embodiments, -11 -the adaptive optical element controller may operate as a slave to a separate master controller which controls the operation of the image sensor. For example, the image sensor controller may signal to the adaptive optical element controller when an image is about to be captured, at which point the adaptive optical element maybe switched to provide the desired illumination pattern. The image sensor controller could then separately trigger the illumination source and read image data from the camera sensor as normal. However, in the present embodiment the controller also controls the operation of the illumination source in step 5604 and the camera sensor in step S6o.
Specifically, in step S6o4 the illumination source is controlled to illuminate the scene, o based on a suitable exposure time and/or exposure intensity necessary to provide the desired illumination pattern. Tn step S6o5, image data is read from the camera sensor whilst the scene is being illuminated.
By adapting the illumination pattern according to the content of a scene, using a method such as the one shown in Fig. 1, embodiments of the present invention can achieve a more uniform exposure across the scene without the need for post-processing.
As explained above, various different types of pre-capture data can be used when determining the illumination pattern. A method of determining an illumination pattern based on pre-capture image data will now be described with reference to Fig. 7.
First, in step 5701 pre-capture data including image data is obtained. The image data may be obtained using the camera sensor, or may be obtained using a separate dedicated sensor. For example, the pre-capture image data may be obtained using a separate low-resolution sensor.
Then, in step 5702 the image data is analysed to identify a foreground element in the scene, and in step 3703 the illumination pattern is determined by setting different illumination lcvcls for thc forcground clcmcnt and for thc background, as dcscribcd above.
By using image data to separate out foreground and background elements in the scene, the adaptive optica' element can be controlled to avoid over-exposing the foreground, which is a common problem when capturing images in low-light conditions using conventional camera and flash systems. Also, by enabling the scene to be correctly exposed by the illumination source, embodiments of the invention can avoid the need -12 -to switch to a higher ISO setting in low-light conditions. Since a higher 150 setting would introduce more noise into the captured image, embodiments of the invention can therefore provide higher-quality images under low-light conditions.
A method of determining an illumination pattern based on pre-capture light meter data will now be described with reference to Fig. 8. First, in step S8oi pre-capture data including light meter data is obtained. As described above, the light meter data may be obtained from the image sensor, or from a separate light meter sensor. In the present embodiment, the light meter data includes a plurality of values relating to ambient light jo levels at different locations within the scene.
Then, in step 8802 the illumination pattern is determined based on the light meter data, by setting a higher illumination level for a darker area within the scene and setting a lower illumination level for a lighter area within the scene. This ensures a more uniform exposure across the scene when the final image is captured.
A method of determining an illumination pattern based on colour information will now be described with reference to Fig. 9. First, in step S9o1 pre-capture data including colour information is obtained. The colour information may be obtained using the same image sensor that will be used to capture the final image of the scene when illuminated by the illumination source, or may be obtained using a separate dedicated sensor. For example, the colour information may be obtained using a separate low-resolution sensor.
Then, in step 8902 the image data is analysed to determine the amount of light from the illumination source that will be reflected by each one of a plurality of elements in the scene. As explained above, objects that are paler in colour will reflect more light from the illumination source, and objects that are darker in colour will reflect less light.
Tn stcp 8903 thc illumination pattcrn is dctcrmincd by sctting diffcrcnt illumination levels for the elements according to the colour information for each element, as described above.
By using colour information to determine how much light from the illuminate source will be reflected by each element in the scene, the adaptive optical element can be controlled to avoid over-exposing lighter coloured objects and under-exposing darker coloured objects.
-13 -Referring now to Fig. 10, the controller of Fig. 1 is schematically illustrated. As shown in Fig. 10, the controller 112 includes a foreground separator n2a, an image processor 112b, and an ilhimination pattern generator nc. Depending on the embodiment, the foreground separator maybe implemented using a software algorithm, dedicated hardware, or a combination of hardware and software. Various methods for separating one or more foreground objects in an image from the background are known in the art, and a detailed description will not be provided here.
jo The foreground separator na and image processor 112b receive the pre-capture data, including image data and light meter data, from the image sensor 102 and light meter io8. The foreground separator na performs foreground separation based on the pre-capture image data, and determines a relative size and position of one or more foreground elements in the scene. The image processor 112b uses the image data and light meter data to obtain information about light levels in the x, y plane, foreground colour information in the x, y plane, and light levels in the background. This information is passed to the illumination pattern generator n2c, which a'so retrieves dynamic range information stored in a memory. The illumination pattern generator 112c determines an illumination pattern by setting illumination levels within the x, y plane based on the information received from the foreground separator iia and image processor 112b, taking into account the dynamic range of the image sensor 102. The illumination pattern generator 112c then outputs control signals to the adaptive optical element 110 and illumination source io6. Depending on the embodiment, the control signals sent to the adaptive optical element may include a command to change the shape of one or more shape-controlled lenses, and/or a command to switch an array or microlenses and/or micromirrors, and/or a command to move one or more moveable lenses into or out of the optical path. Also, depending on the embodiment, the control signal signals sent to the illumination source 106 may include a command to set the cxposurc timc and/or a command to sct thc cxposurc intcnsity.
A system such as the one shown in Fig. 10 can be used to perform any of the above-described methods, in order to provide more uniform iflumination levels when a scene is illuminated by the illumination source.
Embodiments of the invention have been described in which an illumination pattern is determined based on pre-capture data, for example, data received from a light meter sensor and/or a camera image sensor. However, in other embodiments pre-capture data may not be used, and another approach may be nsed to determine a suitable illuminate pattern. Tn some embodiments, the image capture device may store various predefined illumination patterns each associated with a different camera mode. For example, when a portrait' mode is selected, it maybe assumed that a foreground object 122 is ocated approximately centrally in the scene 120. Accordingly, a predefined illumination pattern can be used in which a lower illumination level is set for a central region, and a higher illumination level is set for a peripheral region.
o Whilst certain embodiments of the invention have been described herein with reference to the drawings, it WiB be understood that many variations and modifications will be possible without departing from the scope of the invention as defined in the accompanying claims.

Claims (14)

  1. -15 -Claims 1. A device for illuminating a scene, comprising: an illumination source arranged to illuminate the scene; an adaptive optical element controllable to change how the scene is illuminated by the illumination source; and a controller arranged to determine an illumination pattern for the scene, and to control the adaptive optical element to provide the illumination pattern when the scene is illuminated by the illumination source.
  2. 2. The device of claim 1, wherein the controller is further arranged to obtain pre-capture data relating to the scene to be captured, and to determine the illumination pattern based on the pre-capture data.
  3. 3. The device of claim 2, wherein the pre-capture data includes data relating to light levels within the scene, and the controller is arranged to set a higher illumination level for a darker area within the scene and set a lower illumination level for a Ughter area within the scene when determining the illumination pattern.
  4. 4. The device of claim 3, wherein the controller is arranged to obtain the data relating to light levels from an image sensor.
  5. 5. The device of claim 2, 3 or 4, wherein the pre-capture data includes colour information relating to a plurality of elements within the scene, and when determining the illumination pattern the controller is arranged to determine an amount of light from the illumination source that will be reflected by each one of the plurality of elements, and to set a higher illumination level for an element that will reflect less light from the illumination source and a lower illumination level for an element that will rcflcct morc light from thc illumination sourcc.
  6. 6. The device of any one of claims 2 to, wherein the pre-capture data includes image data captured from the scene using an image sensor, and the device further comprises: a foreground separation module arranged to analyse the image data to identify a foreground element in the scene, wherein the controller is arranged to set different illumination levels for the foreground element and for the background when determining the illumination pattern.
  7. 7. The device of any one of the preceding claims, wherein the controller is further arranged to control an exposure time and/or an exposure intensity of the illumination source.
  8. 8. The device of any one of the preceding claims, wherein the controller is further jo arranged to obtain information about the dynamic range of an image sensor to be used to capture an image of the scene, and to determine the illumination pattern in accordance with the dynamic range of the image sensor.
  9. 9. The device of any one of the preceding claims, wherein the adaptive optical dement comprises an adaptive microlens array and/or a shape controlled lens.
  10. 10. The device of any one of the preceding claims, wherein the adaptive optica' element comprises one or more lenses configured to be moveable into and out of an optical path between the illumination source and the scene.
  11. ii. The device of any one of the preceding claims, included in a mobile communication device.
  12. 12. The device of any one of the preceding claims, further comprising: an image sensor arranged to capture an image of the scene.
  13. 13. The device of any one of the preceding claims, wherein the illumination source comprises a plurality of light emitting elements.
  14. 14. A control method of a device for illuminating a scene, the device comprising an illumination source arranged to illuminate the scene and an adaptive optical element controllable to change how the scene is illuminated by the illumination source, the control method comprising: determining an illumination pattern for the scene; and controlling the adaptive optical element to provide the illumination pattern when the scene is illuminated by the illumination source. -17-15. The control method of claim 14, further comprising: obtaining pre-capture data relating to the scene to be captured, wherein the iflumination pattern is determined based on the pre-capture data.16. The control method of claim 15, wherein the pre-capture data includes data relating to light levels within the scene, and wherein determining the illumination pattern comprises setting a higher illumination level for a darker area within the scene and setting a lower illumination jo level for a lighter area within the scene.17. The control method of claim i6, wherein the data relating to light levels is obtained from an image sensor.18. The control method of claim 15,16 or 17, wherein the pre-capture data includes colour information relating to a plurality of elements within the scene, and determining the illumination pattern comprises: determining an amount of light from the illumination source that will be reflected by each one of the plurality of elements; and setting a higher illumination level for an element that will reflect less light from the illumination source and a lower illumination level for an element that will reflect more light from the illumination source.19. The control method of any one of claims 15 to 18, wherein the pre-capture data includes image data captured from the scene using an image sensor, and the method further comprises: analysing the image data to identify a foreground element in the scene, wherein determining the illumination pattern comprises setting different illumination lcvcls for thc forcground clcmcnt and for thc background whcn determining the illumination pattern.20. The control method of any one of claims 14 to 19, further comprising: controfling an exposure time and/or an exposure intensity of the iflumination source.21. The control method of any one of claims 14 to 20, further comprising: -18 -obtaining information about the dynamic range of an image sensor to be used to capture an image of the scene, wherein the iflumination pattern is determined in accordance with the dynamic range of the image sensor.22. A computer-readable storage medium storing computer program instructions which, when executed, perform the control method of any one of claims 14 to 21.
GB1418279.4A 2014-10-15 2014-10-15 Device for illuminating a scene and control method thereof Expired - Fee Related GB2531299B (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
GB1418279.4A GB2531299B (en) 2014-10-15 2014-10-15 Device for illuminating a scene and control method thereof
KR1020150094930A KR20160044405A (en) 2014-10-15 2015-07-02 Camera for illuminating a scene and control method thereof
US14/870,969 US10154202B2 (en) 2014-10-15 2015-09-30 Apparatus for illuminating a scene and control method thereof
EP15189532.3A EP3010223B1 (en) 2014-10-15 2015-10-13 Apparatus for illuminating a scene and control method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB1418279.4A GB2531299B (en) 2014-10-15 2014-10-15 Device for illuminating a scene and control method thereof

Publications (3)

Publication Number Publication Date
GB201418279D0 GB201418279D0 (en) 2014-11-26
GB2531299A true GB2531299A (en) 2016-04-20
GB2531299B GB2531299B (en) 2017-09-13

Family

ID=52001481

Family Applications (1)

Application Number Title Priority Date Filing Date
GB1418279.4A Expired - Fee Related GB2531299B (en) 2014-10-15 2014-10-15 Device for illuminating a scene and control method thereof

Country Status (2)

Country Link
KR (1) KR20160044405A (en)
GB (1) GB2531299B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12348875B2 (en) 2021-09-24 2025-07-01 Apple Inc. Adaptive-flash photography, videography, and/or flashlight using camera, scene, or user input parameters

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109414120B (en) * 2016-07-08 2021-06-22 皇家飞利浦有限公司 Apparatus for providing semantic information and method of operating the same
CN115866112A (en) * 2021-09-24 2023-03-28 苹果公司 Adaptive flash photography, video and/or flash using camera, scene or user input parameters

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201024906A (en) * 2008-12-26 2010-07-01 Taiwan Green Point Entpr Co Wireless communication device with adjustable light compensation

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8576297B2 (en) * 2008-07-11 2013-11-05 The Invention Science Fund I, Llc Spectral improvement of digital camera color images
US8761594B1 (en) * 2013-02-28 2014-06-24 Apple Inc. Spatially dynamic illumination for camera systems
US20140300803A1 (en) * 2013-04-08 2014-10-09 Lustrous Electro-Optic Co., Ltd. Image-capturing system

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201024906A (en) * 2008-12-26 2010-07-01 Taiwan Green Point Entpr Co Wireless communication device with adjustable light compensation

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12348875B2 (en) 2021-09-24 2025-07-01 Apple Inc. Adaptive-flash photography, videography, and/or flashlight using camera, scene, or user input parameters

Also Published As

Publication number Publication date
GB2531299B (en) 2017-09-13
KR20160044405A (en) 2016-04-25
GB201418279D0 (en) 2014-11-26

Similar Documents

Publication Publication Date Title
EP3010223B1 (en) Apparatus for illuminating a scene and control method thereof
US8154650B2 (en) Electronic device having a camera flash redirector
US9007489B2 (en) Method and apparatus for image background removal and replacement
US9953428B2 (en) Digital camera unit with simultaneous structured and unstructured illumination
US9131197B2 (en) Imaging apparatus capable of controlling exposure including flash amount control of flash apparatus, and control method thereof
CN107529018B (en) Flash, electronic device with flash, and flash control method
US10212330B2 (en) Autofocusing a macro object by an imaging device
US10652450B2 (en) Focusing processing method and apparatus, and terminal device
WO2016171651A1 (en) Tunable filters
US10057503B2 (en) Image processing apparatus and method for applying a virtual light source to an image
US11037277B2 (en) Image processing apparatus, imaging apparatus, lens apparatus, and image processing method
WO2012093519A1 (en) Imaging device and light emission amount control method
CN106454149A (en) Image capturing method, device and terminal equipment
GB2531299A (en) Device for illuminating a scene and control method thereof
WO2017054475A1 (en) Image capturing method, device, and terminal
KR20190075292A (en) Method of generating composite image using a plurality of images with different exposure values and electronic device supporting the same
US11539877B2 (en) Apparatus and control method
JP7103839B2 (en) Electronic devices, control devices, control methods and control programs
KR102506363B1 (en) A device with exactly two cameras and how to create two images using the device
US20190052815A1 (en) Dual-camera image pick-up apparatus and image capturing method thereof
JP7051613B2 (en) Image pickup device and its control method
JP2012074962A (en) Imaging apparatus
US12495200B2 (en) Control apparatus, flashlight apparatus, image pickup apparatus, control method, and storage medium
US20250247604A1 (en) Image pickup apparatus, accessory apparatus, and control method thereof
JP5531433B2 (en) Imaging apparatus, imaging method, and program thereof

Legal Events

Date Code Title Description
PCNP Patent ceased through non-payment of renewal fee

Effective date: 20201015