TWI385434B - Display with integrated light meter and method for controlling display brightness - Google Patents

Description

Display with integrated photometer and method for adjusting display brightness

The invention relates to an image sensor, in particular to an image sensor integrated with an integrated ambient light meter for controlling the brightness of the display.

Electronic devices typically use displays to effectively communicate information. Depending on the application, displays of different technologies may be selected, such as liquid crystal displays (LCDs), organic light emitting diodes (OLEDs), field emission displays, and plasma displays.

Liquid crystal displays and organic light-emitting diodes are flat-panel display technologies that are thinner and lighter than conventional displays (such as cathode ray tube displays) and are therefore commonly used in modern electronic devices.

The liquid crystal display has a transmissive liquid crystal display, a reflective liquid crystal display, and a transreflective liquid crystal display according to the relative position between the light source and the liquid crystal molecules. The liquid crystal molecular array of the transmissive liquid crystal display is placed in front of and illuminated by the backlight module (referred to as a backlight). A reflective liquid crystal display reflects peripheral light by a reflector for illuminating liquid crystal molecules. Transflective liquid crystal displays use both backlight and reflective methods to illuminate liquid crystal molecules.

The light source of the backlight described above may use various light emitting elements such as a light emitting diode (LED), a cold cathode tube (CCFL) or an electroluminescence panel. Among these light-emitting elements, the light-emitting diode has been gradually paid attention to because of its high service life, low cost, high vibration tolerance, low pressure, and precise control of brightness.

The organic light emitting diode system is composed of an organic light emitting material. This material emits light when excited by current, so there is no need to use any backlight. Since the current can be actively controlled in the organic light-emitting diode, the brightness of the display composed of it can be controlled.

In order to save power consumption, especially for portable or portable electronic devices, the brightness of the display can be adaptively adjusted according to the ambient light of the display. For example, when in a dark room, the brightness of the display is dimmed; and when there is sufficient light, the brightness of the display is increased. Moreover, the controllability of the brightness of the liquid crystal display can also reduce the fatigue of the user's eyes and increase the visibility of the display. Conventional display brightness control uses a separate photo sensor to measure ambient light; a control loop is used to adjust the brightness of the display based on the measured ambient light.

Image sensors are one of the most important electronic components in modern electronic devices. Semiconductor-related image sensors, such as charge-coupled devices (CCDs) or complementary MOS image sensors (often referred to as CIS), are commonly used in cameras or cameras to convert visible light images into electronic signals. And subsequently stored, transferred or displayed. Since the power consumption of image sensors is much higher than that of separate light sensors, image sensors are rarely used to control backlights, especially for portable electronic devices or battery devices.

For electronic devices that contain image sensors, it is still necessary to use separate peripheral light sensors to adjust the display brightness. This increases the cost, space, and complexity of the device design. For these reasons, it is desirable to propose a novel method and apparatus that uses an image sensor and automatically adjusts the display brightness based on ambient light.

In view of the above, it is an object of the present invention to provide a method and apparatus in which an image sensor automatically adjusts display brightness based on ambient light. Thereby, system cost and space can be reduced, and power consumption can be saved.

In accordance with an embodiment of the present invention, the present invention provides an image sensor having an integrated photometer to control display brightness. When the image sensor is not capturing and outputting the image data, since the photodiode of the image sensor continuously exposes and collects the optical signal, the optical signal can be directly measured by integrating the peripheral photometer. When the image sensor captures and outputs the image data, an automatic exposure (AE) control loop is used to estimate the required exposure gain product (EGP) to approximate the light intensity. For image sensors that do not have an on-chip (AE) control loop, an off-chip auto-exposure (AE) control loop can be used to calculate and set the exposure of the image sensor. And gain to approximate the light intensity. The image sensing chip includes a dedicated light intensity measuring circuit that quantifies the light intensity when the exposure and gain cannot be adjusted. It is correlated with the integrated photometer by dedicated circuit measurement or by the light intensity of the EGP, and is used to control the display brightness.

FIG. 1A shows a device according to an embodiment of the present invention, wherein an image sensor 12 is integrated with an integrated ambient light meter, and the liquid crystal display (LCD) 11 can be automatically adjusted according to peripheral light (or other). Display brightness of the form of the display). In this embodiment, image sensor 12 and other blocks in the drawing collectively function as a photometer. In the present embodiment, image sensor 12 is a complementary gold-oxygen (CMOS) image sensor (often abbreviated as CIS), however, other forms of sensors, such as a charge coupled device (CCD), may be used. The integrated peripheral photometer has two modes: in mode I, the image sensor 12 captures and outputs image data (ie, imaging); in mode II, the image sensor 12 is not used to capture and output images. Information (ie, non-imaging).

Mode I

In the embodiment shown in FIG. A, an on-chip automatic exposure (AE) control loop 18A and image sensor 120 are fabricated on the same wafer 120. The automatic exposure (AE) control loop 18A includes dedicated circuitry and/or algorithms for measuring the intensity of light impinging on the image sensor 12. The measurement is used to control the exposure time of the image sensor 12 and the gain of the sensing signal, so that the image generated by the image sensor 12 can be appropriately exposed. The algorithm of the automatic exposure (AE) 18A and the image signal processor (ISP) 14 are all conventional techniques, and therefore, the details are omitted except for those related to the embodiment.

When the image sensor 12 is capturing and outputting the image data, the signal generated by the light of the photodetector irradiated by the image sensor 12 is amplified and read, and the digital equivalent of the signal The value is output via an analog signal chain 13 . The output digital signal is usually further processed by the image signal processor (ISP) and sent to the video port or video bus 15 . Next, the display controller 16 (which is a primary video signal generator) detects the digital signal and feeds it to the liquid crystal display driver 17, which is driven to display the image on the display 11.

When the image sensor 12 is capturing and outputting the image data, the light intensity of the surrounding scene can be directly measured from the image data. In the present embodiment, the auto exposure (AE) control loop 18A uses analog gain, digital gain, integration time to control the exposure time (ie, the exposure time of the image sensor 12), and control image sensing. The output (analog/digital) gain of the output signal of the device 12. The automatic exposure (AE) control loop 18A compares the measured light brightness with a programmable target light intensity to obtain an exposure time, an analog/digital gain, and a measurement brightness tendency. Programmable target brightness. The exposure gain product (EGP) of the automatic exposure (AE) is obtained by multiplying the exposure time by the (analog and digital) total gain. The first B diagram illustrates a typical exposure gain product (EGP) curve showing the relationship between EPG and illuminance (or light intensity). According to this curve, EGP has a substantially linear relationship with illuminance, so EGP can be used to determine the light intensity of the surrounding light. Next, the EGP curve of the first B-picture is quantized and digitized and stored (block 19). When the system of Figure A operates, block 19 determines the light intensity of the ambient light and then compares it to a predetermined threshold in block 20 for controlling backlight module 10. In some embodiments, however, the brightness of display 11 can be controlled by adjustment of the light source; in still other embodiments, it can be controlled by adjustment of the brightness of the display pixels.

The first C diagram shows a detailed block diagram of block 19 of the first A diagram. The data for the EGP curve is first input analogous to the digital converter (ADC) 190. The converted digital signal is latched by a latch 192 and sequentially stored by a counter 196 in a user accessible scratchpad 194.

Figure 2A shows a device in accordance with another embodiment of the present invention in which the auto-exposure (AE) control loop 18B and the ISP wafer 140 are on the same wafer, but the image sensor 12 is on a different wafer. This automatic exposure (AE) control loop 18B determines and provides the appropriate exposure time, analog gain, and digital gain to the sensor.

In this embodiment, the EGP curve is as shown in the second B-picture, which is the same as the curve of the first B-picture. According to this EGP curve (second B picture), EGP has a substantially linear relationship with illuminance, so EGP can be used to determine the light intensity of the peripheral light. In the automatic exposure (AE) control loop, exposure time and gain must be limited to optimize image quality. When the exposure and gain products cannot be adjusted, the intensity of the light captured by the sensor is directly related to the situation of the surrounding light. In this case, the image sensor can quantify the light intensity by means of separate measurement circuits. The EGP curve (Fig. B) and the measured light intensity are then quantized/digitized and stored (block 19).

Continuing with the embodiment of Figure A, its automatic exposure (AE) control loop 18A controls the exposure gain product (EGP) based on the target object in the scene. This scene can be divided into multiple windows, with an automatic exposure (AE) algorithm giving each window an appropriate weighting factor. The weighting factor determines the extent to which each window affects the normalized measured brightness of the target scene. Thereby, according to the target object of the scene, the EGP thus provides an integrated photometer for measuring the intensity of the peripheral light. Since the weight coefficient of each window is adjustable, the display brightness can be flexibly controlled according to different states and applications.

Mode II

When the image sensor 12 does not capture and output the image data, the analog signal chain circuit 13, the image signal processor (ISP) 14, the video port 15 and the automatic exposure (AE) control loop 18A /B becomes idle, as shown in Figure A, where the dashed box indicates that the block is idle. In this mode, the image sensor 12 continuously draws light without starting the output path, thereby enabling low power operation. Since the photodiode of the image sensor 12 continuously collects light, the integrated peripheral photometer described above can be used to directly measure the optical signal.

The third B picture shows the resetting and integration of the image sensor 12 when the image data is not captured and output. In this mode, all of the photodiodes 126 of the image sensor 12 pass through a turn-on reset gate 121 and a transfer gate 122 to obtain a peripheral light intensity. The control gate 124 is first reset (turned on to the A position) for a period of time to connect to the supply voltage. Next, the control gate 124 forms an open circuit (located at the B position) such that the reset voltage can vary according to the charge of the photodiode 126.

In this mode, an intensity curve as shown in the third C-picture is obtained, which shows the relationship between the optical signal (voltage value) and the illuminance (or peripheral light intensity). The third D graph shows the difference between the intensity curve of the third C map and the EGP curve of the first A/B map. The EGP curve of the first A/B map can be correlated or approximated to the intensity curve of the third C map by a programmable gain factor.

For the various embodiments described above, the output signal of block 20 can be coupled to the INTERRUPT pin of the control device of backlight 10. Thereby, each time the preset threshold value of the block 20 is met, the measured light intensity of the peripheral light can be read from the block 19, and an interrupt signal is generated to adjust the backlight 10. In another embodiment, a conventional pulse width modulation (PWM) circuit is used. That is, each time the preset threshold of block 20 is met, the measured light intensity of the ambient light can be read from block 19 and a PWM signal can be generated to adjust the backlight 10.

The various embodiments described above can also be used to measure individual color channels of image sensor 12 to provide an integrated color photometer to perform sensing functions of color light. The measured individual color channels can be used to control individual color illuminating elements within backlight 10, respectively.

According to the above mode I and mode II, the image sensor 12 integrated with the low power photometer can automatically adjust the display brightness according to the peripheral light in two modes. Thereby, the image sensor 12 of the present invention can not only integrate a low-power peripheral photometer; and the system of the present invention can be greatly reduced compared to the conventional system using a separate photo sensor. Use space and cost.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the invention should be included in the following Within the scope of the patent application.

10. . . Backlight

11. . . LCD Monitor

12. . . Image sensor

13. . . Analog signal chain circuit (chain)

14. . . Image Signal Processor (ISP)

15. . . Video/video bus

16. . . Display controller

17. . . Driver

18A. . . (same wafer) automatic exposure (AE) control loop

18B. . . (different wafers) automatic exposure (AE) control loop

19. . . Digitalization and storage of light intensity

20. . . Compared with the critical value

120. . . (image sensor) chip

121. . . Reset gate

122. . . Transfer gate

124. . . Control gate

126. . . Light diode

140. . . (ISP) chip

190. . . Analog to digital converter (ADC)

192. . . Latch

194. . . Register

196. . . counter

FIG. 1A shows a device according to an embodiment of the present invention, wherein the image sensor is integrated with a peripheral photometer to automatically adjust the display brightness of the liquid crystal display according to the peripheral light.

The first B diagram illustrates a typical exposure gain product (EGP) curve showing the relationship between EPG and illuminance.

The first C diagram shows a detailed block diagram of block 19 of the first A diagram.

Figure 2A shows a device in accordance with another embodiment of the present invention with an automatic exposure (AE) control loop and an image sensor located on different wafers.

The second B diagram illustrates an exposure gain product (EGP) curve showing the relationship between EPG and illuminance.

The third A figure shows that when the image sensor does not capture and output the image data, the brightness of the display is controlled by integrating the peripheral photometer.

Figure 3B shows the reset and integration of the image sensor when image data is not captured and output.

The third C diagram illustrates an intensity curve showing the relationship between the optical signal (voltage value) and the illuminance.

The third D graph shows the difference between the intensity curve of the third C map and the EGP curve of the first A/B map.

10. . . Backlight

11. . . LCD Monitor

12. . . Image sensor

13. . . Analog signal chain circuit (chain)

14. . . Image Signal Processor (ISP)

15. . . Video/video bus

16. . . Display controller

17. . . Driver

18A. . . (same wafer) automatic exposure (AE) control loop

19. . . Digitalization and storage of light intensity

20. . . Compared with the critical value

120. . . (image sensor) chip

140. . . (ISP) chip

Claims (19)

A display with an integrated photometer includes: an image sensor; and an automatic exposure (AE) control loop, the automatic exposure (AE) control loop measures the peripheral light when the image sensor captures an image The intensity is used to adjust the brightness of the display; wherein the image sensor does not capture the image, the image sensor continues to expose and collect the optical signal for measuring the peripheral light intensity; wherein the image sensor is A plurality of photodiodes are included, each of the photodiodes corresponding to a transmission gate and a reset gate; wherein the image sensor does not capture an image, the transmission gate and the reset gate of the photodiode are turned on, To output the optical signal. The display with integrated photometer according to claim 1, wherein the automatic exposure (AE) control loop and the image sensor are located on the same wafer. The display with integrated photometer according to claim 1, wherein the automatic exposure (AE) control loop and the image sensor are located on different wafers. The display with integrated photometer according to claim 1 of the patent application, when the image sensor is capturing images, the automatic exposure (AE) control loop obtains an exposure gain product (EGP). If the image sensor does not capture an image as described in claim 4, an intensity curve is obtained, which indicates the relationship between the optical signal of the image sensor and the intensity of the peripheral light. . A display with an integrated photometer as described in claim 5 of the patent application, further comprising associating the exposure gain product (EGP) with the intensity curve. The display with integrated photometer according to claim 6 of the patent application scope further includes a digitizing device for digitizing and storing the intensity curve or the associated exposure gain product (EGP). The display with integrated photometer according to claim 7 further includes a threshold device for comparing the data read by the digitizing device with a predetermined threshold. The display with integrated photometer according to claim 8 of the patent application scope further includes a backlight module controlled by the output of the threshold value device. The display module with integrated photometer according to claim 9 of the patent application, the backlight module further comprises an interrupt pin, which is read by the digitizing device whenever the comparison standard of the threshold device is met. The data is used to generate an interrupt signal to adjust the backlight module. The display with integrated photometer according to claim 9 of the patent application scope further includes a pulse width modulation (PWM) circuit, and when the comparison standard of the threshold device is met, the data read by the digitizing device is used. A PWM signal is generated to adjust the backlight module. A method for adjusting display brightness, comprising: providing an image sensor; when the image sensor captures an image, using an automatic exposure (AE) control loop to measure the ambient light intensity; when the image sensor is used When the image is not captured, the image sensor collects the optical signal to measure the ambient light intensity; and adjusts the display brightness according to the peripheral light intensity; wherein the image sensor includes a plurality of light diodes, each of which The light diode corresponds to a transmission gate and a reset gate. When the image sensor does not capture the image, the transmission gate and the reset gate of the photodiode are turned on to output the optical signal. A method of adjusting display brightness as described in claim 12, wherein the automatic exposure (AE) control loop and the image sensor are located on the same wafer. A method of adjusting display brightness as described in claim 12, wherein the automatic exposure (AE) control loop and the image sensor are located on different wafers. The method of adjusting display brightness as described in claim 12, when the image sensor is capturing images, the automatic exposure (AE) control loop obtains an exposure gain product (EGP). For example, in the method of adjusting the display brightness as described in claim 15, when the image sensor does not capture the image, an intensity curve is obtained, which indicates the relationship between the optical signal of the image sensor and the intensity of the peripheral light. The method of adjusting the display brightness as described in claim 16 of the patent application further includes associating the exposure gain product (EGP) with the intensity curve. The method of adjusting the display brightness as described in claim 17 further includes a step of digitizing and storing the intensity curve or the associated exposure gain product (EGP). The method for adjusting the display brightness as described in claim 18 of the patent application further includes a step of comparing the output of the digitized intensity curve with a predetermined threshold.