HK1228139A1 - Method and system to detect a light-emitting diode - Google Patents

Abstract

The subject application relates to a method and system to detect a light-emitting diode. A method of detecting light-emitting diode (LED) light starts with a control circuitry generating a shutter signal that is transmitted to a pixel array to control image acquisition by the pixel array and to establish a set exposure time. The readout circuitry may then read out the image data from the pixel array that includes reading out the image data from a plurality of successive and overlapped frames having the set exposure time. The set exposure time may be the same for each of the frames. The successive and overlapped frames may be interlaced frames. Other embodiments are also described.

Description

Method and system for detecting light emitting diode

Technical Field

Examples of the invention relate generally to image sensors. More specifically, examples of the disclosure relate to methods and systems to detect Light Emitting Diodes (LEDs) so that flicker-free image capture may be performed.

Background

High-speed image sensors have been widely used in many applications in different fields including the automotive field, the machine vision field, and the professional video photography field. Some of these applications require the detection and capture of LED lights, which has proven difficult. Given that the LED pulses are narrow (e.g., 1 millisecond (ms)), conventional high-speed image sensors may miss the LED lamp or capture images containing flicker.

Disclosure of Invention

Aspects of the invention relate to a method of detecting a Light Emitting Diode (LED) by an imaging system, the method comprising: generating, by a control circuit, a shutter signal that is emitted to a pixel array to control image acquisition by the pixel array and establish a set exposure time; and reading out image data from the pixel array by a readout circuit, including reading out the image data from a plurality of consecutive and overlapping frames having the set exposure time, wherein the set exposure time is the same for each of the frames.

In another aspect of the present invention, a method of detecting a Light Emitting Diode (LED) by an imaging system includes: simultaneously enabling all pixels within the pixel array to simultaneously capture image data during a single acquisition window; and reading out image data from the pixel array, including reading out the image data from a plurality of consecutive and interlaced frames having the same exposure time.

In yet another aspect of the present invention, an imaging system to detect Light Emitting Diodes (LEDs) includes: a pixel array for acquiring image data, the pixel array comprising a plurality of rows and columns; and readout circuitry coupled to the color pixel array to readout image data from the pixel array, including readout of the image data from multiple consecutive and interleaved frames having the same exposure time.

Drawings

Embodiments of the present invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements throughout the various views unless otherwise specified. It should be noted that references to "an" or "one" embodiment of the invention in this disclosure are not necessarily to the same embodiment, and they mean at least one. In the drawings:

FIG. 1 is a block diagram illustrating an exemplary imaging system detecting LEDs, according to one embodiment of the invention.

Fig. 2 is a block diagram illustrating details of the sensing circuit in fig. 1 according to one embodiment of the invention.

Fig. 3 is a timing diagram illustrating capturing LED lights according to one embodiment of the present invention.

FIG. 4 is a timing diagram illustrating synchronizing a video frequency with a street light LED frequency according to one embodiment of the present invention.

Fig. 5 is a timing diagram illustrating LED communication between automobiles, according to one embodiment of the present invention.

Fig. 6 is a flow chart illustrating a method of detecting an LED according to one embodiment of the present invention.

Corresponding reference characters indicate corresponding parts throughout the several views of the drawings. Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help improve understanding of various embodiments of the present invention. Additionally, common but well-understood elements that are useful or necessary in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various embodiments of the present invention.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown to avoid obscuring the understanding of this description.

Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. The particular features, structures, or characteristics may be included in integrated circuits, electronic circuits, combinational logic circuits, or other suitable components that provide the desired functionality.

FIG. 1 is a block diagram illustrating an exemplary imaging system 100 for detecting and capturing LEDs, according to one embodiment of the invention. The imaging system 100 may be a complementary metal oxide semiconductor ("CMOS") image sensor. Since the LED pulses are narrow (e.g., 1ms), conventional imaging systems may miss the LED lamp or capture an image containing flicker due to the LED lamp. In one embodiment, imaging system 100 ensures that imaging system 100 captures frames continuously by detecting LEDs using consecutive and overlapping frames rather than a single frame. For example, consecutive and overlapping frames may be interlaced frames. In one embodiment, each of the consecutive and overlapping frames may have the same exposure time and the minimum overlap time of the frames is the sum of the LED pulse width (Tled) and the frame transfer time (Tft). The LED pulse width (Tled) may be 1ms and the frame transfer time (Tft) may be 1 ms. In one embodiment, a high frame transfer time (Tft) is preferred to minimize the effect of background. In this embodiment, a hybrid stacked chip may be used. The hybrid stacked chip may have a frame transfer time (Tft) of 1000 frames per second (fps), 60fps input-output (IO), and 1 frame buffer.

As shown in the depicted example in fig. 1, the imaging system 100 includes a pixel array 105 coupled to control circuitry 120 and readout circuitry 110 (which is coupled to functional logic 115 and logic control 108).

The illustrated embodiment of pixel array 105 is a two-dimensional (2D) array of imaging sensors or pixel cells (e.g., pixel cells P1, P2 …, Pn). In one example, each pixel cell is a CMOS imaging pixel. As illustrated, each pixel cell is arranged in rows (e.g., rows R1-Ry) and columns (e.g., columns C1-Cx) to capture image data of a person, place, or object, etc., which can then be used to render an image of the person, place, or object, etc.

In one example, after each pixel has acquired its image data or image charge, the image data is read out by readout circuitry 110 through readout column bitlines 109 and then transferred to functional logic 115. In one embodiment, the logic circuitry 108 may control the readout circuitry 110 and output image data to the functional logic 115. In various examples, readout circuitry 110 may include amplification circuitry (not illustrated), analog-to-digital conversion (ADC) circuitry 220, or others. Function logic 115 may simply store the image data or even manipulate the image data by applying a post-image effect (e.g., crop, rotate, remove red-eye, adjust brightness, adjust contrast, or otherwise). In one example, readout circuitry 110 may readout a row of image data at a time along readout column lines (illustrated) or may readout the image data using various other techniques (not illustrated), such as a serial readout or a full parallel readout of all pixels simultaneously.

In one example, control circuitry 120 is coupled to pixel array 105 to control operating characteristics of pixel array 105. For example, the control circuit 120 may generate a shutter signal for controlling image acquisition. In one example, the shutter signal is a global shutter signal used to simultaneously enable all pixels within the pixel array 105 to simultaneously capture their respective image data during a single acquisition window. In another example, the shutter signal is a rolling shutter signal such that each row, column, or group of pixels is sequentially enabled during successive acquisition windows. The shutter signal may also establish an exposure time, which is the length of time the shutter remains open. In one embodiment, the exposure time is set to be the same for each of the frames.

Fig. 2 is a block diagram illustrating details of readout circuitry 110 of imaging system 100 in fig. 1, according to one embodiment of the invention. As shown in fig. 2, the readout circuit 110 may include a scan circuit 210 and an ADC circuit 220. The scan circuitry 210 may include amplification circuitry, selection circuitry (e.g., multiplexers), etc. to read out a row of image data at a time along the read column bitlines 109 or may read out the image data using various other techniques, such as serial readout or simultaneous full parallel readout of all pixels. The ADC circuit 220 may convert each of the image data from the scanning circuit 210 from analog to digital. In one embodiment, readout circuitry 110 reads out image data from the pixel array, including reading out image data from two or more consecutive and overlapping frames having set exposure times. In other words, the set exposure time is the same for each of the frames. In one embodiment, the two or more consecutive and overlapping frames are interlaced frames. For the interlaced frame, the readout circuit 110 can read out the interlaced frame with odd-numbered even lines and read out the interlaced frame with even-numbered odd lines. Similarly, the readout circuit 110 can also read out odd-numbered interlaced frames and even-numbered interlaced frames. As shown in fig. 3, a timing diagram illustrates the capture of an LED lamp according to one embodiment of the invention. In fig. 3, the LED lamp is fully captured by (Frame _ n,1) and (Frame _ n +1,0), where the odd rows of (Frame _ n,1) are read out and the even rows of (Frame _ n +1,0) are read out. In one embodiment, each of the consecutive and overlapping frames may have the same exposure time and the minimum exposure time overlap between two or more consecutive and overlapping frames is the sum of the LED pulse width (Tled) and the frame transfer time (Tft). The LED pulse width (Tled) may be 1ms and the frame transfer time (Tft) may be 1 ms. In one embodiment, a high frame transfer time (Tft) is preferred.

As discussed above, functional logic 115 may simply store the image data or even manipulate the image data by applying a post-image effect. In one embodiment, functional logic 115 further receives an image data readout from readout circuitry 110 and determines whether to capture an LED light based on the image data readout. To capture a flicker free image, function logic 115 may calculate the frequency of the LED lamp based on a determination of the number of times the LED lamp was captured and the identification of the frame in which the LED lamp was captured. Functional logic 115 may then synchronize imaging system 100 with the calculated frequency of the LED lamp. In one embodiment, the image frames per second (fps) and LED frequencies preferably have small skew. FIG. 4 is a timing diagram illustrating synchronizing a video frequency with a street light LED frequency according to one embodiment of the present invention. On the left side of the timing diagram in fig. 4, the video avoids the LED lamp pulses because the frame is not synchronized with the street lamp LED frequency. For example, in a rolling shutter image sensor, the lights may blink during the non-exposure time of any of the rows of the image sensor. On the right side of the timing diagram in fig. 4, once functional logic 115 calculates the LED frequency and synchronizes the video frequency with the street light LEDs, the video frame is able to capture a flicker-free image of the street light LED lights. In one embodiment, the street light LEDs may be synchronized with the AC power at some offset. In one embodiment, the functional logic 115 in the imaging system 100 may perform LED detection periodically.

The imaging system 100 may also be used for LED communication between automobiles. Fig. 5 is a timing diagram illustrating LED communication between automobiles, according to one embodiment of the present invention. As shown in fig. 5, the timing diagram for the LEDs in one transaction may include a plurality of header bits 510, data bits 520, and a plurality of footer bits 530. In one embodiment, the functional logic may identify a plurality of cars from the image data based on the detected LEDs and perform LED communication of the identified cars. In this embodiment, many-to-one (M:1) communication may be performed.

Furthermore, the following embodiments of the invention may be described as a process which is usually depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. Additionally, the order of the operations may be rearranged. A process is terminated when its operations are completed. A process may correspond to a method, a program, etc.

Fig. 6 is a flow chart illustrating a method 600 of detecting an LED according to one embodiment of the present invention. The method or process 600 begins at block 601 with the following operations: the control circuit 120 generates a shutter signal that is transmitted to the pixel array 105 to control image acquisition by the pixel array 105 and establish a set exposure time. The shutter signal may be a global shutter signal that enables all pixels in the pixel array 105 at the same time. The shutter signal may also enable all pixels in the pixel array 105 to simultaneously capture image data during a single acquisition window. At block 602, readout circuitry 110 reads out image data from pixel array 105, which includes reading out image data from multiple consecutive and overlapping frames having set exposure times. The set exposure time may be the same for each of the frames. In one embodiment, the consecutive and overlapping frames are interlaced frames. In this embodiment, reading out image data from the frame includes reading out an even row odd numbered frame and reading out an odd row even numbered frame by readout circuitry 110, or vice versa. The consecutive and overlapping frames may overlap by a minimum exposure overlap time. This minimum overlap time may be the sum of (a) the LED pulse width (Tled) and (b) the frame transit time (Tft). In one embodiment, control circuitry 120, pixel array 105, and readout circuitry 110 are included in an imaging system that is a hybrid stacked chip with higher frame transfer times. At block 603, functional logic 115 determines whether to capture an LED based on the readout from readout circuitry 110. To capture a flicker-free image of the LED, at block 604, functional logic 115 determines the frequency of the LED based on the number of times the LED was captured and the identification of the frame in which the LED was captured and at block 605, functional logic 115 synchronizes imaging system 100 with the frequency of the LED. To enable LED communication, at block 606, functional logic 115 identifies a plurality of cars based on the detected LEDs to perform LED communication of the cars.

The process explained above is described in terms of computer software and hardware. The described techniques may constitute machine-executable instructions embodied within a machine (e.g., computer) readable storage medium, which when executed by a machine, will cause the machine to perform the operations described. Additionally, the processes may be embodied within hardware such as an application specific integrated circuit ("ASIC") or the like.

The above description of illustrated examples of the invention, including what is described in the Abstract of the disclosure, is not intended to be exhaustive or to be limited to the precise forms disclosed. Although specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various equivalent modifications can be made without departing from the broader spirit and scope of the invention.

These modifications can be made to the examples of the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification and the claims. Rather, the scope is to be determined entirely by the following claims, which are to be construed in accordance with established doctrines of claim interpretation. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims (26)

1. A method of detecting a light emitting diode, LED, by an imaging system, comprising:

generating, by a control circuit, a shutter signal that is emitted to a pixel array to control image acquisition by the pixel array and establish a set exposure time; and

reading out image data from the pixel array by a readout circuit, including reading out the image data from a plurality of consecutive and overlapping frames having the set exposure time, wherein the set exposure time is the same for each of the frames.

2. The method of claim 1, wherein the shutter signal is a global shutter signal that enables all pixels in the pixel array simultaneously.

3. The method of claim 1, wherein the plurality of consecutive and overlapping frames are interlaced frames.

4. The method of claim 3, wherein reading out image data from the frame by the readout circuitry comprises:

reading out odd-numbered frames of even-numbered lines by the readout circuit; and

the even-numbered frames of the odd rows are read out by the readout circuit.

5. The method of claim 3, wherein reading out image data from the frame by the readout circuitry comprises:

reading out odd-numbered frames of odd-numbered rows by the readout circuit; and

the even numbered frames of the even rows are read out by the readout circuit.

6. The method of claim 1, wherein the consecutive and overlapping frames overlap by a minimum overlap time, wherein the minimum overlap time is a sum of (i) a Light Emitting Diode (LED) pulse width and (ii) a frame transfer time.

7. The method of claim 1, wherein the imaging system is a hybrid stacked chip with higher frame transfer times.

8. The method of claim 1, further comprising:

determining, by functional logic, whether to capture the LED based on the readout from the readout circuit.

9. The method of claim 8, further comprising:

calculating a frequency of the LED based on the number of times the LED was captured and an identification of a frame in which the LED was captured; and

synchronizing an image sensor with the frequency of the LED.

10. The method of claim 8, further comprising:

identifying a plurality of cars to perform LED communication of cars based on the detected LEDs.

11. A method of detecting a light emitting diode, LED, by an imaging system, comprising:

simultaneously enabling all pixels within the pixel array to simultaneously capture image data during a single acquisition window; and

reading out image data from the pixel array includes reading out the image data from a plurality of consecutive and interlaced frames having the same exposure time.

12. The method of claim 11, further comprising:

generating, by a control circuit, a shutter signal that is emitted to the pixel array to control image acquisition by the pixel array, wherein the shutter signal is a global shutter signal.

13. The method of claim 11, wherein reading out the image data from the consecutive and interleaved frames comprises:

reading out the even-numbered lines of the first frame by a readout circuit; and

odd-numbered rows of a second frame are read out by the readout circuitry, wherein the first frame and the second frame are consecutive and interleaved.

14. The method of claim 13, wherein the consecutive and interleaved frames overlap by a minimum overlap time, wherein the minimum overlap time is a sum of (i) a Light Emitting Diode (LED) pulse width and (ii) a frame transfer time.

15. The method of claim 14, wherein the imaging system is a hybrid stacked chip with higher frame transfer times.

16. The method of claim 11, further comprising:

determining, by functional logic, whether to capture the LED based on the readout from the readout circuit.

17. The method of claim 16, further comprising:

calculating a frequency of the LED based on the number of times the LED was captured and an identification of a frame in which the LED was captured; and

synchronizing the imaging system with the frequency of the LED.

18. The method of claim 17, further comprising:

identifying a plurality of cars to perform LED communication of cars based on the detected LEDs.

19. An imaging system to detect light emitting diodes, LEDs, comprising:

a pixel array for acquiring image data, the pixel array comprising a plurality of rows and columns; and

readout circuitry coupled to a color pixel array to readout image data from the pixel array, including readout of the image data from multiple consecutive and interleaved frames having the same exposure time.

20. The imaging system of claim 19, further comprising:

control circuitry to generate and transmit shutter signals to a pixel array to control image acquisition by the pixel array and establish the exposure time, wherein the shutter signals are global shutter signals.

21. The imaging system of claim 19, wherein reading out the image data from the consecutive and interleaved frames comprises:

reading out the even-numbered lines of the first frame; and

reading out odd-numbered lines of a second frame, wherein the first frame and the second frame are consecutive and interleaved.

22. The imaging system of claim 21, wherein the consecutive and interleaved frames overlap by a minimum overlap time, wherein the minimum overlap time is a sum of (i) a Light Emitting Diode (LED) pulse width and (ii) a frame transfer time.

23. The imaging system of claim 19, further comprising:

functional logic to receive the image data readout from the readout circuitry and determine whether to capture the LED based on the readout of image data.

24. The imaging system of claim 23, wherein the functional logic is further to

Calculating the frequency of the LEDs based on the number of times the LEDs are captured and the identification of the frame in which the LEDs are captured, an

Synchronizing the imaging system with the frequency of the LED.

25. The imaging system of claim 24, wherein the functional logic is further to

Identifying a plurality of cars to perform LED communication of cars based on the detected LEDs.

26. The imaging system of claim 19, further comprising:

a logic circuit to control the readout circuit and output image data from the readout circuit to functional logic, and

wherein the readout circuit comprises: a scan circuit to select a row of pixels to be read out from the pixel array; and an analog-to-digital conversion ADC circuit to convert each of the image data from analog to digital.