TWI857988B - Image sensor, recording device, and resetting method - Google Patents

Abstract

The present invention improves the accuracy of the reference level used for event detection and seeks to improve the accuracy of event detection. The image sensor of the present invention comprises: a light receiving unit, which obtains an electrical signal corresponding to the amount of light received as a light receiving signal; an event detection unit, which uses the level of the past light receiving signal as a reference level, obtains the difference with the level of the current light receiving signal, and treats the change of the amount of light received as an event; a holding unit, which inputs a detection signal indicating the detection of an event from the event detection unit and holds it; a reading unit, which reads the detection signal held in the holding unit as an event signal; and a reset unit, which resets the reference level to the current light receiving signal level of the light receiving unit after the event detection unit detects the event and before the reading unit reads the event signal.

Description

Image sensor, recording device, and resetting method

The present invention relates to an asynchronous image sensor that detects changes in the amount of received light as an event and selectively reads signals from pixels that have detected the event, a recording device having the image sensor, and a method for resetting event detection of the image sensor.

Synchronous image sensors that acquire image data (frames) synchronously with a synchronization signal such as a vertical synchronization signal are widely used. In this synchronous image sensor, since the image data acquisition cycle is limited to the synchronization signal cycle (e.g., 1/60 second), it is difficult to cope with the situation requiring higher-speed processing in fields related to transportation or robots. Therefore, the industry has proposed an asynchronous image sensor in which a detection circuit is provided for each pixel to detect the change of the light receiving amount of the pixel in real time as an event (e.g., refer to the following patent document 1). For example, as known as DVS (Dynamic Vision Sensor), in this type of asynchronous image sensor, the change in the amount of received light is detected as an event, and the signal of the pixel where the event is detected is selectively read out, thereby speeding up the acquisition of image data. In addition, since only the signal of the pixel where the event is detected needs to be read out, power saving is sought.

[Prior Art Literature] [Patent Literature]

[Patent Document 1] Japanese Patent Publication No. 2017-535999

Here, the event detection is performed by taking the level of the past light-receiving signal as the reference level and obtaining the difference between the reference level and the current light-receiving signal level. When an event is detected, the arbitrator is notified of the intention and the event detection signal of the pixel is read. Furthermore, when an event is detected, the reference level is updated (reset) to the current light-receiving signal level so that a new event can be detected.

Here, the reset of the reference level is considered to be performed in accordance with the reading of the event signal.

However, since it takes time for the arbitrator to read out the signal indicating the pixel after receiving the notification of event detection, if the reference level is reset in accordance with the readout as described above, a delay occurs when resetting the reference level. In other words, the reference level cannot be quickly updated to the light-receiving signal level at the time of event detection, and there is a risk that the accuracy of the reference level will decrease. When the accuracy of the reference level decreases, there is a risk that the accuracy of event detection will decrease, such as causing missed event detection.

This invention was completed in view of the above situation, and its purpose is to provide a method to improve the accuracy of the benchmark level used for event detection, and to improve the accuracy of event detection.

The image sensor of the present invention comprises: a light receiving unit, which obtains an electrical signal corresponding to the amount of light received as a light receiving signal; an event detection unit, which uses the level of the previous light receiving signal as a reference level, obtains the difference between the level of the previous light receiving signal and the level of the current previous light receiving signal, and detects the change of the above-mentioned light receiving amount as an event; a holding unit, which inputs a detection signal indicating the detection of the above-mentioned event from the above-mentioned event detection unit and holds it; a reading unit, which reads the above-mentioned detection signal held in the above-mentioned holding unit as an event signal; and a reset unit, which resets the above-mentioned reference level to the current light receiving signal level of the above-mentioned light receiving unit after the above-mentioned event detection unit detects the above-mentioned event and before the above-mentioned reading unit reads the above-mentioned event signal.

In this way, the reference level used for event detection can be quickly reset (updated) to the current light-receiving signal level in accordance with event detection.

In the image sensor of the present invention, the reset unit resets the reference level based on the detection signal output by the event detection unit.

In this way, the reference level can be reset after the event detection unit detects the event and before the reading unit reads the event signal.

In the above-mentioned image sensor of the present invention, it is considered that the aforementioned reset unit resets the aforementioned reference level based on a signal that delays the aforementioned detection signal.

In this way, the timing of resetting the reference level (i.e., the timing of starting detection for a new event) can be delayed until the level of the detection signal held in the holding unit reaches a certain level or above.

In the above-mentioned image sensor of the present invention, it is considered that the aforementioned reset unit resets the aforementioned reference level based on the output signal of the aforementioned holding unit.

In this way, the reference level is reset in response to the level of the detection signal held in the holding unit becoming above a certain level.

In the above-mentioned image sensor of the present invention, it is considered to have a discontinuing unit, which makes the output of the aforementioned detection signal from the aforementioned event detection unit to the aforementioned holding unit intermittent in accordance with the level of the aforementioned detection signal held in the aforementioned holding unit.

This allows control to be performed to appropriately output event signals, such as blocking the output of the detection signal to the holding unit when the level of the detection signal of the holding unit reaches a certain level or above, or releasing the blocking state of the output of the detection signal to the holding unit when the event signal is read from the holding unit and the level of the detection signal of the holding unit falls below a certain level.

In the image sensor of the present invention, the output is cut off accordingly when the level of the detection signal of the interrupter and the holding unit is above a certain level.

This can prevent the detection signal of a new event generated before reading from being assimilated with the detection signal of the immediately preceding event.

In the image sensor of the present invention, the interruption part and the holding part release the output blocking state accordingly when the level of the detection signal drops below a certain level.

In this way, when the level of the detection signal of the holding unit drops below a certain level while the signal is read from the holding unit, the state in which the detection signal can be input to the holding unit can be switched, and the holding unit holds the detection signal of the new event.

In the image sensor of the present invention, the disconnection unit is configured to have a switch, which is inserted into the output line that outputs the detection signal from the event detection unit to the holding unit, and is turned on/off according to the output signal of the holding unit.

Thus, the blocking part only needs to have a switch that turns on/off according to the output signal of the holding part as a component.

In the image sensor of the present invention, the aforementioned holding unit counts and holds the number of times the aforementioned detection signal is obtained in the aforementioned event detection unit as the number of times the aforementioned event is generated, and the aforementioned reading unit reads the count value held in the aforementioned holding unit as the aforementioned event signal.

In this way, even if the waiting time from the occurrence of an event to its reading is long, a signal showing the number of times the event has occurred until it is read can be output as an event signal.

In the above-mentioned image sensor of the present invention, it is considered that there are a plurality of the aforementioned light-receiving parts, and each of the aforementioned light-receiving parts has the aforementioned event detection part, and the aforementioned reset part resets the aforementioned reference level for the plurality of aforementioned event detection parts according to a common reset signal.

In this way, the number of reset parts that should be prepared can be reduced while realizing the reset of each event detection part.

In the above-mentioned image sensor of the present invention, there are a plurality of Bayer-arranged pixels having four aforementioned light-receiving parts, and each of the aforementioned pixels has the aforementioned reset part.

Thus, when a pixel structure capable of capturing color images is adopted, the circuit scale can be reduced compared to the case where the unit of event detection is set to an appropriate unit as a pixel unit and a reset unit is provided for each event detection unit.

Furthermore, the recording device of the present invention comprises an image sensor and a recording unit for recording the event signal read by the reading unit, and the image sensor comprises: a light receiving unit for obtaining an electrical signal corresponding to the amount of light received as a light receiving signal; and an event detecting unit for detecting the change of the amount of light received as an event by obtaining the difference between the level of the previous light receiving signal and the level of the current light receiving signal based on the level of the previous light receiving signal as a reference level. ; a holding unit, which receives the detection signal indicating the detection of the aforementioned event from the aforementioned event detection unit and holds it; a reading unit, which reads the aforementioned detection signal held in the aforementioned holding unit as an event signal; and a reset unit, which resets the aforementioned reference level to the current light receiving signal level of the aforementioned light receiving unit after the aforementioned event detection unit detects the aforementioned event and before the aforementioned reading unit reads the aforementioned event signal.

According to this recording device, the same function as the image sensor of the present invention mentioned above is also obtained.

Furthermore, the reset method of the present invention uses the level of the previous light receiving signal of the light receiving unit that obtains the electrical signal corresponding to the light receiving amount as the light receiving signal as the reference level, obtains the difference from the level of the current previous light receiving signal, and detects the change of the light receiving amount as an event; inputs and holds the detection signal indicating the detection of the event; reads the held detection signal as an event signal; and after the event is detected and before the event signal is read, resets the reference level to the current light receiving signal level of the light receiving unit.

According to this reset method, the same effect as the image sensor of the present invention mentioned above can be obtained.

1: Image sensor

11: Pixel array section

12: Arbitrator

13: Reading Department

13a: Transmission Department

14:Signal processing circuit

15: Pixels

15A: Pixels

15B: Pixels

16: Light receiving part

16-1: Light receiving part

16-2: Light receiving part

16-3: Light receiving part

16-4: Light receiving part

16a: Light receiving element

16b: Logarithmic conversion unit

16c: Buffer

17: Event Detection Department

17’: Event Detection Department

17a: Subtractor

17b: Quantizer

17b’: Quantizer

18:Maintenance Department

18’: Maintaining part

18A: Holding part

18a: Holding circuit/OR gate circuit

18b:OR gate circuit

18i: Inverter

18m: Second holding circuit

18p: First holding circuit

19: Reset Department

19’: Reset Department

19A: Reset Department

19a:OR gate circuit

19B: Reset Department

19b: Delay circuit

19c: Judgment Department

20: Output control unit

20a:OR gate circuit

25: Maintenance Department

25’: Maintaining part

25a: Counter

25m: Second counter

25p: First counter

30-1: Light receiving unit

30’-1: Light receiving unit

30-2: Light receiving unit

30’-2: Light receiving unit

30-3: Light receiving unit

30’-3: Light receiving unit

30-4: Light receiving unit

30’-4: Light receiving unit

50: Controller

55: Row driver

56: Output circuit

100: Camera device

101:Photographic lens

102: Recording Department

103: Control Department

A: Node

B: Blue

C1: Capacitor

C2: Capacitor

D: Delay

Evnm: event signal

Evnm’: event signal

Evnp: event signal/output signal

Evnp’: event signal

G: Green

L1: signal line

Lc: Benchmark

Lv1: Level

Lv2: Level

Lv2’: Level

Q1~Q5: Transistor

Q7~Q12: Transistor

R: Red

RST: reset signal

SWr: Reset switch

SWm: switch

SWp: switch

T1: Time point

T2: Time point

t1~t7: Time point

TH: Threshold value

Vbdif: voltage

Vbias: bias voltage

VDD: power supply voltage

Vhigh: voltage

Vlow: voltage

Vom: output voltage

Vop: output voltage

FIG1 is a diagram showing an example of the configuration of an imaging device having an image sensor as an embodiment of the present invention.

FIG2 is a diagram showing an example of the internal structure of an image sensor as an implementation form.

FIG3 is a diagram showing an example of the configuration of a pixel array portion of an image sensor as an implementation form.

FIG. 4 is a diagram for explaining an example of the circuit configuration of a light receiving unit and an event detection unit provided in an image sensor as an implementation form.

Figure 5 is an explanatory diagram of the structure of the event signal readout in the implementation form.

FIG6 is a diagram for explaining an example of the configuration of the reset portion of the first embodiment.

Figure 7 is a timing diagram used to illustrate the operation of the circuit shown in Figure 6.

FIG8 is a diagram for explaining the meaning of quickly resetting the baseline level in response to event detection.

Figure 9 is an explanatory diagram for the variation of the detection signal retention.

FIG. 10 is a diagram for explaining an example of the internal structure of a pixel provided in an image sensor as the second embodiment.

Figure 11 is a timing diagram used to illustrate the operation of the circuit shown in Figure 10.

FIG. 12 is a diagram for explaining an example of the internal structure of a pixel provided in an image sensor as the first example of the third embodiment.

FIG. 13 is a diagram for explaining an example of the internal structure of a pixel provided in an image sensor as the second example of the third embodiment.

Figure 14 is a diagram used to illustrate the structure of an image sensor when a scanning method is used.

Figure 15 is an illustration of a situation where the amount of received light changes significantly.

Figure 16 is a diagram used to illustrate an example of the internal structure of a pixel when a counting method is used.

Figure 17 is a diagram illustrating the operation of the counter in the implementation form.

Figure 18 is a diagram illustrating the function of the counting method.

FIG19 is a diagram showing an example of the internal structure of a pixel when the time division method is applied to the counting method.

FIG. 20 is a diagram showing another example of the internal structure of a pixel when the time division method is applied to the counting method.

FIG. 21 is a diagram showing an example of the configuration of a pixel of an image sensor as a first variation.

FIG. 22 is a diagram for explaining an example of the structure of a holding portion provided in the image sensor as the first variation.

FIG. 23 is a diagram showing an example of the configuration of a pixel included in an image sensor as a second variation.

FIG. 24 is a diagram for explaining an example of the configuration of a reset portion provided in the image sensor as the second variation.

Below, with reference to the attached drawings, the implementation form of the present invention is described in the following order.

<1. First implementation form>

[1-1. Composition of the imaging device]

[1-2. Reset section of the first implementation form]

[1-3. Example of changes in detection signal retention]

<2. Second implementation form>

<3. The third implementation form>

<4. The fourth implementation form>

[4-1. Overview of scanning method]

[4-2. Counting method]

[4-3. Examples of changes related to counting methods]

<5. Examples of changes in pixel composition>

<6. Other changes>

<7. Summary of implementation forms>

<8. The present invention>

<1. First implementation form>

[1-1. Composition of the imaging device]

FIG. 1 is a diagram showing an example of the configuration of an imaging device 100 having an image sensor 1 as an embodiment of the present invention.

The imaging device 100 has an image sensor 1, an imaging lens 101, a recording unit 102, and a control unit 103. The imaging device 100 may be a camera mounted on an industrial robot or a car-mounted camera, for example.

The imaging lens 101 collects the incident light and guides it to the image sensor 1. The image sensor 1 performs photoelectric conversion on the incident light and obtains an electrical signal corresponding to the amount of light received as a light receiving signal, and detects the change in the amount of light received as an event based on the light receiving signal. Furthermore, the image sensor 1 outputs an event signal (event signals Evnp and Evnm described later) indicating the detection result of the event to the recording unit 102 via the signal line L1.

In addition, the specific internal structure of the image sensor 1 will be described later.

The recording unit 102 records the event signal output from the image sensor 1.

The control unit 103 is configured as a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and the CPU executes processing according to a program to control the operation of the imaging device 100. In particular, the control unit 103 controls the image sensor 1 to execute the output operation of the above-mentioned event signal, or controls the recording unit 102 to execute the recording operation of the event signal.

FIG2 is a diagram showing an example of the internal structure of the image sensor 1.

As shown in the figure, the image sensor 1 includes: a pixel array unit 11, an arbitrator 12, a readout unit 13, and a signal processing circuit 14.

In the pixel array unit 11, a plurality of pixels (pixels 15 described later) are arranged in a two-dimensional grid pattern. Hereinafter, a set of pixels arranged in the horizontal direction in the pixel array unit 11 is referred to as a "row", and a set of pixels arranged in a direction perpendicular to the row is referred to as a "row".

In this example, each pixel detects whether an event occurs by checking whether the change in the amount of light received exceeds a specific threshold value, and when an event occurs, a request is output to the arbitrator 12.

The readout unit 13 reads out the event signal from the pixel of the pixel array unit 11 to the signal processing circuit 14.

The arbitrator 12 mediates the pixel request from the pixel array unit 11, and controls the readout operation of the readout unit 13 based on the modulation result. Specifically, the control is performed by reading out the event signal of the pixel that generates the event.

Here, in this example, the event signal is set to be a signal quantized (binarized) by the quantizer 17b described later.

The signal processing circuit 14 performs specific signal processing on the read event signal and outputs it to the recording unit 102 via the signal line L1.

FIG3 is a diagram showing an example of the configuration of the pixel array unit 11.

As shown in the figure, in the pixel array unit 11, a plurality of pixels 15 are arranged in the column direction and the row direction. Each pixel 15 has: a light receiving unit 16, an event detection unit 17, a holding unit 18, and a reset unit 19.

The light receiving unit 16 obtains an electrical signal corresponding to the amount of light received as a light receiving signal.

The event detection unit 17 detects the change in the amount of received light as an event by taking the level of the past light-receiving signal as the reference level Lref and obtaining the difference between the level of the current light-receiving signal. Specifically, the event detection unit 17 detects the presence or absence of an event by displaying whether the level (absolute value) of the difference signal between the reference level Lref and the level of the current light-receiving signal is above a specific threshold value TH.

The event detection unit 17 of this example is configured to detect respectively: an event in which the amount of received light changes toward the increasing side, that is, an event in which the difference with the reference level Lref becomes positive (hereinafter referred to as a "conduction event"), and an event in which the amount of received light changes toward the decreasing side, that is, an event in which the difference with the reference level Lref becomes negative (hereinafter referred to as a "disconnection event").

The holding unit 18 uses the event detection unit 17 to input the detection signal (hereinafter referred to as "detection signal Sd") indicating the detection result of the event and holds it.

The holding unit 18 of this example is configured to hold the detection signal Sd indicating the detection result of the above-mentioned conduction event and the detection signal Sd indicating the detection result of the disconnection event.

The reset unit 19 resets the reference level Lref used by the event detection unit 17 for event detection to the current light receiving signal level of the light receiving unit 16. In particular, the reset unit 19 of this embodiment resets the reference level Lref after the event detection unit 17 detects the event and before the reading unit 13 reads the event signal.

By resetting the reference level Lref to the current light-receiving signal level, a new event detection can be performed based on the change in the light-receiving signal level from that point in time. That is, the resetting of the reference level Lref functions as a process for controlling the event detection unit 17 to a state in which a new event detection can be performed.

FIG. 4 is a diagram for explaining a specific circuit configuration example of the light receiving unit 16 and the event detection unit 17 of the pixel 15.

As shown in the figure, the light receiving unit 16 includes a light receiving element 16a, a logarithmic conversion unit 16b, and a buffer 16c, and the event detection unit 17 includes a subtractor 17a and a quantizer 17b.

In the light receiving unit 16, the light receiving element 16a is a photoelectric conversion element that performs photoelectric conversion on incident light to generate electric charge. In this example, a photodiode is used as the light receiving element 16a.

The logarithmic conversion unit 16b converts the photocurrent (current corresponding to the amount of light received) obtained by the light receiving element 16a as a photodiode into a voltage signal of the logarithm.

The buffer 16c corrects the voltage signal input by the logarithmic conversion unit 16b and outputs it to the subtractor 17a of the event detection unit 17.

As shown in the figure, the logarithmic conversion unit 16b has a transistor Q1, a transistor Q2, and a transistor Q3. In this example, MOSFET (MOS: Metal-Oxide-Semiconductor, FET: field-effect transistor) is used for each transistor starting with the transistors Q1, Q2, Q3 and up to the transistor Q12 described later.

In the logarithmic conversion unit 16b, transistor Q1 and transistor Q3 are set as N-type transistors, and transistor Q2 is set as a P-type transistor.

The source of transistor Q1 is connected to the light-receiving element 16a (connected to the cathode of the photodiode), and the drain is connected to the power supply terminal. In addition, "VDD" in the figure means the power supply voltage.

Transistor Q2 and transistor Q3 are connected in series between the power terminal and the ground terminal. Furthermore, the connection point between transistor Q2 and transistor Q3 is connected to the gate of transistor Q1 and the input terminal of buffer 16c (the gate of transistor Q5 described later). Furthermore, a specific bias voltage Vbias is applied to the gate of transistor Q2.

The drains of transistors Q1 and Q3 are connected to the power supply side, and this circuit is called a source follower. The photocurrent from the photosensitive element 16a is converted into a logarithmic voltage signal by the two source followers connected in a ring. In addition, transistor Q2 supplies a certain current to transistor Q3.

The buffer 16c includes transistors Q4 and Q5, which are P-type transistors, and the transistors Q4 and Q5 are connected in series between a power terminal and a ground terminal.

The connection point between transistor Q4 and transistor Q5 is set as the output terminal of buffer 16c, and the corrected voltage signal is output to subtractor 17a of event detection unit 17 through the output terminal.

In the event detection unit 17, the subtractor 17a reduces the level of the voltage signal from the buffer 16c according to the reset signal RST output by the reset unit 19. The subtractor 17a outputs the reduced voltage signal to the quantizer 17b.

The quantizer 17b quantizes the voltage signal from the subtractor 17a into a digital signal and outputs it to the holding unit 18 as a detection signal (equivalent to the aforementioned detection signal Sd). The quantizer 17b in this example outputs the voltage signal from the subtractor 17a: the detection signal Sd indicating the detection result of the conduction event (the output voltage Vop in the figure), and the detection signal Sd indicating the detection result of the disconnection event (the output voltage Vom in the figure).

Subtractor 17a includes capacitor C1 and capacitor C2, transistor Q7 and transistor Q8, and reset switch SWr. Transistor Q7 is a P-type transistor, and transistor Q8 is an N-type transistor.

Transistor Q7 and transistor Q8 are connected in series between a power terminal and a ground terminal to form an inverter. Specifically, the source of transistor Q7 is connected to the power terminal, the drain is connected to the drain of transistor Q8, and the source of transistor Q8 is connected to the ground terminal. In addition, a voltage Vbdif is applied to the gate of transistor Q8.

One end of capacitor C1 is connected to the output terminal of buffer 16c, and the other end is connected to the gate of transistor Q7 (inverter input terminal). One end of capacitor C2 is connected to the other end of capacitor C1, and the other end is connected to the connection point between transistor Q7 and transistor Q8.

One end of the reset switch SWr is connected to the connection point between capacitor C1 and capacitor C2, and the other end is connected to the connection point between transistor Q7 and transistor Q8 and the connection point of capacitor C2, and is connected in parallel with capacitor C2. The reset switch SWr is a switch that turns on/off according to the reset signal RST from the reset unit 19.

The inverter formed by transistor Q7 and transistor Q8 inverts the voltage signal input through capacitor C1 and outputs it to quantizer 17b.

Here, in the subtractor 17a, the potential generated at the buffer 16c side of the capacitor C1 at a certain point in time is set as the potential Vinit. And at this time, the reset switch SWr is turned on. When the reset switch SWr is turned on, the side opposite to the buffer 16c of the capacitor C1 becomes a virtual ground terminal. For convenience, the potential of the virtual ground terminal is set to zero. At this time, if the capacitance of the capacitor C1 is set to Cp1, the charge CHinit accumulated in the capacitor C1 is expressed by the following [Formula 1].

CHinit=Cp1×Vinit‧‧‧[Formula 1]

Furthermore, when the reset switch SWr is turned on, the two ends of the capacitor C2 are short-circuited, so the accumulated charge becomes zero.

Next, the reset switch SWr is turned off. If the amount of light received changes, the potential on the buffer 16c side of the capacitor C1 changes from the above-mentioned Vinit. If the changed potential is set as Vafter, the charge CHafter accumulated in the capacitor C1 is expressed by the following [Formula 2].

CHafter=Cp1×Vafter‧‧‧[Formula 2]

On the other hand, if the capacitance of capacitor C2 is set to Cp2 and the output voltage of subtractor 17a is set to Vout, the charge CH2 accumulated in capacitor C2 is expressed by the following [Formula 3].

CH2=-Cp2×Vout‧‧‧[Formula 3]

At this time, since the total charge of capacitors C1 and C2 does not change, the following [Equation 4] holds.

CHinit=CHafter+CH2‧‧‧[Formula 4]

If [Formula 1] to [Formula 3] are substituted into [Formula 4] and transformed, the following [Formula 5] is obtained.

Vout=-(Cp1/Cp2)×(Vafter-Vinit)‧‧‧[Formula 5]

[Equation 5] represents the subtraction operation of the voltage signal, and the gain of the subtraction result becomes Cp1/Cp2.

According to [Formula 5], the subtractor 17a outputs a signal showing the difference between the level (Vinit) of the past light-receiving signal and the level (Vafter) of the current light-receiving signal.

Here, the potential Vinit is equivalent to the reference level Lref mentioned above. According to the above description, the potential Vinit, i.e., the reference level Lref, is reset to the level of the current light-receiving signal, in other words, the level of the light-receiving signal at the time when the reset switch SWr is turned on, by turning on the reset switch SWr.

The quantizer 17b includes transistor Q9, transistor Q10, transistor Q11, and transistor Q12, and constitutes a 1.5-bit quantizer.

Transistors Q9 and Q11 are set as P-type transistors, and transistors Q10 and Q12 are set as N-type transistors.

As shown in the figure, transistors Q9 and Q10, and transistors Q11 and Q12 are connected in series between the power terminal and the ground terminal, respectively, and the output voltage (Vout) of the subtractor 17a is input to each gate of transistors Q9 and Q11. In addition, a voltage Vhigh is applied to the gate of transistor Q10, and a voltage Vlow is applied to the gate of transistor Q12.

An output voltage Vop indicating a detection result of a conduction event is obtained at the connection point of transistor Q9 and transistor Q10, and an output voltage Vom indicating a detection result of a disconnection event is obtained at the connection point of transistor Q11 and transistor Q12.

Specifically, on the transistor Q9 and Q10 side, when the output voltage (Vafter-Vinit) of the subtractor 17a is above the positive side critical value corresponding to the voltage Vhigh, an H-level output voltage Vop is obtained at the connection point of the transistor Q9 and the transistor Q10, and when the output voltage of the subtractor 17a does not reach the positive side critical value, an L-level output voltage Vop is obtained. That is, at the connection point of the transistor Q9 and the transistor Q10, a signal indicating whether the amount of light received changes in the increasing direction above a specific critical value, that is, a detection signal Sd indicating the detection result of the conduction event is obtained.

Furthermore, on the transistor Q11 and Q12 side, when the output voltage level of the subtractor 17a is below the critical value of the negative side corresponding to the voltage Vlow, an H-level output voltage Vom is obtained at the connection point between the transistor Q11 and the transistor Q12, and when the output voltage level of the subtractor 17a is greater than the critical value of the negative side, an L-level output voltage Vom is obtained. In this way, at the connection point between the transistor Q11 and the transistor Q12, a signal indicating whether the amount of light received changes above a specific critical value in the decreasing direction, that is, a detection signal Sd indicating the detection result of the disconnection event is obtained.

The holding unit 18 inputs the output voltage Vop and the output voltage Vom of the quantizer 17b and holds them. The holding unit 18 outputs the held output voltage Vop, i.e., the detection signal Sd indicating the detection result of the conduction event, as the event signal Evnp, and outputs the held output voltage Vom, i.e., the detection signal Sd indicating the detection result of the disconnection event, as the event signal Evnm.

When a conduction event occurs, the holding unit 18 outputs a detection signal indicating that a conduction event has been detected, and when a disconnection event occurs, the holding unit 18 outputs a detection signal indicating that a disconnection event has been detected.

Figure 5 is an explanatory diagram of the structure of the event signal reading.

As shown in the figure, the readout unit 13 has a transmission unit 13a for each pixel 15, and the event signal (Evnp, Evnm) is input to each transmission unit 13a from the holding unit 18 of each corresponding pixel 15. Each transmission unit 13a transmits the input event signal to the signal processing circuit 14 based on the instruction from the arbitrator 12.

Furthermore, the detection signals Sd (output voltage Vop and output voltage Vom) obtained by the event detection unit 17 of each pixel 15 are respectively supplied to the arbitrator 12. Here, the detection signals Sd function as signals for requesting the arbitrator 12 to read the event signal.

The arbitrator 12 inputs the detection signals Sd from the event detection units 17, and controls the reading action of the reading unit 13 based on the result of modulating the request based on the detection signals Sd. Specifically, the transmission unit 13a of the reading unit 13 is instructed to transmit the event signal to the signal processing circuit 14 in a manner of reading the event signal of the pixel 15 generating the event.

Furthermore, the holding unit 18 resets the detection signal Sd (output voltage Vop and output voltage Vom) held in response to the transmission instruction (read instruction) from the arbitrator 12 to the transmission unit 13a. Specifically, the holding unit 18 resets the detection signal Sd held in response to the transmission instruction from the arbitrator 12 to the output destination of its own event signal, that is, the transmission unit 13a, to the L level.

[1-2. Reset section of the first implementation form]

Here, it can be understood from the above description that the detection of an event is obtained by taking the level of the past light-receiving signal as the reference level Lref and obtaining the difference between the reference level Lref and the current light-receiving signal level. Moreover, when an event is detected, the reference level Lref is reset to the current light-receiving signal level in a manner that allows the detection of a new event.

For example, the reset of the reference level Lref is performed in accordance with the reading of the event signal.

However, since the arbitrator 12 needs a certain amount of time from receiving the notification of event detection to instructing the signal reading of the pixel 15, if the reference level Lref is reset in accordance with the reading, a delay occurs when resetting the reference level Lref. In other words, the reference level cannot be quickly updated to the light-receiving signal level at the time of event detection, and the accuracy of the reference level Lref is reduced, resulting in the possibility of reduced event detection accuracy such as missed event detection.

Therefore, in this embodiment, after the event detection unit 17 detects the event and before the reading unit reads the event signal, the reference level Lref is reset.

FIG6 is a diagram for explaining an example of the configuration of the reset unit 19 of the first embodiment.

In addition, in FIG6, the internal configuration examples of the light receiving unit 16, the event detection unit 17, and the holding unit 18 are simultaneously shown together with the internal configuration example of the reset unit 19.

As shown in the figure, the holding unit 18 has: a first holding circuit 18p, a second holding circuit 18m, and an inverter 18i. The output voltage Vop is input to the first holding circuit 18p, and the output voltage Vom is input to the second holding circuit 18m via the inverter 18i. The first holding circuit 18p and the second holding circuit 18m are configured as latch circuits, for example. In addition, the first holding circuit 18p and the second holding circuit 18m can be latch circuits or digital memory circuits such as flip-flops, or sampling circuits such as switches, plastics, capacitors, etc.

The reset section 19 has an OR gate circuit 19a and a delay circuit 19b. The OR gate circuit 19a inputs the output voltage Vop and the output voltage Vom via the inverter 18i, and the delay circuit 19b delays the output signal of the OR gate circuit 19a and outputs it as a reset signal RST to the reset switch SWr. In addition, a structure such as connecting a plurality of inverters in series can be used as the delay circuit 19b.

FIG. 7 is a timing diagram for explaining the operation of the circuit shown in FIG. 6, showing: the waveform of the potential change of the node A shown in FIG. 6, and the waveform of the output signal of the first holding circuit 18p, i.e., the event signal Evnp.

The time point t1 in the figure indicates the time point when the event detection unit 17 detects an event (a conduction event at this time). Corresponding to the event detection, the potential of the node A gradually rises. That is, the output voltage Vop gradually increases to the H level.

There is a time lag before the potential increase of the node A is reflected in the output signal Evnp. At the time point t2 in the figure, the output signal Evnp is increased to the H level. The time point t2 can be said to be the timing of the end of the holding of the event signal indicating the detection result of the event generated in the first holding circuit 18p.

Then, at time t3, which is the time when the delay time caused by the delay circuit 19b has passed since time t1, the reset signal RST is output by the reset unit 19 to reset the reference level Lref.

Here, the delay time caused by the delay circuit 19b is set to be longer than the time from time point t1 to time point t2. That is, considering the above-mentioned time lag, it is set in a way that can ensure sufficient waiting time until the level of the detection signal held in the holding unit 18 becomes a certain level or more as the H level.

At time t4, when the required time has passed since time t3, the event signal is read from the holding unit 18 to the signal processing circuit 14. As mentioned above, a corresponding amount of time is required from event detection to the reading of the event signal.

According to the above description, it can be understood that the holding unit 18 holds the detection signal Sd until the reading is performed. That is, when the detection signal Sd is raised to the H level corresponding to the event detection, the level is maintained until the reading is performed.

By holding the detection signal Sd in accordance with event detection by the holding unit 18 as described above, even if the change in the amount of received light is instantaneous, the signal indicating that the event has been detected can be continuously held until reading is performed.

As described above, in the image sensor 1 of the embodiment, after the event detection unit 17 detects the event and before the readout unit 13 reads the event signal, the reference level Lref is reset.

In this way, the reference level used for event detection can be quickly reset (updated) to the current light-receiving signal level in accordance with event detection.

Therefore, the accuracy of the reference level Lref used for event detection can be improved, and the accuracy of event detection can be improved.

FIG8 is a diagram for explaining the meaning of quickly resetting the reference level Lref in response to event detection.

In the upper and lower parts of the figure, the waveform represented by the solid line shows the change of the amount of light received, and the levels Lv1, Lv2, and Lv2' respectively represent the reference level Lref. In addition, the two arrows in the figure schematically show the threshold value TH used for event detection.

The upper part of the figure illustrates the situation where the reference level Lref is quickly reset in response to event detection. At this time, the first event is detected at time T1, corresponding to the difference between the reference level Lref displayed as "Lv1" and the current light receiving signal level being greater than the critical value TH. In addition, the second event is detected at time T2, corresponding to the difference between the reference level Lref of "Lv2" reset after the first event detection and the current light receiving signal level being greater than the critical value TH.

On the other hand, the following example shows the case where the reference level Lref is reset after a certain delay D after the first event detection (time point T1). At this time, the light-receiving signal level rises during the delay D, and the reference level Lref is reset to "Lv2'" which is greater than "Lv2" at the time point of the first event detection. Therefore, at time point T2, the difference between "Lv2'" and the current light-receiving signal level will not exceed the critical value TH, resulting in missed event detection.

By quickly updating the reference level Lref as described above, it is possible to prevent missed event detection, thereby improving event detection accuracy.

[1-3. Example of changes in detection signal retention]

Here, in this embodiment, a configuration is adopted in which the reference level Lref is reset before the signal is read from the holding unit 18. In this configuration, a detection signal for a new event may be generated in the event detection unit 17 during the period from the reset of the reference level Lref to the signal reading. That is, when a new event is generated after event detection and before the signal for the event is read, the detection signal for the new event is input to the holding unit 18 that holds the detection signal for the immediately preceding event, and the detection signal for the new event is assimilated with the detection signal for the immediately preceding event in the holding unit 18, resulting in output omission of the event signal.

Therefore, an output control unit 20 as shown in FIG. 9 may be provided to prevent the detection signal of a new event generated before reading from being assimilated with the detection signal of the immediately preceding event.

In addition, in the following description, the same symbols are given to the parts that are the same as the parts that have been explained, and the explanation is omitted.

As shown in the figure, the output control unit 20 has an OR gate circuit 20a, a switch SWp and a switch SWm.

The input of the OR gate circuit 20a is: the output signal of the first holding circuit 18p, i.e. the event signal Evnp, and the output signal of the second holding circuit 18m, i.e. the event signal Evnm.

The switch SWp inserts the output voltage Vop from the event detection unit 17 to the supply line of the first holding circuit 18p, and the switch SWm inserts the output voltage Vom from the event detection unit 17 to the supply line of the inverter 18i, and the on/off control is performed by the output signal of each OR gate circuit 20a. Here, when the output signal of the OR gate circuit 20a is at the L level, the switch SWp and the switch SWm are set to the off state.

According to this configuration, when the level of either event signal Evnp or event signal Evnm becomes higher than one of the H levels, switches SWp and SWm are turned off, thereby preventing the detection signal of a new event generated before reading from being assimilated with the detection signal of the immediately preceding event.

Furthermore, according to the above configuration, the detection signals held in the first holding circuit 18p and the second holding circuit 18m are read out in accordance with event detection, and when the levels of the detection signals of the first holding circuit 18p and the second holding circuit 18m are reduced to below a certain level (i.e., when reset to the L level), the switches SWp and SWm are turned on. Thus, the first holding circuit 18p and the second holding circuit 18m can hold the detection signals of new events.

In this way, by providing the output control unit 20 shown in FIG. 9 , the event signal for a new event generated before reading can be output as a signal different from the event signal for an event generated immediately before, thereby preventing the output of the event signal from being missed.

In addition, regarding the output control unit 20 shown in FIG. 9, in the configuration in which event detection is performed only for conduction events (or only for disconnection events), the OR gate circuit 20a is not required. That is, at this time, the components of the output control unit 20 may at least only use the switch SWp or the switch SWm.

<2. Second implementation form>

FIG. 10 is a diagram for explaining an example of the internal structure of a pixel 15 provided in the image sensor 1 as the second embodiment.

The difference from the first embodiment is that a reset portion 19A is provided to replace the reset portion 19.

The reset section 19A has an OR gate circuit 19a, which inputs: the output signal of the first holding circuit 18p, i.e., the event signal Evnp, and the output signal of the second holding circuit 18m, i.e., the event signal Evnm. The output signal of the OR gate circuit 19a is output as a reset signal RST to the reset switch SWr of the event detection section 17.

FIG11 is a timing diagram for explaining the operation of the circuit shown in FIG10, showing: the waveform of the potential change of node A shown in FIG10, and the waveform of the event signal Evnp.

The time point t5 in the figure indicates the time point when the event detection unit 17 detects the event (conduction event), and the time point t6 indicates the time point when the event signal Evnp is raised to the H level. At the time point t6, the reset unit 19A outputs the reset signal RST to reset the reference level Lref.

Time point t7 indicates the time point when the event signal Evnp is read out corresponding to the event detection.

As described above, the reset unit 19A of the second embodiment resets the reference level Lref based on the output signal of the holding unit 18.

Thus, the reference level Lref can be reset in response to the level of the detection signal held in the holding unit 18 being above a certain level (i.e., H level), and the event signal can be output appropriately. Moreover, the delay circuit 19b for delaying the reset timing until the level of the detection signal held in the holding unit 18 is above a certain level is not necessary.

In addition, in the second embodiment, the output control unit 20 shown in FIG. 9 may also be provided.

<3. The third implementation form>

In the description so far, an example is given in which the event detection unit 17 detects the conduction event and the disconnection event simultaneously and in parallel, but the conduction event and the disconnection event can also be detected in time division. Regarding the third implementation form of this time division detection, the first and second examples are described below.

FIG. 12 is a diagram for explaining an example of the internal structure of a pixel 15 provided in the image sensor 1 as the first example of the third embodiment.

The image sensor 1 as the first example shown in FIG. 12 is different from the first embodiment ( FIG. 4 ) in that an event detection unit 17' is provided instead of the event detection unit 17, a holding unit 18' is provided instead of the holding unit 18, a reset unit 19' is provided instead of the reset unit 19, and a switch SWp, a switch SWm, and a controller 50 are provided.

A quantizer 17b' is provided in the event detection unit 17' instead of the quantizer 17b. The quantizer 17b' is different from the quantizer 17b in that the transistor Q11 and the transistor Q12 are omitted. At this time, the output signal of the quantizer 17b' becomes only one system of the signal obtained at the connection point of the transistor Q9 and the transistor Q10.

The holding part 18' only has a single holding circuit 18a as a holding circuit and an inverter 18i.

As shown in the figure, the output line of the self-quantizer 17b' is divided into: a line through the switch SWp and a line through the switch SWm. The line through the switch SWp is connected to the holding circuit 18a, and the line through the switch SWm is connected to the holding circuit 18a after merging with the line through the switch SWp through the inverter 18i.

The reset part 19' is different from the reset part 19 in that the OR gate circuit 19a is omitted, and the signal obtained at the junction of the line passing through the switch SWp and the line passing through the switch SWm is input to the delay circuit 19b of the reset part 19'.

The controller 50 performs: control of the gate voltage of the transistor Q10 of the quantizer 17b', and on/off control of the switch SWp and the switch SWm.

Specifically, the controller 50 controls the gate voltage of the transistor Q10 to switch between the aforementioned voltage Vhigh and the voltage Vlow. In addition, the controller 50 controls the switch SWp and the switch SWm to be turned on/off alternately in synchronization with the switching of the voltage Vhigh/Vlow. That is, the following control is performed, namely: during the selection period of the voltage Vhigh, the switch SWp is turned on and the switch SWm is turned off, and during the selection period of the voltage Vlow, the switch SWp is turned off and the switch SWm is turned on.

According to the description of the first embodiment, during the selection period of the voltage Vhigh, the detection signal Sd indicating the detection result of the conduction event is obtained in the quantizer 17b'. On the other hand, during the selection period of the voltage Vlow, the detection signal Sd indicating the detection result of the disconnection event is obtained in the quantizer 17b'.

Therefore, the controller 50 performs on/off control of the switches SWp and SWm in synchronization with the switching of the voltage Vhigh/Vlow as described above, and during the selection period of the voltage Vhigh, the detection signal Sd indicating the detection result of the on event is input to the holding circuit 18a via the switch SWp. On the other hand, during the selection period of the voltage Vlow, the detection signal Sd indicating the detection result of the off event is input to the holding circuit 18a via the switch SWm and the inverter 18i.

Thus, in the configuration shown in FIG12, the detection of conduction events and disconnection events is performed in time.

Moreover, in the structure shown in FIG12, corresponding to the time-division detection, only a single holding circuit 18a is provided in the holding section 18', and the holding circuit 18a holds the detection signal Sd of the conduction event and the detection signal Sd of the disconnection event in a time-division manner.

When a conduction event occurs during the selection period of the voltage Vhigh, the detection signal Sd held in the holding circuit 18a is read out as the event signal Evnp of the conduction event by the arbitrator 12. Similarly, when a disconnection event occurs during the selection period of the voltage Vlow, the detection signal Sd held in the holding circuit 18a is read out as the event signal Evnm of the disconnection event by the arbitrator 12.

Here, the reset unit 19' inputs the signal of the junction point of the line through the switch SWp and the line through the switch SWm, but according to this configuration, in each case where a conduction event occurs during the selection period of the voltage Vhigh and a disconnection event occurs during the selection period of the voltage Vlow, the reference level Lref can be reset before the event signal Evp is read.

In addition, the controller 50 can be set for each pixel 15, or a common controller 50 can be set for each plurality of pixels 15.

FIG. 13 is a diagram for explaining an example of the internal structure of a pixel 15 provided in the image sensor 1 as the second example of the third embodiment.

The difference from the first example shown in FIG. 12 is that the holding unit 18' is provided to replace the holding unit 18, and the detection signal Sd of the conduction event and the detection signal Sd of the disconnection event obtained by time division are respectively held in the first holding circuit 18p and the second holding circuit 18m.

Furthermore, corresponding to the above-mentioned provision of the holding unit 18 instead of the holding unit 18', the resetting unit 19 is provided instead of the resetting unit 19'. Thus, in each case where a conduction event occurs during the selection period of the voltage Vhigh and a disconnection event occurs during the selection period of the voltage Vlow, the reference level Lref can be reset before the event signal Evp is read.

<4. The fourth implementation form>

[4-1. Overview of scanning method]

In the description so far, an example of using an arbitrator method (asynchronous reading method) is described in which the arbitrator 12 reads the event signal Evn of the pixel 15 in response to the detection of the event. However, the reading of the event signal Evn can also utilize a scanning method in which each row of the pixel array unit 11 is synchronously read.

FIG. 14 is a diagram for explaining the structure of the image sensor 1 when the event signal Evn is read out by scanning. In addition, FIG. 14 only extracts the structure of the part of the event signal Evn that is read out and displays it.

As described above, a plurality of pixels 15 are arranged in a two-dimensional grid pattern in the pixel array section 11. At this time, the structure inside the pixel 15 is the same as the structure described in the first embodiment such as FIG. 6 or FIG. 9, FIG. 10, etc.

The row driver 55 selects the row of the pixel array unit 11. Specifically, the row driver 55 selects the plurality of rows of the pixel array unit 11 row by row in a preset cycle. If the row driver 55 selects the row, the values held in the holding unit 18 are output to the output circuit 56 by each pixel 15 of the selected row. Specifically, since the holding unit 18 of the pixel 15 in this case is provided with a first holding circuit 18p and a second holding circuit 18m, the holding value of the first holding circuit 18p and the holding value of the second holding circuit 18m are output from each pixel 15 of the selected row. The description of "Evnp" and "Evnm" of each pixel 15 in the figure indicates it.

Based on the output values from each pixel 15 of the selected row, the output circuit 56 determines whether each pixel 15 is an event detection pixel, that is, whether it is a pixel that detects at least one of an on event and a off event, and generates and outputs information about the pixel address for the event detection pixel. The pixel address here is in the form of (row, column) coordinates, for example. As an example, the output circuit 56 obtains the row coordinates from the row selected by the row driver 55, and generates the pixel address information in correspondence with the row information of the event detection pixel in the row.

In addition, the output circuit 56 can output the information of the event detection pixel indicating whether it is a conduction event or a disconnection event together with the information of the pixel address of the event detection pixel.

Using the above-mentioned structure, the event signal Evn of the pixel 15 that detects the event is read out in row order. In other words, the reading is performed synchronously in row units.

[4-2. Counting method]

Here, when the scanning method as described above is adopted, the time delay from the detection of the event to the reading of the event signal Evn becomes longer compared to the case of adopting the arbitrator method.

If the time delay until reading becomes longer as described above, even when the amount of received light changes greatly, as shown in Figure 15, only one event is detected. That is, for the part where the amount of received light changes greatly as described above, it should be treated as a case where multiple events are detected, but only one event is detected, and there is a risk of missing the detection of an event.

Therefore, in this embodiment, a reset unit 19 is provided to reset the reference level Lref in read standby mode, that is, a configuration in which a plurality of events can be detected in read standby mode, and a configuration in which the number of occurrences of events is counted in read standby mode and the count value is read as an event signal.

FIG. 16 is a diagram for explaining an example of the internal structure of pixel 15 when a counting method is used to count the number of times an event occurs.

The structure of the pixel 15 at this time is different from that in FIG. 6 in that a holding portion 25 is provided instead of the holding portion 18.

The holding unit 25 includes a first counter 25p, a second counter 25m, and an inverter 18i. As shown in the figure, the output voltage Vop from the event detection unit 17 is input to the first counter 25p. The output voltage Vom from the event detection unit 17 is input to the second counter 25m via the inverter 18i.

Figure 17 is a diagram illustrating the operation of the counter.

In addition, Figure 17 is used as an explanation diagram of the action on the first counter 25p side, but since the action on the second counter 25m side is the same as that on the first counter 25p side, repeated explanation is avoided.

When a large change occurs in the amount of received light as shown in the example of FIG. 15 above, the potential of node A (see FIG. 16) changes from the H level to the L level repeatedly as shown in the figure each time the reset unit 19 resets the reference level Lref.

The counting reference value Lc as shown in the figure is set in the first counter 25p. The first counter 25p increases the counting value every time the potential of the node A becomes above the counting reference value Lc. In this way, the number of occurrences of the event in standby can be counted and read.

Figure 18 shows an illustration of the function of the counting method.

Even when a large change in the amount of received light occurs as shown in FIG15 , the number of times an event occurs can be counted during the readout standby period by self-resetting the reference value level Lref by the reset unit 19 and performing the counting operation of the first counter 25p (or the second counter 25m).

This can prevent the omission of event detection. In addition, it is possible to estimate not only whether there is a change in the amount of light received, but also the amount of change in the amount of light received.

Here, in the configuration shown in FIG. 16, the output voltage Vop and the output voltage Vom function as "a detection signal indicating that an event has been detected" when the corresponding events of the conduction event and the disconnection event occur. Therefore, the first counter 25p and the second counter 25m count the number of occurrences of the events based on the output voltages Vop and Vom, respectively, and maintaining the count value is equivalent to maintaining the "detection signal" obtained by the number of occurrences of the event, and thus corresponds to maintaining a state of the "detection signal".

Furthermore, if holding the count value corresponds to holding the "detection signal" as described above, it can be said that reading the count value held in the holding unit 25 corresponds to reading the "detection signal" held in the holding unit 25 as an event signal. Based on this aspect, the count values read from the first counter 25p and the second counter 25m are described as event signals Evnp' and Evnm' in FIG. 16 .

In addition, FIG. 16 illustrates a configuration for outputting a reset signal RST based on the output voltage Vo, but it may also be replaced by a configuration for outputting a reset signal RST based on the count update timing of the first counter 25p and the second counter 25m.

Here, when the counting method is adopted, the values of the first counter 25p and the second counter 25m are input to the output circuit 56 shown in FIG. 14 using each pixel 15 of the column selected by the column driver 55. At this time, the output circuit 56 outputs the pixel address of the pixel 15 with a count value of 1 or more as the pixel address of the event detection pixel, and outputs information of the event detection pixel indicating whether it is an on event, a disconnection event, or any other event, and the count value for each event detection pixel.

[4-3. Examples of changes related to counting methods]

Even when the above counting method is used, event detection using the time division method described in the third embodiment can also be applied.

Figures 19 and 20 respectively show examples of the internal structure of pixel 15 when the time-division method is applied to the counting method.

In the example of FIG. 19, a holding unit 25 is provided at the rear stage of the configuration for time division detection (event detection unit 17' and switches SWp, SWn) shown in FIG. 12 or FIG. 13 above, and the counting of conduction events and the counting of disconnection events are performed by the first counter 25p and the second counter 25m respectively.

Here, in the case where the count of the conduction event and disconnection event is performed using a separate counter as shown in FIG19, the time-division count of the conduction event and disconnection event and the reading of the count value (the reading of the event signal Evnp’ and Evnm’) are considered as follows.

That is, when the period from when the row driver 55 selects a row until the row is selected again is set as the "reading waiting period Pw", the first half (or second half) of the reading waiting period Pw is set as the counting period of the conduction event (i.e. the conduction period of the switch SWp), and the second half (or first half) is set as the counting period of the disconnection event (the conduction period of the switch SWm), and the count values of the first counter 25p and the second counter 25m are read simultaneously when the row driver 55 selects the row.

Alternatively, the even-numbered readout waiting period Pw may be set as the counting period for the conduction event, and the odd-numbered readout waiting period Pw may be set as the counting period for the disconnection event, and the counting period for the conduction event and the counting period for the disconnection event may be switched alternately for each column selection. At this time, the reading of the count value of the first counter 25p and the reading of the count value of the second counter 25m are performed alternately for each column selection of the driver 55.

The example of FIG. 20 is a case where a holding unit 25' is provided to replace the holding unit 25 of the example of FIG. 19. As shown in the figure, the holding unit 25' has a single counter 25a and an inverter 18i, and a signal obtained at the junction of the line from the quantizer 17b' via the switch SWp and the line from the quantizer 17b' via the switch SWm and the inverter 18i is input to the counter 25a. That is, a structure is formed in which the common counter 25a is used to count each of the conduction event and the disconnection event. Specifically, the counter 25a can count the number of conduction events and the number of disconnection events in time by switching the switches SWp and SWm alternately by the controller.

In the case of using a common counter 25a for counting conduction events and disconnection events as shown in FIG20, it is impossible to switch the count of conduction events and the count of disconnection events in the middle of the read waiting period Pw. Therefore, in the case of using a common counter 25a as shown in FIG20, the count period of conduction events and the count period of disconnection events are alternately switched by setting the even-numbered read waiting period Pw as the count period of conduction events and the odd-numbered read waiting period Pw as the count period of disconnection events, etc.

In addition, the above content gives an example of the case where the counting method adopts the scanning method, but it can also be applied to the case where the counting method adopts the arbitrator method. In the case of the arbitrator method, the number of times the event occurs from the occurrence of the event until the reading is performed according to the instruction of the arbitrator 12.

Furthermore, when the counting method is applied to the arbitrator method, the time division method can also be applied. When the time division method is applied, the detection and processing of the conduction event and the disconnection event are performed by time division. If a conduction event is detected, the count value of the conduction event is read according to the instruction of the arbitrator 12, and if a disconnection event is detected, the count value of the disconnection event is read according to the instruction of the arbitrator 12. At this time, the counter can be common or can be set individually for each conduction event and disconnection event.

<5. Examples of changes in pixel composition>

Here, in the description so far, an example is given in which only one light receiving unit 16 and event detection unit 17 (or event detection unit 17') are provided for one pixel 15, but a configuration in which a plurality of light receiving units 16 and event detection units 17 are provided for each pixel 15 may also be adopted. As an example of providing a plurality of light receiving units 16 and event detection units 17 for each pixel 15, for example, a light receiving unit 16 that receives light of R (red), G (green), and B (blue) in a Bayer arrangement may be given.

At this time, the unit for event detection is not the unit of the light receiving unit 16, but the unit of the pixel. That is, when an event is detected in at least one of the multiple event detection units 17 in the pixel 15, it is regarded as an event detected in the pixel 15 and processed.

Moreover, as the reset unit 19 at this time, it is considered that it is not set for each event detection unit 17, but only one is set for one pixel 15. That is, as in the first variation described below, a single reset unit 19 is set for one pixel, and when any event detection unit 17 in the pixel 15 detects an event, the reference level Lref of all event detection units 17 in the pixel 15 is reset accordingly.

FIG. 21 shows an example of the configuration of a pixel 15A of the image sensor 1 as the first variation.

As shown in the figure, the pixel 15A has four light receiving units 30, namely, a light receiving unit 30-1 for receiving R light, a light receiving unit 30-2 and a light receiving unit 30-3 for receiving G light, and a light receiving unit 30-4 for receiving B light, and has a reset unit 19 and a holding unit 18A.

The light receiving unit 30-1 has a light receiving unit 16-1 having a color filter for R light and receiving R light, and an event detection unit 17. The light receiving unit 30-2 and the light receiving unit 30-3 have a light receiving unit 16-2 and a light receiving unit 16-3 having a color filter for G light and receiving G light, and have an event detection unit 17. The light receiving unit 30-4 has a light receiving unit 16-4 having a color filter for B light and receiving B light, and an event detection unit 17.

In pixel 15A, light receiving parts 16-1, 16-2, 16-3, and 16-4 are arranged in a Bayer pattern.

FIG. 22 is a diagram for explaining a configuration example of the holding portion 18A provided in the image sensor 1 as the first variation. FIG. 22 shows an internal configuration example of the reset portion 19, a light receiving portion 16, an event detection portion 17, and an internal configuration example of the holding portion 18A together with an internal configuration example of the holding portion 18A.

As shown in the figure, the holding section 18A includes an OR gate circuit 18a, an OR gate circuit 18b, and four inverters 18i together with the first holding circuit 18p and the second holding circuit 18m.

As shown in the figure, the output voltage Vop is input from the event detection unit 17 of each light receiving unit 30 (30-1 to 30-4) to the OR gate circuit 18a, and the output voltage Vom from the event detection unit 17 of each light receiving unit 30 is input to the OR gate circuit 18b through a corresponding inverter 18i.

The output signal of the OR gate circuit 18a is supplied as the input signal of the first holding circuit 18p and the OR gate circuit 19a of the reset section 19, and the output signal of the OR gate circuit 18b is supplied as the input signal of the second holding circuit 18m and the OR gate circuit 19a.

At this time, the reset signal RST of the reset unit 19 is output to the reset switch SWr of the event detection unit 17 of each light receiving unit 30.

According to this configuration, in the image sensor 1 as the first variation, in response to the detection of an event in at least one of the plurality of event detection units 17 in the pixel 15A, the reference level Lref of all event detection units 17 in the pixel 15A is reset using the reset unit 19.

Furthermore, according to the configuration of the holding unit 18A of the first variation, the event signal (in this example, either of the event signals Evnp and Evnm) output in response to the detected event becomes a single system for each pixel 15A.

In the first variation example described above, when multiple light receiving units 16 and event detection units 17 are provided for each pixel, the reference level Lref of all event detection units 17 in the pixel is reset under the condition that any event detection unit 17 in the pixel detects an event. In addition, in the first variation example, an example is given in which the event signals Evnp and Evnm output for each pixel are respectively set as a single system.

However, as in the second variation described below, the following configuration may be adopted, namely, when a plurality of event detection units 17 in a pixel detect an event, the reference level Lref of all event detection units 17 in the pixel is reset, and the output system of the event signal Evnp, Evnm of each pixel is not set to a single system, but the events Evnp, Evnm of all systems (light receiving channels) that detect events can be output.

FIG. 23 shows a configuration example of a pixel 15B provided in the image sensor 1 as the second variation. The difference from the pixel 15A of the first variation is that light receiving units 30'-1, 30'-2, 30'-3, 30'-4 are provided instead of light receiving units 30-1, 30-2, 30-3, 30-4, and a reset unit 19B is provided instead of reset unit 19.

The light receiving units 30'-1, 30'-2, 30'-3, and 30'-4 are different from the light receiving units 30-1, 30-2, 30-3, and 30-4 in that they are each additionally provided with a holding portion 18.

FIG. 24 is a diagram for explaining a configuration example of the reset portion 19B provided in the image sensor 1 as the second variation. FIG. 24 shows an internal configuration example of the holding portion 18 of each of the light receiving units 30'-1, 30'-2, 30'-3, and 30'-4 together with an internal configuration example of the reset portion 19B.

As shown in the figure, the reset unit 19B has a determination unit 19c and a delay circuit 19b. At this time, the delay circuit 19b delays the output signal of the determination unit 19c and outputs it as a reset signal RST to the reset switch SWr of the event detection unit 17 of each light receiving unit 30' (30'-1 to 30'-4).

The output voltage Vop of the event detection unit 17 of each light receiving unit 30' is input to the determination unit 19c, and the output voltage Vom of the event detection unit 17 is input through the inverter 18i of each corresponding holding unit 18.

The determination unit 19c increases the output signal to the H level in response to any N (N is a natural number greater than 2 and less than 4) of the output voltages Vop and Vom of each of the four systems input as described above becoming the H level.

Thus, under the condition that multiple events are detected in the event detection unit 17 in the pixel 15B, the reference level Lref of all event detection units 17 in the pixel 15B is reset.

Furthermore, according to the structure shown in FIG24, the system events Evnp and Evnm of all detected events can be output for each pixel 15B.

In addition, the same is true for the first and second variations mentioned above, and can be applied as the configuration of the second implementation form. Specifically, for the first variation, the configuration of inputting event signals Evnp and Evnm to the OR gate circuit 19a can be adopted. Moreover, for the second variation, the configuration of inputting event signals Evnp and Evnm output by each light receiving unit 30 (30-1' to 30-4') to the determination unit 19c can be adopted. In addition, at this time, the delay circuit 19b is not required in the reset unit 19, 19B.

Furthermore, the same structure as the output control unit 20 shown in FIG. 9 can be applied to the first and second variations to prevent the detection omission of events.

Furthermore, the configurations of the first and second variations can also be similarly applied to the time-division detection described in the third embodiment, and the scanning and counting methods described in the fourth embodiment.

<6. Other changes>

In addition, the above specific example is ultimately just one example, and the present invention can be used as a structure of many variations.

For example, in the above content, an example is given of applying the present invention to the case where the event detection unit 17 or the event detection unit 17' detects both the conduction event and the disconnection event, but the present invention can also be preferably applied to the configuration where the event detection unit 17 only detects the conduction event (or only detects the disconnection event).

Furthermore, the specific circuit configuration of each part shown in the example, especially the circuit configuration for reading the event signal, is shown as an example, and other configurations may be adopted.

<7. Summary of implementation forms>

As described above, the image sensor (same as 1) as an implementation form comprises: a light receiving unit (same as 16), which obtains an electrical signal corresponding to the amount of light received as a light receiving signal; an event detection unit (same as 17, 17'), which detects the change of the amount of light received as an event by taking the level of the past light receiving signal as a reference level (same as Lref) and obtaining the difference between the level of the current light receiving signal and the level of the previous light receiving signal; and a holding unit (same as 18, 18A, 18', 25, 25') which uses the event The detection unit inputs and displays the detection signal (output voltage Vop, Vom) of the detected event and holds it; the reading unit (same as 13, or the column driver 55 and the output circuit 56) reads the detection signal held in the holding unit as the event signal; and the reset unit (same as 19, 19A, 19B) resets the reference level to the current light receiving signal level of the light receiving unit after the event detection unit detects the event and before the reading unit reads the event signal.

In this way, the reference level used for event detection can be quickly reset (updated) to the current light-receiving signal level in accordance with event detection.

Therefore, the accuracy of the baseline level used for event detection can be improved, and the accuracy of event detection can be improved.

Furthermore, by providing a holding unit, even if the amount of received light changes in the direction of differential reduction between event detection by the event detection unit and reading by the reading unit, an event signal can be appropriately output.

Furthermore, in the image sensor as an implementation form, the reset unit (same as 19, 19B) resets the reference level based on the detection signal output by the event detection unit.

In this way, the reference level can be reset after the event detection unit detects the event and before the reading unit reads the event signal.

Therefore, the accuracy of the baseline level used for event detection can be improved, and the accuracy of event detection can be improved.

Furthermore, in the image sensor as an implementation form, the reset unit resets the reference level based on a signal that delays the detection signal.

In this way, the timing of resetting the reference level (i.e., the timing of starting detection for a new event) can be delayed until the level of the detection signal held in the holding unit reaches a certain level or above.

Therefore, event signals can be output appropriately.

Furthermore, in the image sensor as an implementation form, the reset unit (same as 19A) resets the reference level based on the output signal of the holding unit.

In this way, the reference level is reset in response to the level of the detection signal held in the holding unit becoming above a certain level.

Therefore, the event signal can be output appropriately. In addition, there is no need for a delay circuit for delaying the reset timing until the level of the detection signal held in the holding unit reaches a certain level or above.

In addition, the image sensor as an implementation form is provided with a disconnection unit (output control unit 20), and the disconnection unit (output control unit 20) disconnects the output of the detection signal from the event detection unit to the holding unit according to the level of the detection signal held in the holding unit (refer to FIG. 9).

In this way, control can be performed to appropriately output event signals, such as blocking the output of the detection signal to the holding unit when the level of the detection signal of the holding unit reaches a certain level or above, or releasing the blocking state of the output of the detection signal to the holding unit when the event signal is read from the holding unit and the level of the detection signal of the holding unit falls below a certain level.

Therefore, it is possible to seek to improve event detection accuracy.

Furthermore, in the image sensor as an implementation form, the output is cut off accordingly when the level of the detection signal of the interruption part and the holding part is above a certain level.

This can prevent the detection signal of a new event generated before reading from being assimilated with the detection signal of the immediately preceding event.

Therefore, the event signal for a new event generated before reading can be output as a signal different from the event signal of the event generated immediately before, thereby preventing the output of the event signal from being missed. In other words, the accuracy of the event signal output operation can be improved.

Furthermore, in the image sensor as an implementation form, when the level of the detection signal of the interruption part and the holding part is reduced to a level below a certain level, the output blocking state is released accordingly.

In this way, when the level of the detection signal of the holding unit drops below a certain level while the signal is read from the holding unit, the state in which the detection signal can be input to the holding unit can be switched, and the holding unit holds the detection signal of the new event.

Therefore, the event signal for a new event generated before reading can be output as a signal different from the event signal for the event generated immediately before, thereby preventing the output of the event signal from being missed. In other words, the accuracy of the event signal output operation can be improved.

In addition, in the image sensor as an implementation form, the disconnection part is configured to have a switch (same as SWp, SWm), and the switch inserts the detection signal from the event detection part to the output line of the holding part and is turned on/off by the output signal of the holding part.

Thus, as long as at least a switch that turns on/off by the output signal of the holding unit is provided as a component of the blocking unit, it will suffice.

Therefore, it is possible to simplify the structure of the disconnection part and reduce the circuit size.

Furthermore, in the image sensor as an implementation form, the holding unit (25, 25') counts and holds the number of times the detection signal is obtained in the event detection unit as the number of times the event is generated, and the reading unit reads the count value held in the holding unit as the event signal.

In this way, even if the waiting time from the occurrence of an event to its reading is long, a signal showing the number of times the event has occurred until it is read can be output as an event signal.

Therefore, it is possible to estimate not only whether there is a change in the amount of light received, but also the amount of change in the amount of light received.

Furthermore, in the image sensor as an implementation form, there are a plurality of light receiving parts, each light receiving part has an event detection part, and the reset part resets the reference level of the plurality of event detection parts by a common reset signal (refer to Figures 13 and 15).

In this way, the number of reset parts that should be prepared can be reduced while realizing the reset of each event detection part.

Therefore, it is possible to achieve the miniaturization of the circuit scale when resetting each event detection unit, and to achieve the miniaturization of the image sensor.

Furthermore, in the image sensor as an implementation form, there are a plurality of Bayer-arranged pixels with four light-receiving parts, and each pixel has a reset part.

Thus, when a pixel structure capable of capturing color images is adopted, the circuit scale can be reduced compared to the case where the unit of event detection is set to an appropriate unit as a pixel unit and a reset unit is provided for each event detection unit.

Therefore, in order to construct an image sensor capable of capturing color images, it is possible to achieve both the appropriateness of the event detection unit and the miniaturization of the circuit scale.

Furthermore, the recording device (camera 100) as an implementation form is provided with: the image sensor as an implementation form described above, and a recording unit (same as 102) for recording the event signal read out by the reading unit of the image sensor.

The recording device according to this embodiment can also obtain the same function and effect as the image sensor according to the above embodiment.

In addition, as a reset method of an implementation form, the level of the past light receiving signal of the light receiving unit that obtains the electrical signal corresponding to the light receiving amount as the light receiving signal is used as the reference level, and the difference between the level of the current light receiving signal and the level of the light receiving signal is obtained, so as to detect the change of the light receiving amount as an event; the detection signal of the event is input and displayed and maintained; the maintained detection signal is read as an event signal; and after the event is detected and before the event signal is read, the reference level is reset to the current light receiving signal level of the light receiving unit.

According to the reset method of this implementation form, the same function and effect as the image sensor of the above-mentioned implementation form can be obtained.

In addition, the effects described in this manual are ultimately illustrative and not limiting, and may have other effects.

<8. The present invention>

In addition, the present invention may also adopt the following structure.

(1)

An image sensor comprises: a light receiving unit, which obtains an electrical signal corresponding to the amount of light received as a light receiving signal; an event detection unit, which uses the level of the previous light receiving signal as a reference level, obtains the difference between the level of the previous light receiving signal and the level of the current previous light receiving signal, and detects the change of the above-mentioned light receiving amount as an event; a holding unit, which receives a detection signal indicating the detection of the above-mentioned event from the above-mentioned event detection unit and holds it; a reading unit, which reads the above-mentioned detection signal held in the above-mentioned holding unit as an event signal; and a reset unit, which resets the above-mentioned reference level to the current light receiving signal level of the above-mentioned light receiving unit after the above-mentioned event detection unit detects the above-mentioned event and before the above-mentioned reading unit reads the above-mentioned event signal.

(2)

As in the image sensor of (1) above, the reset unit resets the reference level based on the detection signal output by the event detection unit.

(3)

As in the image sensor of (2) above, the reset section resets the reference level based on a signal that delays the detection signal.

(4)

As in the image sensor of (1) above, the reset unit resets the reference level based on the output signal of the holding unit.

(5)

The image sensor as described in any one of (1) to (4) above has a discontinuation section, which discontinuously outputs the detection signal from the event detection section to the holding section in accordance with the level of the detection signal held in the holding section.

(6)

As in the image sensor of (5) above, when the level of the detection signal of the interrupter and the holding unit is above a certain level, the output is accordingly cut off.

(7)

As in the image sensor of (6) above, when the level of the detection signal of the interruption part and the holding part drops below a certain level, the output blocking state is correspondingly released.

(8)

As in the image sensor of (7) above, the disconnection unit is configured to have a switch, the switch is inserted into the output line that outputs the detection signal from the event detection unit to the holding unit, and is turned on/off according to the output signal of the holding unit.

(9)

An image sensor as described in any one of (1) to (3) above, wherein the holding unit counts and holds the number of times the detection signal is obtained in the event detection unit as the number of times the event occurs; and the reading unit reads the count value held in the holding unit as the event signal.

(10)

The image sensor as described in any one of (1) to (9) above has a plurality of light-receiving parts, and each light-receiving part has an event detection part; and the reset part resets the reference level of the plurality of event detection parts according to a common reset signal.

(11)

The image sensor as described in (10) above has a plurality of Bayer-arranged pixels having four aforementioned light-receiving parts; and each of the aforementioned pixels has the aforementioned reset part.

12: Arbitrator

13: Reading Department

13a: Transmission Department

14:Signal processing circuit

16: Light receiving part

17: Event Detection Department

18:Maintenance Department

19: Reset Department

Evnm: event signal

Evnp: event signal/output signal

L1: signal line

RST: reset signal

SWr: Reset switch

Vom: output voltage

Vop: output voltage

Claims (13)

An image sensor comprises: a light receiving unit that obtains an electrical signal corresponding to the amount of light received as a light receiving signal; an event detection unit that uses the level of the previous light receiving signal as a reference level, obtains the difference between the level of the previous light receiving signal and the level of the current previous light receiving signal, and detects the change of the above-mentioned light receiving amount as an event; a holding unit that receives and holds the detection signal indicating the detection of the above-mentioned event from the above-mentioned event detection unit; a reading unit that reads the above-mentioned detection signal held in the above-mentioned holding unit as an event signal; and a reset unit that resets the above-mentioned reference level to the current light receiving signal level of the above-mentioned light receiving unit after the above-mentioned event detection unit detects the above-mentioned event and before the above-mentioned reading unit reads the above-mentioned event signal. The image sensor of claim 1, wherein the reset unit resets the reference level based on the detection signal output by the event detection unit. The image sensor of claim 2, wherein the reset unit resets the reference level based on a signal that delays the detection signal. The image sensor of claim 1, wherein the reset unit resets the reference level based on the output signal of the holding unit. The image sensor of claim 1 is provided with an interruption section, which interrupts the output of the detection signal from the event detection section to the holding section in accordance with the level of the detection signal held in the holding section. The image sensor of claim 5, wherein the output is blocked when the level of the detection signal of the interruption section and the holding section is above a certain level. The image sensor of claim 6, wherein the interruption section and the holding section release the output blocking state accordingly when the level of the detection signal of the interruption section drops below a certain level. The image sensor of claim 7, wherein the disconnection unit is configured to have a switch, the switch is inserted into the output line for outputting the detection signal from the event detection unit to the holding unit, and is turned on/off according to the output signal of the holding unit. The image sensor of claim 1, wherein the aforementioned holding section counts and holds the number of times the aforementioned detection signal is obtained in the aforementioned event detection section as the number of times the aforementioned event is generated; and the aforementioned reading section reads out the count value held in the aforementioned holding section as the aforementioned event signal. The image sensor of claim 1 has a plurality of light-receiving parts, and each light-receiving part has an event detection part; and the reset part resets the reference level of the plurality of event detection parts according to a common reset signal. The image sensor of claim 10 has a plurality of Bayer-arranged pixels having four aforementioned light-receiving parts; and each of the aforementioned pixels has the aforementioned reset part. A recording device, comprising: an image sensor; and a recording unit, which records the event signal read by the reading unit; and the image sensor comprises: a light receiving unit, which obtains an electrical signal corresponding to the amount of light received as a light receiving signal; an event detection unit, which uses the level of the light receiving signal in the past as a reference level, obtains the difference between the level of the light receiving signal in the present and the level of the light receiving signal in the past, and detects the change of the amount of light received as an event. Detection; A holding unit, which receives the detection signal indicating the detection of the event from the event detection unit and holds it; A reading unit, which reads the detection signal held in the holding unit as an event signal; and A resetting unit, which resets the reference level to the current light receiving signal level of the light receiving unit after the event detection unit detects the event and before the reading unit reads the event signal. A reset method, which uses the level of the previous light receiving signal of a light receiving unit that obtains an electrical signal corresponding to the light receiving amount as a light receiving signal as a reference level, obtains the difference from the level of the current previous light receiving signal, and detects the change of the light receiving amount as an event; inputs a detection signal indicating the detection of the above event and holds it; reads the held detection signal as an event signal; and after the above event is detected and before the above event signal is read, resets the reference level to the current light receiving signal level of the above light receiving unit.