WO2016034408A1 - Method and device for the simplified detection of a depth image - Google Patents

Abstract

The invention relates to a device and a method for controlling a light propagation time camera, wherein distance information for a three-dimensional image is ascertained from a propagation time difference or a phase shift between a light signal (L) ascertained from a light source of the propagation time camera and a light signal received by an image sensor of the propagation time camera by scattering or reflecting the emitted light signal (D). At least three sub-images are captured. At least one light pulse is emitted, and a closure device (shutter) is clocked using at least three control signals (CO, C1, C2) such that the phase offset between the light pulse and the shutter is varied by means of the different phase positions of the control signals between the sub-images. In this manner, three measurement values (SO, S1, S2) are obtained per pixel. The measurement values can be fed to an operator which is constructed analogously to a hue operator and which ascertains a rotational angle about the space diagonal of the SO-S1-S2 space. The rotational angle can then be converted into the distance information using a proportionality constant.

Description

 Method and device for simplified detection of a

 depth image

The invention relates to an apparatus and a method for detecting a three-dimensional depth image based on image information from an image sensor with one or two-dimensional pixel arrangement. For the electronic capture of images nowadays mostly digital cameras are used. Such a digital camera is described, for example, in US4131919A and in EP2367360A2 and is hereinafter referred to as a conventional digital camera. Brightness images are taken using conventional digital cameras. In such a brightness image, brightness values are assigned to the pixels.

Fig. 1 shows a schematic construction of a conventional digital camera 10 which captures a scene 11 consisting of objects 17, in which the image is captured by means of an optical system 12, e.g. is imaged by means of a lens on at least one image sensor 13, whose signals are converted by an electronic image processing device 14 into a digital image, which is output by means of an interface 15 via a signal transmission line 16. In this case, it is possible for a storage to take place beforehand in the electronic image processing device 14 or for the signal transmission line 16 to lead to a storage medium (not shown) in which the image is stored.

Fig. 2 shows a schematic structure of an image sensor 170 for conventional cameras, which will be referred to as a conventional image sensor. Such image sensors usually consist of a periodic arrangement of pixels 171 (hereinafter referred to as pixels). Predominantly one-dimensional arrays are referred to as line sensors, and predominantly two-dimensional arrays are referred to as area sensors. The image sensor shown in FIG. 2 is thus an area sensor with conventional pixels 171. Regardless of the technology, the pixels 171 have in common that they each have a photosensitive area 172, typically a photodiode (PD) or a so-called pinned Photodiode (PPD), which is designed so that during an exposure time in response to the incident light generates an electrical variable that is a measure of the amount of light received by the pixel in question. This electrical quantity may be a charge, a voltage, a current or even a time-coded signal, such as a pulse train.

Often, such image sensors are constructed as a so-called charge-coupled device (CCD), as described for example in US4194213A. A special form of CCDs are the so-called inter-line displacement CCDs (Inter-line Transfer CCDs), which are explained, for example, in US Pat. No. 4,665,659A or in EP2150039A1.

A more recent technology is the so-called Complementary Metal-Oxide-Semiconductor (CMOS) image sensors. These nowadays usually have active pixels, as described for example in US5471515A. Numerous different active-pixel circuits for CMOS image sensors are known, including those which include a rolling electronic shutter, called an Electronic Rolling Shutter (ERS). A circuit for such a pixel is shown in US5841126. Other active pixels may form a so-called Global Shutter (GS), as disclosed for example in US6667768. Here are pixels for conventional image sensors regardless of the technology in which they are performed, that is CCD or CMOS, referred to as conventional pixels.

In addition, GS pixels typically have a signal memory 173, as shown in FIG. 2, in which charges or voltages can be stored. Frequently, CMOS pixels are what is known as floating diffusion (FD), while charge storage in CCDs often takes place in metal-oxide semiconductor (MOS) diodes.

In image sensors for conventional cameras, there are also pixels which are equipped with more than one signal memory, for example to perform a so-called Correlated Double Sampling (CDS), or to achieve an extended dynamic range (HDR). An example of this is disclosed in US2012 / 0193516A. The aforementioned pixel architectures according to FIG. 2 have in common that there is only one signal path 174 in each case with which the electrical variable is transmitted from the photosensitive area to the at least one associated signal memory, provided that it is structurally separated from the photosensitive area. Although in some cases, such as in US6667768, the photosensitive area is still connected to an electrical shutter via a control path 175, for example to a so-called reset transistor, this control path 175 does not form a second signal path to a latch, but merely connects the photosensitive area to a supply voltage so that the charge is removed from the pixel outside the exposure time.

There are monochrome cameras called monochrome. Monochrome cameras do not have the capability in the To distinguish image capture between different colors. In addition, there are also color cameras that can detect such a distinction between different colors. For example, they can use an image sensor having a so-called mosaic filter with different colors, as described in US3971065.

Furthermore, there are three-dimensional (3D) cameras which generate so-called distance images or depth images in which distance values are assigned to the pixels, which represent a measure of the distance between the camera and the object.

In this case, it is possible that the depth images are output directly, or that further processing steps are carried out internally, for example the production of so-called point clouds from 3D coordinates or the interpretation of the depth images, for example as a gesture of a hand.

Recently, there are increasingly offered three-dimensionally receiving time-of-flight cameras which perform a light-time-based distance measurement and are referred to as time-of-flight cameras (TOF cameras), as disclosed, for example, in DE102011089636A1.

3 shows a schematic structure of a ToF camera 20, which has a synchronization unit 21, which controls an electronic control device 22 for a light source 23 by means of a control signal 33 in such a way that this light source 23 transmits light or light pulses modulated in time. The emitted light beams 34 and 36 are scattered or reflected by objects 25 and 26 and partly pass back to the camera as object light beams 35 and 37, being delayed because they travel the necessary distance at approximately the speed of light. There they are with an optics 27, eg with a Lens, imaged on the image sensor 28. The image sensor 28 is in turn driven by means of a drive signal 32 from the synchronization unit 21 so that it performs a demodulation of the object light beams. From the raw data supplied by the image sensor 28, depth images are then generated in a computing unit 29 which are output via an interface 30 to a transmission line 31.

Such ToF cameras often have special ToF image sensors for measuring distances, as are known, for example, from DE19704496C2, US8115158 or US20120176476A1. These image sensors often work with pixels that are equipped with so-called photonic mixer detectors (PMD) or work on a related principle. These pixels will hereinafter be referred to as demodulation pixels and the image sensors as demodulation sensors.

4 shows a schematic structure of a ToF demodulation sensor 180 with demodulation pixels 282. Such demodulation pixels 181 have the common feature that there is in each case a photosensitive area 182 which is connected to at least two different signal stores 183 and 184 via at least two different signal paths 185 and 186 is connected. Frequently, in such demodulation pixels, the read-out electronics for the at least two signal memories are present more than once per pixel. Demodulation pixels are predominantly used together with light sources which are in so-called continuous wave operation

Sinusoidally modulated light is used very often or light with a pulse-pause ratio of approx. 50%.

Such image sensors with demodulation pixels are manufactured especially for ToF applications. They are generally more expensive and therefore more expensive than comparable conventional image sensors, or they have fewer pixels at the same price and thus lower spatial resolution. One reason for this is that demodulation pixels require more sophisticated electronics for their characteristic plurality of signal paths than conventional pixels. As a result, they have an increased space requirement compared to the conventional pixels, which leads to a higher resource requirement, for example chip area. Furthermore, in the case of a demodulation pixel, the area fraction of the total area of the pixel which is available for the photosensitive area is reduced compared to a conventional pixel. As a result, conventional pixels are generally more sensitive to light than demodulation pixels. Finally, conventional image sensors are currently produced in far greater numbers than ToF image sensors, resulting in a further price advantage for the conventional image sensors. For these reasons, it would be advantageous to be able to provide conventional image sensors with conventional pixels also for ToF cameras.

The conventional pixels in conventional image sensors, however, as mentioned above, usually have only one signal path and often only one signal memory. Even with such image sensors, however, ToF range images can basically be recorded.

From US5081530 a method is known in which a ToF image is taken with a conventional interline CCD (ILCCD) image sensor. However, this method has the disadvantage that it is applicable only to dark scenes without ambient light (except for the pulse light). The reason for this is that the number of electrons N1 and N2 collected in the CCD are additively increased by the ambient light by the number NU, ie to N1 + NU and N2 + NU, so that the distance is then calculated using the formula R / 2 * (N1 + N2). N2) / (N1 + N2) + R / 2 is erroneously calculated as R / 2 * (Nl-N2) / (N1 + N2 + 2 * NU) + R / 2. The problem associated with ambient or background light is discussed for example in DE19704496C2 as DC component and in EP1040366B1 as ambient light. Furthermore, the fact that only a single light pulse per image is used in US5081530A requires a very powerful and therefore expensive pulse light source.

Another method for capturing a ToF image by means of a conventional CMOS image sensor is known from EP1040366B1. Here, among other things, it is proposed to record three different raw images (referred to there as sensor signals) for determining a distance image, namely a first image with dark current and ambient light, a second raw image in which, depending on the light transit time, a part of the received light is integrated, and a third signal with a higher integration time. In another embodiment, dark current and ambient light are recorded twice, with short and with a long integration time. In addition, there are also conventional image sensors with pixels having not more than one signal path but more than one latch. An example of such a pixel architecture can be found, for example, for a CCD image sensor in US8576319A and for a CMOS image sensor in EP2109306A2. The invention has for its object to provide a light cycle camera, which can provide depth data with high measurement quality and high image resolution.

This object is achieved by a device according to claim 1, a light transit time camera according to claim 9, a A method according to claim 13 and a computer program according to claim 14.

Accordingly, the light source of the time of flight camera for emitting at least one light pulse and the image sensor for demodulating the received light signal are controlled by means of an electrical shutter associated with each pixel of the image sensor, the electrical shutter being clocked at least three times in fixed phase relation to the at least one light pulse (ie opened and closed) to detect at least three signals, wherein the time duration of the opening of the electrical closing device is equal to each, and wherein the phase shift between the first opening and the second opening is equal to the phase shift between the second opening and the third opening , Thus, at least three subimages are recorded, the shutter being clocked such that the phase shift between the light pulse and the shutter is varied by the different phase positions of the timing between the subimages. As a result, three measured values per pixel are obtained. These can be supplied to an operator who, for example, can be constructed analogously to a hue operator and determines a rotation angle about the spatial diagonal of the SO-Sl-S2 space. By means of a proportionality constant this can then be converted into the desired distance information.

Thus, the solution according to the invention can provide image processing with only one signal path per pixel of the image sensor, whereby the conventional image sensor-dependent advantages of low complexity, low noise, high lateral resolution and high frame rate can be maintained even in light-time cameras. Due to the inventive possibility of using conventional image sensors in ToF cameras, it is possible to record both conventional images and distance images with one and the same camera. Optionally, color image sensors can also be used. Since the color filters used in the Bayer pattern are usually transparent to infrared light, a distance image can also be recorded with the conventional use of an infrared light source with a conventional color image sensor. This has the advantage of making it possible to record both color images and distance images with one and the same ToF camera.

Preferably, the phase shift between the first control signal and the second control signal and the phase shift between the second control signal and the third control signal with respect to the period of the light pulse can each be 120 °.

According to a first advantageous development, the phase shift between the time of opening of the electrical closing device by the first of the three control signals and the time of delivery of the light pulse with respect to the period of the light pulse can be -60 °. Furthermore, the time duration of the pulse-shaped control signals for the opening of the electrical closure device with respect to the period of the pulse of light can correspond to a phase angle of 180 ° and the duration of the light pulse with respect to the period of the light pulse can correspond to a phase angle of 60 °. In this way, a particularly large distance measuring range can be achieved for the pulse length of the light pulse. While maintaining the The aforementioned time relationships can be achieved with an increase in the period of the light pulse signal advantageous avoidance of unwanted Tiefenaliasing.

According to a second advantageous development, the phase shift between the time of opening of the electrical closing device by the first of the three control signals and the time of delivery of the light pulse with respect to the period of the light pulse can be 0 °. Furthermore, the time duration of the pulse-shaped control signals for the opening of the electrical closure device with respect to the period of the light pulse, a phase angle of 60 ° and the duration of the light pulse with respect to the period of the light pulse corresponding to a phase angle of 180 °. This allows a high measurement quality with a smaller measuring range. Here too, while maintaining the above-mentioned time relationships, an advantageous avoidance of the undesired depth aliasing can be achieved by increasing the period duration of the light pulse signal.

According to a third advantageous development, the phase shift between the time of opening the electric

Locking means by the first of the three control signals and the time of delivery of the light pulse with respect to the period of the light pulse 0 °, but here the time duration of the pulse-shaped control signals for opening the electric shutter with respect to the period of the light pulse a phase angle of 180 ° and also the duration of the light pulse with respect to the period of the light pulse can correspond to a phase angle of 180 °. This enables high measurement quality with low nonlinear errors. Again, while maintaining the aforementioned time When the period of the light pulse signal is increased, an advantageous avoidance of the undesired depth aliasing can be achieved.

According to a fourth advantageous development, the phase shift between the time of opening of the electrical closure device by the first of the three control signals and the time of delivery of the light pulse with respect to the period of the light pulse can be 0 °, but now the time duration of the pulse-shaped Control signals for the opening of the electric shutter with respect to the period of the light pulse corresponds to a phase angle of 120 ° and also the duration of the light pulse with respect to the period of the light pulse corresponds to a phase angle of 120 °. This modification of the phase relationships makes it possible to determine the distance with a particularly low computational outlay and associated lower resources. Finally, an advantageous avoidance of the undesired depth aliasing can also be achieved while maintaining the above-mentioned time relationships with an increase in the period duration of the light pulse signal. Preferably, the proposed time of flight camera may include calculating means for calculating the distance information of the three-dimensional image based on the values of the three signals based on case discrimination for different ranges determined by mutual size ratios of the values of the three signals. The computing device can advantageously be configured such that the validity of the calculated distance information is determined on the basis of a ratio between signal quality and noise. Additionally or alternatively, the computing device can be designed so that a decision about the validity of the calculated distance information based on the degree of saturation of the three signals. According to the invention, in comparison to light-time cameras with demodulation sensors, it is a further advantage that a pulse-pause ratio which is smaller or even very much less than 50% can be used for the emitted light signal. This results in a lower sensitivity to ambient light and thus an improved measurement quality. This allows camera applications in strong ambient light, such as sunlight or studio spotlights.

The components of the device or time-of-flight camera proposed for achieving the above-mentioned object can be used individually or jointly as discrete circuits, integrated circuits (eg application-specific integrated circuits (ASICs)), programmable circuits (eg field programmable gate arrays (FPGAs). ) be realized. In particular, the arithmetic unit can be realized by an FPGA as a central component. Furthermore, the steps of the method claim can be realized as a software program or software routine for controlling the processor of a computer device for its execution.

Hereinafter, the present invention will be described with reference to preferred embodiments with reference to the drawing figures. Show it:

Fig. 1 shows a schematic structure of a conventional

 Digital camera;

2 shows a Schamtian structure of a conventional image sensor,

3 shows a schematic structure of a ToF camera;

4 shows a schematic structure of a ToF demodulation sensor with demodulation pixels; 5 (a) - (d) are schematic timing charts for a ToF camera according to a first embodiment;

Fig. 6 (a) - (e) are schematic timing charts for explaining the timing for light and control signals according to the first embodiment;

7 (a) and (b) are schematic timing charts showing waveforms and distance detection for the first embodiment;

8 shows a formula for determining the distance in the first embodiment;

FIGS. 9 (a) and (b) show trajectories of S vectors in S space according to FIG

 Prior art or according to the first embodiment;

10 (a) and (b) are timing charts for explaining a shift of a measuring range to a minimum distance;

11 shows a formula for determining the distance d at a shifted measuring range;

12 (a) and (b) are waveforms for explaining another shift of a measurement range to a minimum distance;

13 shows a formula for determining the distance d at a further shifted measuring range;

14 shows a diagram with signal curves taking into account the law of distance; Fig. 15 is a formula for calculating the depth noise;

Fig. 16 is a formula for determining the validity of distance values;

17 is a diagram illustrating an approximation of FIG

 Noise by means of a simplified approximation function;

18 shows a time diagram with the signal curve of an ideal and a real light pulse;

19 shows a diagram with the curve of a real measured distance as a function of the time difference;

FIG. 20 shows time charts with a time controller according to a second embodiment; FIG.

21 (a) and (b) are diagrams showing resultant waveforms according to the second embodiment; Fig. 22 is a formula for determining the distance d in the second embodiment;

Figs. 23 (a) and (b) are timing charts showing the peak value and average value of a pulse frequency modulated signal at different pulse repetition frequencies; Fig. 24 timing diagrams with a timing according to

 third embodiment;

Fig. 25 is a formula for calculating the depth noise in the third embodiment; G. Fig. 26 is a formula for determining the validity of dependency values of the third embodiment;

FIG. 27 shows time charts with a timing according to a fourth embodiment; FIG.

FIG. 28 is a timing chart showing a timing according to a fifth embodiment; FIG.

29 are diagrams showing resulting waveforms according to the fifth embodiment;

Figs. 30 (a) and (b) show a trajectory of the S vectors for the fifth

 Embodiment from two different directions of view;

Fig. 31 is a formula for determining the distance from the

 Signaling of the fifth embodiment;

FIG. 32 shows a curve of a real measured distance as a function of the time difference for the fifth exemplary embodiment; FIG.

33 shows time charts with a timing according to a sixth embodiment;

FIG. 34 is a graph showing resulting waveforms according to the sixth embodiment; FIG.

Fig. 35 is a formula for determining the distance in the sixth

 Embodiment;

Fig. 36 timing diagrams with a timing according to

seventh embodiment; FIG. 37 is diagrams showing resultant waveforms according to the seventh embodiment; FIG.

Figs. 38 (a) and (b) show a trajectory of the S vectors for the seventh

 Embodiment from two different directions of view;

Fig. 39 is a formula for determining the distance in the seventh

 Embodiment;

Fig. 40 is a formula for calculating the depth noise in the seventh embodiment;

41 shows time charts with a time control according to an eighth embodiment;

FIG. 42 are diagrams showing resultant waveforms according to the eighth embodiment; FIG. and

Fig. 43 is a formula for determining the distance in the eighth

 Embodiment.

According to the following embodiments, a ToF camera with modified and thus more effective image processing will be described.

5 (a) to (d) are timing charts showing waveforms according to a first embodiment of operating a ToF camera of FIG. 3 with an image sensor 28, which is preferably a conventional image sensor of FIG. 5 (a) shows the timing for emitting light L from the light source 23 in the camera 20 with two different intensity levels "0" and "1" over time t. For example, at level "1" light is emitted and No light is emitted at the "0" level, the light is emitted as a time-limited light pulse 40 having a temporal pulse length t1 and having a period tp after a dead time to optional as a second light pulse 60 and a third light pulse 61, and so on. For a simplified understanding of the time or phase relationships, the period tp is here equated to 360 ° In the example shown, the pulse length t1 of the light signal then corresponds to a phase value of 60 °, since t1 = 1/6 * tp and corresponding to = 5/6 * tp.

In the camera 20 according to FIG. 3, incident light D is detected by the image sensor 28. The associated waveform is also shown in Fig. 5 (a). Without the influence of the emitted light pulse 40, the incident light initially only consists of the ambient light 43 with the intensity level B assumed to be constant. The incident light pulse 42 is added to this ambient light, resulting in the time profile of D shown. This has traveled between emission and incidence from the light source 23 to the objects 25, 26 and back to the image sensor 28 a distance at about the speed of light and is therefore delayed by a time difference td. Furthermore, the light pulse 40 is also reduced by the propagation in space in accordance with the law of distance and the incomplete reflection on the object in its intensity, so that the received light pulse 42 is generally less intense than the emitted light pulse 40th

According to FIG. 3, the image sensor 28 is driven by the synchronization unit 21 by means of the drive signal 32, so that it carries out a demodulation of the object light beams. This control signal 32 is formed in the following exemplary embodiments by three control signals CO, C1 and C2. The control The image sensor 28 takes place by means of an electric shutter (Shutters) of the type mentioned.

In Fig. 5 (b), the first control signal CO for controlling the electric shutter of the image sensor 28 shown in Fig. 3 is shown over the time t for obtaining a first raw image. Specifically, control signal CO assumes states "0" and "1" with the electric shutter in the "0" state closed and the image sensor 28 unable to receive light while in the "1" state and the image sensor 28 light can receive. The control signal CO is applied in parallel to the emitted light pulses 40, 60 and 61 with fixed phase relation phi0 shown in Fig. 5 (a) to obtain a first raw image. In this case, the phase relation between the opening of the electric shutter with respect to the emission of the light pulse 40 is phiO, in the first exemplary embodiment shown phiO = -60 °. The opening time is 180 °.

As a result, a signal component 47 of the ambient light 43 is detected by the image sensor 28 during the opening time of the electrical shutter, while another signal component 48 of the ambient light 43 is not detected outside the opening time of the electrical shutter. In accordance with its delay, a signal component 45 of the incident light pulse 42 is additionally detected, while another signal component 46 of the incident light pulse is not detected. This can be realized, for example, by removing the charge from the photosensitive region 172 via the electrical shutter (not shown) of FIG. 2 connected to the signal path 175 when the control signal CO becomes "0", and then when the control signal CO is in the "1" state assumes the charge from the photosensitive area 172 is cumulatively supplied to the memory 173 via the signal path 174.

From the incident light, a signal is formed, for example a charge, a voltage, a current or a digital number. If several light pulses 40, 60 and 61 have been emitted and the electrical shutter has been opened several times, the signals of the individual openings of the shutter become analogous, as charges in the memory 173, i. as charge packets, added or cumulated, which is symbolized by the adding function 49, and finally form a signal SO. This signal is assigned to one pixel each. Together with further signals SO from other pixels, a first raw image is produced for the signal SO. Optionally, the adding function 49 could also be realized in the arithmetic unit 29 shown in FIG. 3, if signals of the individual shutter openings are separately digitized and processed. The same applies to the (symbolic) adding functions 54 and 59 mentioned below.

In Fig. 5 (c), a second control signal Gl for controlling the electric shutter over the time t for obtaining a second raw image is shown. The terms used correspond to those of Fig. 5 (b). Here now another phase relation is used. The phase relation between the rising edge 44 of the signal CO and the rising edge 50 of the signal Gl is phil and in the example shown has the value phil = 120 °. The opening time is 180 °.

Thereby, a signal portion 52 of the ambient light 43 is received during the opening time of the electric shutter, while another signal portion 53 of the ambient light 43 is not received outside the opening time of the electric shutter. In this case, according to its delay, the entire signal component 51 of the incident light pulse 42 is received.

A signal is again formed from the incident light, and if several light pulses 40, 60 and 61 have been emitted and the electrical shutter has been opened several times, the signals of the individual apertures of the shutter are analogously represented by a symbolic addition function 54 as charges in the memory 173, i. as charge packets, added or accumulated and finally form a signal Sl. This signal is also assigned to one pixel. Together with further signals Sl from other pixels, a second raw image is produced for the signal S1. Optionally, the adder function 54 could also be implemented in the arithmetic unit 29 shown in FIG. 3 if signals of the individual shutter openings are separately digitized and processed.

Finally, in FIG. 5 (d), a third control signal C2 for controlling the electrical closure over time t is shown for obtaining a third raw image. Again, the terms used correspond to those of Figs. 5 (b) and (c). In the example shown phi2 = 120 °, as well as phil. The opening time is again 180 °.

As a result, only one signal component 58 of the ambient light 43 is received during the opening time of the electrical shutter, while another signal component 57 of the ambient light 43 is not received outside the opening time of the electrical shutter. In this case, according to its delay, the entire signal component 56 of the incident light pulse 42 is not received. The signals of the individual openings of the shutter are also added or accumulated here analogously by means of a symbolic adding function 59 as charges in the memory 173, ie as charge packets, and finally form a signal S2. Also this signal is assigned to one pixel each. Together with further signals S2 from other pixels, a third raw image is produced for the signal S2. Optionally, the adding function 59 could also be realized in the arithmetic unit 29 shown in FIG. 3, if signals of the individual shutter openings are separately digitized and processed.

The control signals CO, C1 and C2 have the same period tp as the light L.

By combining the three raw images for each pixel, there are finally three signals SO, Sl and S2. Assuming that the object has not made any significant movements during the recording of the three raw images and the ambient light 43 and the intensity of the received light pulse 42 are the same for each pixel, for each pixel of the three signals SO, S1 and S2, the Time difference td and by with knowledge of the speed of light, the distance of the imaged on this pixel signal component of the object 25 are calculated. From this a distance image can be calculated in which distance values are assigned to the pixels.

The selection of the number of three control signals and signals is based on the following considerations. In determining the distance of an object over the duration of the light three unknowns are present, namely the distance, the intensity of the ambient light and the intensity of the incident light pulse. To determine this problem with three degrees of freedom at least three different signals are needed. At the same time, to produce high frame rates, it is necessary to keep the number of raw images as small as possible. Therefore, the number of raw images and the associated number of control signals can be selected as three. Figs. 6 (a) to (e) are diagrams for explaining the occurrence of the signals by using the timing for light and control signals. Specifically, Figs. 6 (a) to (e) show how the phase relationship between the incident light D and the control signal C results in a waveform of the signal S over the time difference td.

In Fig. 6 (a), the incident light pulse 70 arrives so that a signal portion 71 of the received light pulse is before the rising edge 73 of the control signal C and is not received, while another signal portion 72 is behind the rising edge 73 and received accordingly becomes. The later now the incident light pulse 70 arrives, the smaller is the non-received signal component 71 of the incident light pulse and the greater is the received signal component 72. Thus, in the case shown, an increasing profile 74 of the signal S results over the time difference td.

In Fig. 6 (b), the incident light pulse 75 arrives and the control signal C becomes "1" during the entire time of arrival, thereby receiving the incident light pulse as the completely received light pulse 76. As long as this is the case a constant high value 77 for the signal S.

In FIG. 6 (c), the incident light pulse 78 arrives so that a signal portion 79 of the received light pulse is before the falling edge 81 of the control signal C and is received, while another signal component 80 is behind the falling edge 81 and accordingly not received. The later now the incident light pulse 78 arrives, the smaller the received signal portion 79 of the incident light pulse and the greater is the non-received signal component 80. Thus results in the case shown, a falling curve 82 of the signal S over the time difference td.

In Fig. 6 (d), the incident light pulse 75 arrives at a time when the control signal continuously assumes the state "0." This completely suppresses the received light pulse 84. As long as this is the case, the signal S results a constant low value 85.

This results in combination of the aforementioned cases of the waveform shown in Fig. 6 (e) 86 of the signal S over the time difference td, which is piecewise composed of a rising edge, a high constant value, a falling edge and a low constant value.

Figs. 7 (a) and (b) are diagrams showing waveforms for the first embodiment. More specifically, Fig. 7 (a) shows the signals SO, Sl and S2, and Fig. 7 (b) shows the value for the distance d determined therefrom.

Accordingly, from the waveforms of the control signals shown in FIGS. 5 (a) to (d), the waveform shown in FIG. 7 (a) is obtained for the signals SO, S1 and S2. The signals thereby move between a background signal Sb, which results only from the contribution of the received ambient light 58 to the signals SO, S1 and S2, and the peak signal Sp, which is the sum of the contribution of the received ambient light and the received light pulses 51 and 52 to the signals SO, Sl and S2.

An S-space can be defined as space, which is spanned from the unit vectors SO, Sl and S2 as a basis. Similarly, the value triples of the signals SO, Sl and Sl define an S-vector having as components the values of the signals SO, Sl and S2.

By comparing the magnitude of the signals in FIG. 7 (a), six different ranges for the time difference td can be distinguished in S-space, namely the range 90 with S2 <= S1 <S0, the range 91 with S2 <S0 <= S1 , the area 92 with S0 <= S2 <S1, the area 93 with S0 <S1 <= S2, the area 94 with S1 <= S0 <S2 and the area 95 with S1 <S2 <= S0. If the time difference td exceeds the period tp, then there are, for example, in the area 96 values for the signals SO, S1 and S2 which can not be distinguished from those in the area 90 and in the area 97 values which are not distinguished from those in the area 91 can. This effect is called depth aliasing. If one looks at the origin of the S-space along the unit diagonal, this division corresponds to a segmentation along the unit diagonal with six pie-shaped segments of 60 ° each.

It can be seen that in Fig. 7 (a) always exactly one of the three signals SO, Sl and S2 assumes a rising or falling course, while a second of the three signals has a constant high value and a third of the signals has a constant low value accepts. In this case, the course of the control signals CO, C1 and C2 shown in FIG. 5 with respect to the course of the light L in the first exemplary embodiment was carefully chosen so that this condition is fulfilled and the signal course shown in FIG. 7 (a) results.

This waveform offers several advantages. First, it is possible to uniquely determine the value of the td from the signals SO, S1 and S2 within an interval of 0 <= td <tp. The following prerequisites are fulfilled for this: At any given time, at least two of the signals SO, S1 and S2 are different. And At any given time, at least one of the signals S0, S1 and S2 changes in a different way than the other two of the signals S0, S1 and S2 as a function of the time difference td. Since only one signal changes at a time and all rising and falling Flanks of the control signals were used to generate a rising or falling waveform, it is recognizable that so can achieve the largest possible measuring range for the time difference td for a given pulse length tl.

From the signals shown in Fig. 7 (a), the distance d of objects 25, 26 shown in Fig. 7 (b) from the ToF camera 20 in Fig. 3 can be determined. It is assumed that the measuring range is as close as possible to the ToF camera 20 and therefore starts with the distance d = 0. If a combination of the signals SO, Sl and S2 is measured, which corresponds to a time difference td of 0, this corresponds to a distance d of 0. With a linearly increasing time difference td results in a first region 100 with proportional increase of the distance d, to finally short before the time difference td = tp almost a maximum distance value dp is measured. From the time tp, the signals SO, S1 and S2 repeat periodically, so that the combination of the signals SO, S1 and S2 at the time td = tp coincides with the combination of the signals SO, S1 and S2 at the time td = 0 and thus a sudden drop 101 to the distance value d = 0. From time t.sub.p, in a second area 102, a periodic repetition of the proportional increase in the distance d ensues. As a result, the distance d in the period 0 <= td <tp can be correctly determined. However, both the precipitous drop 101 and the periodic repetition are measurement errors due to depth aliasing. The proportionality factor for the proportional increase in the first region 100 results from the speed of light c and the pulse length t1, the latter namely the slope of the within the ranges in Fig. 7 (a) respectively increasing or decreasing signal influenced. The proportionality factor is c * tl 12. FIG. 8 shows a formula used in the arithmetic unit 29 for determining the distance d from the signals SO, S1 and S2. The distinction between regions 90 to 95 shown in FIG. 7 (a) results in a linear signal curve within each region, so that in each region the distance d can be determined using a linear equation. Fig. 8 gives a formula by means of which linear equations for the regions 90 to 95 can be combined by means of a case distinction into a single piecewise linear formula. Where d is the distance to be measured, c is the speed of light, and tl is the pulse length. The six case discriminations by inequalities are made to determine to which of the six regions 90 to 95 the S vector belongs and with which linear equation a distance is to be calculated from its components. For each area, a linear equation is given. By adding the numbers 0, 1, 2, 3, 4, 5 a continuity in the determined distance d is achieved. The expression "NaN" means "not a number" and states that in the case S0 = S1 = S2 a distance calculation is fundamentally not possible, since in this case the three unknowns can not be determined. By the formula shown in Fig. 8, from the signals shown in Fig. 7 (a), a value for the distance d shown in Fig. 7 (b) can be correctly determined.

The values of the signals SO, S1 and S2 are thus supplied to an operator, which is constructed analogously to a Hue operator and a rotation angle about the spatial diagonal of the S0-S1-S2 space determined. By means of a proportionality constant, this is then converted into the distance information.

The proposed method makes it possible to achieve a particularly large measuring range for a given pulse width t1. The measuring range extends from a distance d from zero to a maximum distance dp resulting from the period tp by the formula dp = c / 2 * tp.

FIGS. 9 (a) and (b) show a trajectory of S vectors in S space according to the prior art and the first embodiment, respectively.

The knowledge gained from the first embodiment can be visualized as follows. In FIG. 9, an S-space is shown as a cube and the trajectory of the S-vectors is plotted with increasing time difference td. Here, Fig. 9 (a) shows a trajectory according to the prior art and Fig. 9 (b) shows a trajectory according to the first embodiment.

One way of measuring the time difference td by means of the three signals SO, Sl and S2 is to measure the ambient light as a constant signal S2, partially receive an incident light pulse and measure as rising signal Sl and the ambient light plus the full incident light pulse to measure SO as a constant signal. This results in a linear course in S-space, as shown by way of example in FIG. 9 (a), the trajectory drawn here assuming an ambient light of zero.

Fig. 9 (b) shows the trajectory of the S vectors in S space resulting from the waveform shown in Fig. 7 (a) assuming that the intensity of the ambient light is zero. Compared to FIG. 9 (a), it is clear that the first embodiment, a 6 times as long trajectory is achieved. At the same time, it becomes apparent why the selected trajectory has a maximum length under the given boundary conditions. The given boundary conditions consist of the requirement that the time difference td and thus the distance d can be determined independently of the intensity of the ambient light and of the intensity of the incident light pulse. This particularly long trajectory makes it possible to achieve a particularly large measuring range. In certain applications, it may be appropriate that the measurement range having a certain range of distances given by the period tp does not start from zero at a distance d from the camera, but at a different, selectable distance. A possible reason for this is that objects from which distance images are to be created, for example parcels on a conveyor belt, on which a camera mounted above is directed from above, can not exceed a certain height and the measuring range can be optimally adapted to the actually occurring quantities should. Another possible reason is that objects in too close proximity cause the signals of the image sensor to saturate the signals and therefore the distance can not be measured correctly anyway, so that it is advantageous to adapt the measuring range to correctly measurable distances , This circumstance will be explained in more detail in the discussion of FIG. 14.

Figs. 10 (a) and (b) show waveforms when the measuring range is shifted to a minimum distance dmin by 120 °, as an example of a multiple of 60 °.

Since the timing of Fig. 5 (a) to (d) has not been changed for the displacement, the waveform in Fig. 10 (a) is also shown. with respect to the waveform in Fig. 7 (a) unchanged. However, from these same signal values, the shifted distance values shown in Fig. 7 (b) are calculated here. The range of the distance ranges from a minimum distance dmin, which is greater than zero, to almost a maximum distance dmax, which is advantageously greater than dp. The small values of the time difference td, which belong to a rising distance value 103 lying ahead of the measuring range, are then subject to a depth aliasing and do not occur according to the changed task. Within the measuring range in the signal areas 92 to 97, the signals are converted correctly into an increasing distance d in a first area 105. Behind the falling edge 106 of the distance d, a further increase occurs in a second area 107 of the distance, which however subject to a depth aliasing.

FIG. 11 shows a formula used in the arithmetic unit 29 for determining the distance d from the signals SO, S1 and S2 with a measuring range shifted by the two signal ranges 90 and 91. The formula according to FIG. 11 can be realized with the shifted measuring range shown in FIG. Compared to FIG. 7, it can be seen that the signal regions 90 and 91 result in a different distance interpretation for the same signals SO, Sl and S2, which is found in the formula in the offset values "6" and "7". Generally speaking, for the regions affected by an increased interpretation of the distance, the offset value is increased by "6." In principle, this process can also be carried out several times, so that a minimum distance dmin beyond dp can also be realized.

Figs. 12 (a) and (b) show waveforms when the measuring range is shifted to a minimum distance dmin by 150 ° as an example of a multiple of 60 ° Value. The signal curves shown here are represented as a function of a time difference, ie for each value of td, a separate time diagram must run and then supplies the signal value as a result. This shift of the measuring range allows various distance interpretations to be made in different parts of a distance range. For this purpose, the left-hand part of the signal region 92 is interpreted differently than the right-hand part, and a falling edge 108 of the value for the distance d results in the middle of the signal region 92. Thus, the correct measurement range 109 begins within the signal range 92.

FIG. 13 shows a formula used in the arithmetic unit 29 for determining the distance d from the signals SO, S1 and S2 with a measuring range shifted by the two signal ranges 90 and 91 and a part of the signal range 92. This formula indicates the distance interpretation shown in FIG. In this case, in comparison with the formula in FIG. 11, the condition of the first row has been recorded in addition to the already existing inequality S0 <= S2 <S1 with a logical AND and the further inequality S2-S0> = 0.5 * (S1-S0). By this further inequality only the right one becomes

In the same way, in the penultimate line, the inequality S2-S0 <0.5 * (S1-S0) complementary to it is taken into account, by which the left-hand part of the area 92 is taken into account and is provided with an offset value of "8". Here, a factor of "0.5" is used to obtain a division of the center area 92. Another factor between "0" and "1" can also be used to obtain a division at any other location, with values for the factor smaller than "0.5" are more on the left and with values greater than

"0.5" are more to the right of the area, which can be applied directly to other areas as well is that in the case shown in Fig. 12 (a) in the signal region 92, an increase of the signal S2, while the other signals SO and S2 are respectively constant. If such a division is made in a different region in which there is a falling signal instead, then the inequality signs should be reversed, and then values for the factor smaller than "0.5", the right side, and values corresponding to are greater than "0.5", the left side. In addition, the signals SO, Sl and S2 must be exchanged for each other, that results in a similar mathematical meaning.

FIG. 14 shows a signal course for the signals SO, S1 and S2, taking into account the law of distance and starting from the aforementioned explanations. Here is the waveform for the signals SO, Sl and S2 shown with increasing distance d. The spacing law superimposes a 1 / r ² function on the waveform of FIG. 10 (a). The signal moves between the background signal Sb and the saturation signal Ss. The latter arises from the fact that image sensors usually can store only a finite maximum signal in the pixel, which is referred to here as the saturation signal Ss.

This results in a distance range between a distance d of zero and the saturation distance ds, in which at least one of the signals SO, S1 or S2 is saturated. In this saturation range, no correct distance value can be determined. Where exactly the saturation distance ds lies depends, among other things, on the intensity of the light pulses, the reflection properties of the object and the time of the exposure. Since this saturation region can not be used to determine the distance d anyway, it is advantageous to be able to dispense with a use of this range and instead to be able to measure dmax up to a greater maximum distance. FIG. 15 shows a formula used in the calculation unit 29 for calculating the depth noise Nd, which was calculated using the formula according to FIG. 8. Here NSO is the noise of the signal SO, NS1 the noise of the signal Sl, and NS2 the noise of the signal S2 and tl the pulse length and c the speed of light. The operator max () denotes a maximum operator and min () a minimum operator.

For the derivation of the formula was proceeded to the basic idea of the known error propagation. An experimental verification of the results showed that, to obtain a correct result, one should not use the sum of the amounts of the partial derivatives of the different input quantities multiplied by the errors of the different input quantities, but the root of the sum of the squares.

The noise values NSO, NS1 and NS2 can either be measured or calculated. In the latter case, for example, the formulas of standard EMVA1288 release A3.0 can be used. There, the calculation of the temporal noise is determined from the dark noise of the image sensor and from the shot noise of the charge carriers and from the so-called conversion gain. If necessary, the quantization noise can still be considered.

Based on the formula of FIG. 15, it can be seen that the distance noise Nd obtained with the first embodiment is directly proportional to the pulse length t1. A simple way Therefore, to make a high-quality depth measurement with a low noise, it is to select the pulse length tl short. Since the length of the measuring range from "0" to dp or from dmin to dmax is in turn proportional to t1, the measuring range is shortened with a shortening of the pulse length t1. Since, as mentioned above, the measuring range in the first exemplary embodiment is particularly In addition, the distance range can be optimally adjusted by the possibility of shortening the pulse length t1 and thus reducing the distance noise Nd, as explained in FIGS. 10 to 13.

FIG. 16 shows a formula used in the arithmetic unit 29 for determining the validity of range values calculated from the signals SO, S1 and S2.

The knowledge of the noise can be used to decide on the validity of distance values d calculated from signals SO, S1 and S2. In the formula according to FIG. 16, SO, S1 and S2 are the already explained signals as digital values, o is the offset value of the camera as digital value, Nd is the dark noise of the camera, which can be determined according to EMVA1288 and given in electrons e- K, is the conversion gain according to EMVA 1288 in DN / e-, tl the pulse length in a unit of time, c the speed of light in a unit of length per time and Lim a freely selectable parameter for setting a limit value for the depth noise in units of length, eg Meter.

The formula of FIG. 16 is constructed of terms 210 to 213. Term 210 provides a limit value, Term 211 a proportionality factor, Term 212 a measure of the signal quality in digital Numbers and Terra 213 an estimate for the noise. Thus, the formula can be explained as follows: If the signal quality is higher than the noise by an adjustable limit value, the corresponding measured value is valid. This makes it possible to differentiate between valid and invalid measured values.

For the derivation of the formula, a simplification explained below in connection with FIG. 17 was used.

Another criterion for the validity of measured values is that all three signals SO, S1 and S2 are not saturated, that is to say are smaller than the saturation signal Ss.

Fig. 17 shows a diagram with an approximation of the noise by means of a simplified approximation function. Here, the depth noise Nd over the distance d is shown as graph 160. This course was calculated from the formula according to FIG. 15. Since the resulting formula is complex and leads to a high computational effort in practical use, an alternative approximation graph 161 with an approximate formula was calculated. This approximation formula results if instead of the three noise values NSO, NS1 and NS2 only one noise value is used for the mean value of the signals SO, S1 and S2. This approach is based on the formula according to FIG. 16.

Fig. 18 shows a time course of an ideal and a real light pulse. The ideal light pulse 140 has a steep turn-on edge 141 in which the intensity rises from "0" to "1" and a steep turn-off edge 142, in which the intensity drops from "1" to "0", without any relevant increase - or fall time would be needed. In contrast, the real light pulse 143 has a steady turn-on edge 144 and a steady turn-off edge 145 and thus does not generate one ideal temporal intensity curve, which differs significantly from the curve of the ideal light pulse 140. This deviation leads to deviations of the real course of the signals SO, S1 and S2 as a function of the time difference td from the signal profile shown in FIG. 7 (a), for example, and consequently also from the distance d determined therefrom, which in FIG. 7 (b) is shown.

FIG. 19 shows a curve of a real measured distance d as a function of the time difference td. Such a real, determined by measurement course of the measured distance d has compared to the ideal proportional increase in the range 100 in Fig. 7 (b) non-linear deviations. However, the measured distance can be linearized by means of a nonlinear correction function. This can be done, for example, in analytical form using an analytic function f (d) whose input variable is the measured distance d and which as a result yields a linearized distance f. Such a function can be determined, for example, by mathematical modeling by means of a periodic function in which the selectable parameters are adjusted by Fit. The term "fit" refers to an iterative approximation method.

Furthermore, it is also possible to use a numerical correction function which associates the measured distance d with a correct, linearized distance. For this purpose, for example, the distance d can be measured for a plurality of known locations and a memory table can be created from the known correct distances of these locations and the measured distance. On the basis of this table, the arithmetic unit 29 in FIG. 3 can then read out a correct distance value for a measured value of a distance d. Also In this reading, an interpolation between two measuring points can be carried out, wherein the interpolation can be carried out, for example, linear, quadratic, cubic or with an even higher order. Such a non-linear correction function can be used to correct measurement errors that result from deviations of the pulse shape of a real light pulse from the pulse shape of an ideal light pulse.

Since the signals SO, S1 and S2 run cyclically, it is fundamentally possible to modify the timing shown in FIG. 5 to operate with a different value of phi0. If this is the case, the signals SO, S1 and S2 are obtained in a different phase position than those shown in FIG. The formula according to FIG. 8 can then be adapted to this new phase position in the manner shown by way of example in FIGS. 11 and 13, in which case the adaptation does not take place to a new minimum distance dmin but to a new phase position phiO. Similarly, a new phase position can be combined with a new minimum distance. Fig. 20 shows waveforms for a timing according to a second embodiment.

This timing controller has the same pulse length t1, the same opening time ts and the same phase shifts phil and phi2 as the timing shown in Fig. 5, whereas the dead time to, the phase shift phiO and the period tp are changed.

In this case, the values for the phase shift are used here in such a way that their values in degrees still correspond to the original definition in the first exemplary embodiment in relation to refer to there period tp. Thus, in this second exemplary embodiment, the light pulses follow one another with the increased value for the period tp more than 360 ° apart, while the duration of a light pulse continues to be 60 °, the shutter opening time still 180 °, and ph.il and phi2 still 120 ° ,

Figs. 21 (a) and (b) show waveforms of the signals SO, Sl and S2 resulting from the timing of the second embodiment. These signal waveforms have signal range 220 to 229. Compared to Fig. 7 (a), the signal areas 223 to 226 of Fig. 21 (a) qualitatively correspond to the signal areas 90 to 93 of Fig. 7 (a), respectively, but shifted along the axis of time difference td. In addition, new signal areas 220, 221, 222, 227, 228 and 229 show that have a fundamentally different signal course.

If one now calculates a distance from these signals, one obtains the curve for the distance d shown in FIG. 21 (b). This curve has two areas 154 and 158 in which no distance can be calculated because the three signals SO, S1 and S2 are constant and identical there, so that on the basis of which small equation can be resolved after the distance. The curve also has two regions 155 and 157 with a piecewise constant value for the distance d, and a region 156 in which the value for the distance increases linearly with an increasing time difference td and in which the time difference td is correctly set in a value for the distance d can be implemented.

FIG. 22 shows a distance calculation formula used in the arithmetic unit 29 for the second embodiment. The formula is largely identical to that given in FIG. 8, but an additional offset value dmin has been added from the value of phiO, which differs significantly from that in FIG. 5 in the illustration of FIG. Furthermore, in FIG. 22, as compared with FIG. 5, a term is omitted for the case S1 <S2 <= S0 and for the case S1 <= S0 <S2, since these cases do not occur in FIG. 21 (a).

The second embodiment has the following advantages over the first embodiment. Since the period tp in FIG. 20 is significantly greater than in FIG. 5, a periodicity of the value of the distance d, as shown for example in FIG. 7, occurs only at significantly higher values for the time difference tp and thus only in significant greater distances between the camera 20 and the objects 25 and 26 of FIG. In practice, it is even possible to set the value for the period tp to be so great that practically no depth aliasing occurs because it is shifted to such great time differences td with such great distances that due to the law of distance from this distance practically no measurable light returns to the camera 20 or is so much weakened that the values determined therefrom can be recognized as invalid, for example by using the formula shown in FIG.

Furthermore, the design of the profile with the regions 154, 155, 157 and 158 in FIG. 21 (b) achieves a uniqueness of the distance values determined in the linear value range 156. That is, if one of the values for the distance is measured from the range of 156, it can be ensured that this value for the distance is also correct. If a value from the value range 155 or 157 is measured, it can be seen that the measured value lies outside the linear value range for the distance 156 and it is even recognizable whether it lies in the value range 155 before or in the value range 157 behind the linear value range 156. A disadvantage, however, is that in the second embodiment, the linear value range with the correctly determined distance d, which includes the four signal areas 223 to 226, is shorter than in the first execution area, where it includes the six signal areas 90 to 95.

Figures 23 (a) and (b) show peak and average waveforms of a pulse frequency modulated signal at different pulse repetition rates. From these waveforms, a further advantage of the first embodiment over the prior art and the second embodiment over the first embodiment can be removed. There, the relation between peak and mean value is shown for a pulse width or pulse frequency modulated signal. The peak value Pmax and the mean value Pmean of such a signal are linked via the pulse-duty ratio (duty cycle). With a pulse-pause ratio of 50% as in Fig. 23 (a), the average value is 50% of the peak value, with a pulse-pause ratio of 1/6 as in Fig. 23 (b), the average value is 1 / 6 of the peak.

Light sources for ToF cameras usually work with light sources that emit invisible wavelengths to the human eye from the near infrared range (NIR). Optionally, coherent or non-coherent radiation is used. In order to exclude a hazard to the human eye, the usable power is limited in each case, for example by the requirements set out in Directive 2006/25 / EC. In the usual time range of the light pulses used for ToF cameras, usually several nanoseconds, the mean value is the decisive factor. That is, the light source 23 must not exceed a certain average power. On the other hand, on the other hand, it is desirable to emit the highest possible light output so that the light emitted by the light source 23 is brighter or at least of the same order of magnitude as the ambient light. On the other hand, if the light pulses have significantly less intensity on the objects than the ambient light, the noise present in the environment due to the photon statistics results in a poor to very poor signal-to-noise ratio, so that the distance d is no longer determined with sufficient accuracy can.

On the other hand, since the limits are primarily related to the average power, and the ratio to the ambient to the peak power, the performance of a ToF camera can be increased by choosing a low duty cycle. As a result, a high peak value and a low mean value are achieved in the light intensity at the same time, as a result of which both of the aforementioned requirements can be implemented jointly in an advantageous manner.

As a result of the second exemplary embodiment, a particularly high peak value for the intensity can be achieved in comparison with the first exemplary embodiment, with a mean value that is particularly low at the same time. In particular, this embodiment is suitable for the use of so-called pulse laser diodes, which are usually optimized for a pulse-duty ratio of 1/1000. This is a great advantage in particular with respect to all ToF cameras with demodulation image sensors. The advantage comes especially where there is a high intensity of the ambient light, for example in strong sunlight or strong studio headlights. Fig. 24 shows waveforms for a timing according to a third embodiment. This is a modification of the first exemplary embodiment according to FIG. 5 ff in that the duration of the light pulses is extended from 60 ° to 180 °, while in return the pulses of the control signals CO, C1 and C2 are changed from 180 ° to 60 ° ° are shortened and phiO is set to 0 °. The phase shifts phil and phi2 continue to be 120 ° each. By applying this time control, an identical signal pattern is obtained as in the first exemplary embodiment in FIG. 7, so that the same formulas and measures with the same design options can be used to determine the distance d.

The advantage of the third embodiment is that a larger ratio between the pulse length t1 and the period tp can be realized. In order to realize a short measurement range with high accuracy with the first embodiment, it was necessary to select a short pulse length t1. However, it is the case that the customary pulsable light sources, for example light-emitting diodes, laser diodes or even pulse laser diodes, have a finite switch-on time and a finite switch-off time, so that their pulse length can not be reduced below a certain level. Thus, with the first exemplary embodiment, a certain minimum size of a measuring range can not be undershot, and thus only a limited measuring accuracy can be achieved. However, the third embodiment allows to further reduce the measuring range by a factor of three while also achieving a further increase in measurement accuracy.

FIG. 25 shows a formula used in the arithmetic unit 29 for calculating the depth noise Nd in the third embodiment. In comparison with FIG. 15, it can be seen that the first factor in the denominator has a "6" instead of a "2" and thus the noise is basically smaller by a factor of "3." The actually achievable reduction of the noise falls with the same exposure However, time is somewhat smaller, since only a signal portion of the light can be received by the short opening times of the shutter in comparison with the light pulses, and thus the signals SO, S1 and S2 are smaller than in the first exemplary embodiment - Searing effect that the shorter opening time, even a smaller part of the ambient light is received.

FIG. 26 shows a formula used in the arithmetic unit 29 for determining the validity of distance values obtained from the signals SO, S1 and S2 of the third embodiment. Like the formula in Fig. 15, this formula is derived from the formula for the depth noise Nd using a simplification. The different factors in the formula of the depth noise, namely the value "6" in Fig. 25 and the value "2" in Fig. 15, give a different factor in the counter of the term 171 in Fig. 26, namely, "108" instead of "12 ". With this formula, the validity of measured values can also be determined for the third exemplary embodiment. Of course, just as in the first embodiment, the saturation can be used as a criterion. FIG. 27 shows waveforms for a timing according to a fourth embodiment. FIG.

Just as the second embodiment can be derived from the first by increasing the period tp, can be derived by the same measure, the fourth of the third. The same signal forms result as in the second exemplary embodiment, so that the same formulas and measures can also be used for the fourth exemplary embodiment. The fourth embodiment has the same advantages over the third, as the second compared to the first.

Fig. 28 shows waveforms for a timing according to a fifth embodiment.

Here, both the pulse length tl and the opening times of the control signals CO, C1 and C2 are each 180 °, phiO is 0 ° and phil and phi2 each 120 °.

In the third and fourth embodiments, a particularly short opening time of the shutter was used to achieve a high measurement accuracy with a low noise level. However, there are cameras with image sensors in which this particularly short opening time of the shutter can not be realized. There, it is advantageous to be able to work with a longer opening time of the closure. In the fifth embodiment, as compared with the third and fourth embodiments, the opening time of the shutter is extended by a factor of "3." As a result, a larger portion of the emitted light can also be received, resulting in further improvement of the measurement accuracy.

Fig. 29 shows waveforms of the signals SO, Sl and S2 for the timing according to the fifth embodiment. The course of the signals results from the timing according to FIG. 28 with the considerations mentioned in the discussion of FIG. 6 and differs significantly from that in FIG. 7. Figs. 30 (a) and (b) show the trajectory of the S vectors in S space for the fifth embodiment from different angles.

The fifth embodiment is based on the idea of guiding the signal course in S-space on a polygonal line which comes as close as possible to a circular path around the unit diagonal. An S-space having such a polygon line consisting of lines 130 to 135 is shown in Figs. 30 (a) and (b). In this case, FIGS. 30 (a) and (b) show the same polygonal line for a better understanding of the three-dimensional course from a viewing angle rotated by 90 ° relative to one another. The lines connect points on the outer surfaces of the cube, which represents the S-space. As in FIG. 9, in this illustration it is assumed that no ambient light is present. FIG. 31 shows a formula used in the arithmetic unit 29 for determining the distance d from the signals SO, S1 and S2 for the fifth embodiment.

Up to a factor of 1/3, a match with the formula of Fig. 8 for the first embodiment. Thus, taking into account this other factor, all further insights and execution options for the first exemplary embodiment, for example also according to FIGS. 10 to 13, can be transferred to the fifth exemplary embodiment.

FIG. 32 shows a graph of a real measured distance d as a function of the time difference td for the fifth exemplary embodiment.

Analogous to FIG. 19, FIG. 32 shows a course of the measured distance d determined by measurement as a function of the time difference td for the fifth exemplary embodiment. This occurs particular advantage of the fifth embodiment. The nonlinear deviations are particularly small here. This is due to the fact that the effect of the deviations of the real switch-on edge on the ideal switch-off edge and the real switch-off edge on the ideal switch-off edge compensate each other in a particularly favorable manner. The remaining non-linear deviation can be corrected by the same means as proposed for the first embodiment. The noise for the fifth embodiment may be determined by the formula of FIG. 25. Since more light can be received by the longer opening time of the shutter than in the third embodiment, the denominator is generally larger, so that the noise tends to be smaller as a result.

Since the same formula can be applied to the noise as in the third embodiment, the validity of range values can also be determined in the same way as set forth in the third embodiment in FIG. 26. Fig. 33 shows signals of a timing according to a sixth embodiment.

As in the second and fourth embodiments, the period length was also extended here compared to the fifth embodiment. The same or similar advantages arise compared with the fifth exemplary embodiment, as in the second in comparison to the first and in the fourth in comparison with the third embodiment. FIG. 34 shows a profile of the signals S0, S1 and S2 corresponding to the illustration in FIG. 12 at a time control according to the sixth exemplary embodiment.

This results in a region 230 in which the signals S1 and S2 are equal and an exact determination of a distance d is not possible. However, where S0> S1, S0> S2 and S1 = S2 close, the distance is smaller than in the subsequent areas 231 to 237. In the areas 231 to 236, determination of the distance is possible. In contrast, in the region 237, again, it can not be determined, but it can be determined here from the relation S2> S1, S2> S0 and S0 = S1 that the distance d is greater than in the regions 231 to 236. In the region 238, because of S0 = S1 = S2 does not determine a distance and at the same time it is possible to make the statement that no distance can be determined there. From region 239 follows a periodic repetition of region 230.

FIG. 35 shows a formula used in the arithmetic unit 29 for determining the distance d in the sixth embodiment.

This formula indicates how a correct distance d can be determined from the signals shown in FIG. Compared to the formula in FIG. 31, significant differences are evident both in the domain-defining conditions and in the linear terms which are valid within the ranges, which are due to the signal curve from FIG. 34. It can be seen that with the sixth embodiment, in comparison to the second and fourth embodiments, a particularly large measuring range can be covered with particularly good linearity, without losing the advantages listed there.

Fig. 36 shows waveforms of a timing according to a seventh exemplary embodiment. Here, the pulse length and the length of the opening times of the shutter are each 120 °, phil and phi2 also 120 ° and phiO is chosen here with 0 °.

FIG. 37 shows the course of the signals SO, S1 and S2 at a timing according to the seventh embodiment.

From the time control according to FIG. 36, the signal profile shown in FIG. 37 results for the seventh exemplary embodiment on the basis of the considerations discussed with reference to FIG. In contrast to the previous exemplary embodiments, only three different signal regions 200, 201 and 202 result here. The fourth signal region 203 already represents a periodic repetition of the first signal region 200. The smaller number of regions reduces the number of case distinctions and thus also the number of regions Calculation effort in the arithmetic unit 29 of FIG. 3 for determining the distance.

Fig. 38 (a) and (b) show a trajectory of S vectors in S space for the seventh embodiment from different angles.

The seventh exemplary embodiment is based on the idea of guiding the signal course in S-space on a polygonal line which has a trajectory curve that can be described mathematically as simply as possible. Such a trajectory forms, for example, a triangle which, together with the basis vectors of the S-space, spans a triangular pyramid. This is shown in Figs. 38 (a) and (b), and lines 190, 191 and 192 form the triangle. Fig. 38 (a) and (b) both show for better spatial understanding exactly the same polygonal line from a rotated by 90 ° to each other angle. As in FIG. 9 and in FIG. 30, it is assumed in this illustration that no ambient light is present. 39 shows a formula used in the arithmetic unit 29 for determining the distance d from the signals SO, S1 and S2 at a timing according to the seventh embodiment.

Compared to FIGS. 8, 11, 13, 22 and 31, it is noticeable that this formula is structured in a simpler way, and in particular in the realization of the hardware of the arithmetic unit 29 according to FIG. 3, for example in a field -Programmable Gated Array (FPGA), a particularly low resource consumption required. 40 shows a formula used in the arithmetic unit 29 for calculating the depth noise Nd from the signals SO, S1 and S2 for the seventh embodiment.

Here, too, the parameters NS0, NS1 and NS2 characterize the noise values of the signals SO, S1 and S2, which can be measured directly or also calculated according to the formulas of the EMVA 1288 standard. From this formula, a criterion for the validity of measured values can be derived with the same considerations that were discussed in connection with FIGS. 16 and 17.

FIG. 41 shows waveforms for a timing according to an eighth embodiment. FIG.

Here, too, the seventh embodiment is further developed by increasing the period tp to obtain the advantages of the second, fourth, and sixth embodiments. Fig. 42 shows waveforms of the signals SO, S1 and S2 at a timing according to the eighth embodiment. When the timing shown in Fig. 41 is applied, the waveform shown in Fig. 42 is obtained in accordance with the time difference td. In this case, the distance d within the regions 241 and 242 can be determined correctly, can be determined within the regions 240 and 243 as being in front of or behind the measuring region, and can not be determined in the region 244. From region 245, a periodic repetition occurs. For the linearly extending signals within the ranges, a formula can be formulated as in the previous examples.

FIG. 43 shows such a formula used in the arithmetic unit 29 for the eighth embodiment.

As in the seventh embodiment, the formula is also very compact and requires little resource expenditure when implemented in hardware.

The described embodiments have in common that as image sensor 28 in Fig. 3 now a conventional image sensor can be used, which has only one signal path per pixel. As a result, the image sensor 28 can be produced in a particularly simple and therefore cost-effective manner and is sensitive to light or low in noise. Furthermore, it can have a particularly high lateral resolution or can be read out particularly quickly. Since considerably more conventional image sensors are available on the market than demodulation image sensors, it is easy to find a corresponding image sensor which has the corresponding advantage. Finally, a color image sensor can be used, so that the distance and color of objects can be detected with only a single image sensor.

In the aforementioned embodiments, in each case three signals were detected, for the reception of which in each case light was emitted and a shutter with a given and respectively different phase position to the emitted light is opened and closed. The duration ts of the opening of the shutter for the recovery of the three signals were each chosen equal. In each case phase shifts phi2 = phi3 were chosen. The presented alternative solutions according to the exemplary embodiments can be used in connection with a wide variety of image recording devices and cameras according to the TOF principle.

In summary, an apparatus and a method for controlling a time of flight camera have been described in which a distance information for a three-dimensional image from a transit time difference or a phase shift between a light emitted by a light source of the runtime camera light signal and one of an image sensor of the runtime camera by scattering or reflection of the emitted light signal light signal is detected. At least three subimages are recorded. In this case, at least one light pulse is emitted and a shutter (shutter) is clocked by means of at least three control signals so that the phase shift between the light pulse and shutter is varied by the different phase angles of the control signals between the subimages. This gives three readings per pixel. These can be supplied to an operator, which is constructed analogously to a Hue operator and determines a rotation angle about the spatial diagonal of the SO-S1-S2 space. By means of a proportionality constant this can then be converted into the distance information.

Claims

Claims: 1. A device for controlling a light transit time camera (20), in which a distance information for a three-dimensional image of a transit time difference or a phase shift between one of a light source (23) of the runtime camera (20) emitted light signal and one of an image sensor (28) of the runtime camera (20) is detected by scattering or reflection of the emitted light signal received light signal, wherein the device (21) is configured: - For controlling the light source (23) for emitting at least one light pulse; - For controlling the image sensor (28) for demodulating the received light signal by means of each image point of the image sensor associated electrical closure device; and - To open or close the electrical shutter means by means of a control signal having at least three different pulse-shaped waveforms (CO, C1, C2) with a fixed phase relationship to the at least one light pulse to detect at least three signals (SO, Sl, S2); - Wherein the time duration (tl) of the pulse-shaped control signals (Cl, C2, C3) for the opening of the electrical closure device is the same in each case; and - wherein the phase shift (phil) between the first control signal (CO) and the second control signal (Cl) is equal to the phase shift (phi2) between the second control signal (Cl) and the third control signal (C2), - Wherein the at least three signals (SO, Sl, S2) are each formed by accumulation of the signals obtained when opening the electric shutter device according to the respective pulse-shaped waveform (CO, Cl, C2). 2. Device according to claim 1, wherein the device (21) is designed such that the phase shift (phil) between the first control signal (CO) and the second control signal (Cl) and the phase shift (phi2) between the second control signal (Cl) and the third control signal (C2) is 120 ° with respect to the period of the light pulse. 3. Device according to claim 2, wherein the device (21) is designed such that the phase shift (phiO) between the time of opening of the electrical closing device by the first (CO) of the three control signals (Cl, C2, C3) and the time the output of the light pulse with respect to the period of the light pulse is -60 °. 4. The device according to claim 2, wherein the device (21) is configured such that the phase shift (phiO) between the time of opening the electrical closing device by the first (CO) of the three control signals (Cl, C2, C3) and the time the output of the light pulse with respect to the period of the light pulse is 0 °. 5. The apparatus of claim 4, wherein the device (21) is configured such that the time duration (tl) of the pulse-shaped control signals (Cl, C2, C3) for the opening of the electrical closure device with respect to the period of the light pulse a phase angle of 60 ° and that the duration of the light pulse with respect to the period of the light pulse corresponds to a phase angle of 180 °. 6. Apparatus according to claim 4, wherein the device (21) is designed such that the time duration (tl) of the pulse-shaped control signals (Cl, C2, C3) for the opening of the electrical closure device with respect to the period of the light pulse a phase angle of 180 ° and that the duration of the light pulse with respect to the period of the light pulse corresponds to a phase angle of 180 °. 7. The apparatus of claim 4, wherein the device (21) is configured such that the time duration (tl) of the pulse-shaped control signals (Cl, C2, C3) for the opening of the electrical closure device with respect to the period of the light pulse a phase angle of 120 ° and that the duration of the light pulse with respect to the period of the light pulse corresponds to a phase angle of 120 °. 8. The apparatus of claim 3, wherein the device (21) is designed such that the time duration (tl) of the pulse-shaped control signals (Cl, C2, C3) for the opening of the electrical closure device with respect to the period of the light pulse a phase angle of 180 ° and that the duration of the light pulse in With respect to the period of the light pulse corresponds to a phase angle of 60 °. 9. light cycle camera (20) with a light source (23), an image sensor (28) and a device (21) according to any one of claims 1 to 8. 10. A time of flight camera according to claim 9, comprising computing means (29) for calculating the distance information of the three-dimensional image from the values of the three signals (SO, Sl, S2) based on a case discrimination for different ones by mutual size ratios of the values of the three signals ( SO, Sl, S3) fixed areas (90-97). 11. The time of flight camera according to claim 10, wherein the calculating means (29) is configured to determine the validity of the calculated distance information based on a relationship between signal quality and noise. 12. The time of flight camera according to claim 1, wherein the computing device is configured to decide on the validity of the calculated distance information on the basis of the degree of saturation of the three signals (SO, S1, S2). A method for controlling a light runtime camera (20), the be a distance information for a dreidimensiona len image representation of a transit time difference or a phase shift between one of a light source (23) of the runtime camera (20) emitted light signal and one of an image sensor (28) of running time camera (20) is detected by scattering or reflection of the emitted light signal received light signal, the method comprising the following steps: - controlling the light source (23) to deliver at least one light pulse; - Controlling the image sensor (28) for demodulating the received light signal by means of an electrical closure device associated with each pixel of the image sensor; and - Opening and closing the electrical shutter at least three times with fixed phase relation to the at least one light pulse to detect at least three signals (SO, Sl, S2); - Wherein the time duration (tl) of the opening of the electrical closure device is the same in each case; - wherein the phase shift (phil) between the first opening and the second opening is equal to the phase shift (phi2) between the second opening and the third opening; and - Wherein the at least three signals (SO, Sl, S2) are respectively detected by accumulation of the signals obtained when opening the electric shutter device according to the respective pulse-shaped waveform (CO, Cl, C2). A computer program for storing on a volume and causing a computing device to execute The control method of claim 13, when the computer program is executed on the computing device