TWI696841B - Computation apparatus, sensing apparatus and processing method based on time of flight - Google Patents

Abstract

A computation apparatus, a sensing apparatus and a processing method based on time-of-flight (ToF) are provided. In the method, intensity information of at least two phases corresponding to at least one pixel is obtained. The intensity information is generated by sensing a modulation light with time delays using these phases. Whether to abandon the intensity information of the at least two phases of the pixel is determined according to the difference between the intensity information of the at least two phases. Accordingly, the influence caused by motion blur would be improved for depth information estimation.

Description

Computing device, sensing device and processing method based on time-of-flight ranging

The present invention relates to an optical measurement technology, and particularly relates to a computing device, a sensing device, and a processing method based on Time of Flight (ToF) distance measurement.

With the development of technology, optical three-dimensional measurement technology has gradually matured. Time-of-flight distance measurement is a common active depth sensing technology. The basic principle of ToF ranging technology is that modulated light (for example, infrared light, laser light, etc.) will be reflected after encountering an object after being emitted, and converted by the reflection time difference or phase difference of the reflected modulated light The distance of the object being shot can produce depth information relative to the object.

It is worth noting that, referring to the timing diagram of FIG. 1A, the time when the ToF ranging technology senses dimming is called the exposure time, which is similar to the camera shutter time. For example, logic 1 means exposure/sensing; logic 0 means stop exposure. Among them, when the exposure time increases, the amount of data received for dimming will also increase. However, compared Long exposure time may easily cause motion blur. For example, FIG. 1B shows the afterimage caused by the movement of the object to be measured, and FIG. 1C shows the light track caused by the movement of the vehicle lights. However, when using ToF ranging technology to calculate depth information, if it encounters dynamic blur, it will lead to inaccurate depth distance or blurry picture. Therefore, how to provide a simple and effective method to reduce the impact of dynamic blur has become one of the goals to be worked on in the related field.

In view of this, embodiments of the present invention provide a computing device, a sensing device, and a processing method based on time-of-flight ranging, which can effectively avoid invalid depth calculation caused by dynamic blur.

An operation device based on time-of-flight distance measurement according to an embodiment of the present invention includes a memory and a processor. The memory records the intensity information of at least two phases corresponding to at least one pixel and the program code corresponding to the processing method for the computing device, and the intensity information is obtained by using those phases to delay the modulation of dimming in time. The processor is coupled to the memory and is configured to execute the code. The processing method includes the following steps: obtaining intensity information of at least two phases. Based on the difference between the intensity information of those phases, it is decided whether to discard the intensity information of at least two phases corresponding to this pixel.

A sensing device based on time-of-flight distance measurement according to an embodiment of the present invention includes a dimming light transmitting circuit, a dimming light receiving circuit, a memory, and a processor. The dimming light emitting circuit emits dimming light. The dimming light receiving circuit uses at least two phases in time delay to receive the dimming light. The memory records at least two phases corresponding to at least one pixel Strength information and the corresponding code of the processing method used for the sensing device. The processor is coupled to the dimming receiving circuit and the memory, and is configured to execute the program code, and the processing method includes the following steps. Obtain the intensity information of at least two phases, and the intensity information is obtained by using the phases to delay the dimming in time. Based on the difference between the intensity information of those phases, it is decided whether to discard the intensity information of those phases corresponding to this pixel.

On the other hand, the processing method based on time-of-flight ranging in the embodiment of the present invention includes the following steps: obtaining intensity information of at least two phases corresponding to at least one pixel, and the intensity information uses those phases to delay in time Obtained by measuring dimming. Based on the difference between the intensity information of those phases, it is decided whether to discard the intensity information of those phases corresponding to this pixel.

Based on the above, the operation device, sensing device and processing method based on the time-of-flight distance measurement of the embodiments of the present invention evaluate whether dynamic blur occurs according to the difference between the intensity information of the two phases, and accordingly The pixels are discarded and re-shot or only valid pixels are used. In this way, the effect of motion blur on depth information estimation can be effectively reduced.

In order to make the above-mentioned features and advantages of the present invention more obvious and understandable, the embodiments are specifically described below and described in detail in conjunction with the accompanying drawings.

10: Ranging system

100: sensing device

110: Modulated light emitting circuit

120: Modulated light receiving circuit

122: Photoelectric sensor

130: processor

140: signal processing circuit

150: memory

160: computing device

170: attitude sensor

CA, CB: capacitance

QA, QB: the amount of charge changed

CS: control signal

CSB: Inverted control signal

DS: Sensing signal

EM: dimming

MS: Modulation signal

NA, NB: Node

REM: Modulated light reflected

SW1, SW2: switch

V _A , V _B : voltage signal

TA: target object

S410~S430, S710~S790, S810~S890, S910~S950: Steps

FIG. 1A is an example illustrating a timing diagram of exposure time and modulated light signal.

1B and 1C are two examples to illustrate the phenomenon of dynamic blur.

2 is a schematic diagram of a distance measuring system according to an embodiment of the invention.

FIG. 3A is a schematic circuit diagram of a modulated light receiving circuit according to an embodiment of the invention.

FIG. 3B is a signal waveform diagram according to the embodiment of FIG. 3A.

4 is a flowchart of a processing method based on time-of-flight ranging according to an embodiment of the present invention.

5A~5D are an example to illustrate the local motion blur phenomenon.

6A~6D are an example to illustrate the global dynamic blur phenomenon.

7 is a flowchart of a processing method based on time-of-flight ranging according to the first embodiment of the present invention.

8 is a flowchart of a processing method based on time-of-flight ranging according to a second embodiment of the present invention.

9 is a flowchart of a processing method based on time-of-flight ranging according to a third embodiment of the present invention.

10A and 10B are schematic diagrams illustrating an example of abandoning invalid data.

2 is a schematic diagram of a distance measuring system 10 according to an embodiment of the invention. Referring to FIG. 2, the ranging system 10 includes a ToF-based sensing device 100 and a target object TA.

The sensing device 100 includes but is not limited to the modulating light emitting circuit 110, the modulating The light receiving circuit 120, the processor 130, the signal processing circuit 140, the memory 150, and the attitude sensor 170. The sensing device 100 can be applied to fields such as three-dimensional model modeling, object recognition, vehicle auxiliary systems, positioning, production line testing, or error correction. The sensing device 100 may be a stand-alone device, or may be loaded on other devices through modularization, and is not intended to limit the scope of the present invention.

The modulated light emitting circuit 110 is, for example, a laser diode or a collimated light generating device, and the modulated light receiving circuit 120 is, for example, an imaging device or a light source sensing device (including at least a light sensor, a reading circuit, etc.). The signal processing circuit 140 is coupled to the dimming light transmitting circuit 110 and the dimming light receiving circuit 120. The signal processing circuit 140 is used to provide a modulation signal MS to the modulation light transmitting circuit 110 and a control signal CS to the modulation light receiving circuit 120. The modulating light emitting circuit 110 is used to emit the modulating light EM according to the modulating signal MS, and the modulating light EM is, for example, infrared light, laser light or collimated light in other wavelength bands. For example, the modulation signal MS is a pulse signal, and the rising edge of the modulation signal MS corresponds to the trigger time of the modulation light EM. The modulated light EM will be reflected after meeting the target object TA, and the modulated light receiving circuit 120 may receive the reflected modulated light REM. The modulated light receiving circuit 120 demodulates the reflected modulated light REM according to the control signal CS to generate a sensing signal DS.

More specifically, FIG. 3A is a circuit schematic diagram of the modulated light receiving circuit 120 according to an embodiment of the invention. Please refer to FIG. 3A. For the convenience of explanation, this drawing takes the unit/single pixel circuit as an example. The circuit corresponding to the unit/single pixel in the modulated light receiving circuit 120 includes a photoelectric sensor 122, a capacitor CA, a capacitor CB, a switch SW1 and a switch SW2. The photoelectric sensor 122 is, for example, a photodiode or other light-sensing device having a function similar to that used to sense the reflected modulated light REM. One end of the photoelectric sensor 122 receives a common reference voltage (eg, ground GND), and the other end thereof is coupled to one end of the switch SW1 and the switch SW2. The other end of the switch SW1 is coupled to the capacitor CA through the node NA and is controlled by the inverted signal CSB of the control signal CS. The other end of the switch SW2 is coupled to the capacitor CB through the node NB and is controlled by the control signal CS. Voltage on the output node NA 120 modulation light receiving circuit (or current) signal V _{A and the} voltage at the node NB (or current) signal V _B as a sensing signal DS. In another embodiment, the modulated light receiving circuit 120 can also select the difference between the output voltage signal V _A and the voltage signal V _B as the sensing signal DS (which can be used as intensity information).

The embodiment of FIG. 3A is for illustration only, and the circuit architecture of the modulated light receiving circuit 120 is not limited thereto. The modulated light receiving circuit 120 may have multiple photoelectric sensors 122, or more capacitors or switches. Those with common knowledge in this field can make appropriate adjustments based on the common knowledge and actual needs.

FIG. 3B is a signal waveform diagram according to the embodiment of FIG. 3A. 3A and 3B, when the inverting control signal CSB is at a low level (for example, logic 0), the switch SW1 is turned on, and the control signal CS will be at a high level (for example, logic 1), and the switch SW2 Not conductive. Conversely, when the control signal CS is at a low level (for example, logic 0), the switch SW2 is turned on. At this time, the inverted control signal CSB is at a high level (for example, logic 1), and the switch SW1 is not turned on. In addition, when the photoelectric sensor 122 is turned on, the photoelectric sensor 122 can receive the reflected modulated light REM. When both the photoelectric sensor 122 and the switch SW1 are turned on, the capacitor CA is discharged (or charged). QA in FIG. 3B represents the amount of charge changed by the capacitor CA, and the voltage signal V _A on the node NA changes accordingly. When both the photoelectric sensor 122 and the switch SW2 are turned on, the capacitor CB is discharged (or charged). QB in FIG. 3B represents the amount of charge changed by the capacitor CB, and the voltage signal V _B on the node NB changes accordingly.

The processor 130 is coupled to the dimming light receiving circuit 120. The processor 130 may be a central processing unit (Central Processing Unit, CPU), or other programmable general-purpose or special-purpose microprocessor (Microprocessor), digital signal processor (DSP), programmable Controller, application-specific integrated circuit (Application-Specific Integrated Circuit, ASIC) or other similar components or a combination of the above components. In the embodiment of the present invention, the processor 130 can calculate the phase difference between the control signal CS and the reflected modulated light REM according to the sensing signal DS, and perform distance measurement based on the phase difference. For example, referring to FIG. 3B, according to the difference between the voltage signal V _A and the voltage signal V _B , the processor 130 can calculate the phase difference between the control signal CS and the reflected modulated light REM. It should be noted that, in some embodiments, the processor 130 may be built-in or electrically connected with an analog-to-digital converter (Analogy-to-Digital, ADC), and convert the sensing signal DS through the analog-to-digital converter Signal in digital form.

The memory 150 is coupled to the processor 130. The memory 150 may be any type of fixed or removable random access memory (RAM), flash memory (Flash Memory), and traditional hard disk (Hard Disk) Drive (HDD), solid-state disk (Solid-State Disk, SSD), non-volatile (non-volatile) memory or similar components or a combination of the above components. In this embodiment, the memory 150 Used to store buffered or permanent data (for example, intensity information corresponding to the sensing signal DS, threshold, etc.), code, software modules, operating systems, applications, drivers, and other data or files, and their details To be detailed in subsequent embodiments. It is worth noting that the program code recorded in the memory 150 is used for the processing method of the sensing device 100, and the processing method will be described in detail in subsequent embodiments.

The attitude detector 170 is coupled to the processor 130, and the attitude detector 170 may be a gravity sensor (G-sensor)/accelerometer (Accelerometer), an inertial (Inertial) sensor, a gyroscope (Gyroscope), a magnetic sensor Magnetometer or its combination, and used to detect motion or posture such as acceleration, angular velocity, azimuth, etc., and generate posture information accordingly (for example, three-axis gravity acceleration, angular velocity or magnetic force etc. are recorded).

It should be noted that, in some embodiments, the processor 130 and the memory 150 may be separated into the computing device 160. The computing device 160 can be a device of a desktop computer, a notebook computer, a server, a smartphone, or a tablet computer. The computing device 160 and the sensing device 100 further have communication transceivers that can communicate with each other (for example, transceivers that support communication technologies such as Wi-Fi, Bluetooth, and Ethernet), so that the computing device 160 can obtain The sensing signal DS or the corresponding intensity information of the sensing device 100 (which can be recorded in the memory 140 for the processor 130 to access).

In order to facilitate understanding of the operation process of the embodiment of the present invention, a number of embodiments will be described in detail below to describe the operation process of the sensing device 100 and/or the computing device 160 in the embodiment of the present invention. Hereinafter, the methods described in the embodiments of the present invention will be described with various elements and modules in the sensing device 100 and the computing device 160. The various processes of the method can be adjusted according to the implementation situation, and it is not limited to this.

4 is a flowchart of a processing method based on time-of-flight ranging according to an embodiment of the present invention. Referring to FIG. 4, the processor 130 obtains intensity information of at least two phases corresponding to at least one pixel (step S410). Specifically, in the embodiment of FIG. 3B, the modulation signal MS and the control signal CS are synchronized, but the signal processing circuit 140 can also make the modulation signal MS and the control signal CS out of synchronization. That is to say, there may be a reference phase between the control signal CS and the modulation signal MS. The signal processing circuit 140 delays or advances the phase of the modulation signal MS or the control signal CS according to different reference phases, so that the modulation signal MS and the control signal CS have a phase difference/phase delay.

In the continuous wave (Continuous Wave, CW) measurement mechanism, the phase difference is, for example, 0 degrees, 90 degrees, 180 degrees, and 270 degrees, which is a four-phase method. Different phases correspond to different charge accumulation time intervals at different start and end time points. In other words, the modulated light receiving circuit 120 uses four phases in time delay to receive the reflected modulated light REM. By using those phases to delay sensing the reflected modulated light REM in time, the sensing signals DS corresponding to different phases can be obtained, and the sensing signals DS can be further used as intensity information. This intensity information may record the amount of charge accumulated by a single pixel (one pixel corresponds to the circuit of FIG. 3A) or may be further converted into an intensity value. That is, the intensity information of each pixel is obtained by using those phases to delay the reflected modulated light REM in time.

Then, the processor 130 determines whether to discard the intensity information of at least two phases corresponding to the pixel according to the difference between the intensity information of the at least two phases (step S430). Specifically, it can be known from experiments that the dynamic blur phenomenon will cause differences in intensity information between different phases. For example, Figures 5A~5D are an example to illustrate this domain (local) dynamic blur phenomenon. Please refer to FIG. 5A. The figure shows an image generated by the sensing signal DS when the target object TA (taking a chair as an example) and the sensing device 100 are in a stationary state (eg, no shaking, no jumping, etc.) Take the resolution of 240×180 as an example). Please refer to FIG. 5B, which shows an image generated by subtracting the intensity value corresponding to any two phases corresponding to the same time point (ie, the difference in intensity information). Please refer to FIG. 5C is the scaled image of FIG. 5B, and it can be observed that the intensity difference of all pixels is approximately the same and is equal to or close to zero. Next, assume that the target object TA moves. Please refer to FIG. 5D, which shows the image generated by the subtraction of the intensity values of any two phases corresponding to the same time point (ie, the difference in intensity information), and the intensity of some pixels can be observed compared to FIG. 5C big different.

6A~6D are an example to illustrate the global dynamic blur phenomenon. Please refer to FIG. 6A. The figure shows an image generated by the sensing signal DS when the target object TA (taking a chair as an example) and the sensing device 100 are at rest (taking a resolution of 240×180 as an example). Please refer to FIG. 6B, which shows an image generated by subtracting the intensity value corresponding to any two phases corresponding to the same time point (ie, the difference in intensity information). Please refer to FIG. 6C is the scaled image of FIG. 6B, and it can be observed that the intensity difference of all pixels is approximately the same and is equal to or close to zero. Next, assume that the sensing device 100 moves. Please refer to FIG. 6D, which shows the image generated by subtracting the intensity value corresponding to any two phases corresponding to the same time point (ie, the difference in intensity information). Compared with FIG. 6C, the intensity of some pixels can be observed. big different.

It can be seen that, no matter whether it is a dynamic blur phenomenon in the local domain or the global domain, the difference between the intensity information of the two phases will increase. Conversely, if there is no motion blur, then The difference in intensity information of the two phases will be equal to or approach zero. Therefore, the difference between the intensity information of the two phases can be used to assess whether dynamic blurring has occurred.

In one embodiment, for each pixel, the processor 130 can determine whether the difference between the intensity information of at least two phases is greater than the difference threshold, and if the difference is greater than the difference threshold, the processor 130 discards the pixel The intensity information of those corresponding phases. Specifically, inevitably, the difference in intensity information of the two phases may not be exactly equal to zero. Therefore, the embodiment of the present invention improves the tolerance, so that the processor 130 can be preset or set by the user with a different threshold (for example, 10, 20, or 40, etc.). If the difference is less than the difference threshold, the processor 130 can be regarded as that no dynamic blur phenomenon has occurred. Conversely, if the difference is greater than the difference threshold, the processor 130 can be directly regarded as the occurrence of dynamic blurring, or it needs to be further evaluated through other information. It is worth noting that the intensity information of pixels whose intensity value difference is greater than the difference threshold may affect the results of subsequent depth information estimation. Therefore, the embodiments of the present invention will discard the intensity information of any four phases that have a pixel below the difference threshold according to some conditions. If the processor 130 discards the intensity information of those phases, it will determine the intensity information of those phases corresponding to different time points using this pixel or the intensity information of different phases corresponding to other pixels that have not been discarded. If the intensity information of at least two phases corresponding to any pixel at the current time point is discarded, it needs to be sensed again through the modulation light receiving circuit 120 to reacquire the pixel at different time points (subsequent time points) Corresponding to the intensity information of those phases, and then evaluate whether to use these intensity information. Or, if only the intensity information of the phases corresponding to some pixels at the current time point is discarded, those pixels that are not discarded (or reserved pixels) correspond to different phases Strength information will be used. In addition, the processor 130 can calculate the depth information according to the last adopted intensity information

It should be noted that for any pixel, the processor 130 can compare the difference between any two phases (for example, 0 degrees and 180 degrees, 180 degrees and 270 degrees, etc.) with the difference threshold (the value of which needs to be adjusted accordingly) Compare. In other embodiments, the processor 130 may also select the values of the two phases with the largest difference to compare with the difference threshold (the value of which needs to be adjusted accordingly). Alternatively, the processor 130 may randomly select more phase intensity information for comparison. If more than two differences are obtained, they can be further averaged or compared with a specific linear combination to the difference threshold.

In the following, those abandon conditions and corresponding processing methods will be described in detail below: FIG. 7 is a flowchart of a processing method based on time-of-flight ranging according to the first embodiment of the present invention. Referring to FIG. 7, for each pixel, the processor 130 obtains intensity information of at least two phases (step S710), and determines whether the difference between the intensity information is greater than the difference threshold (step S730), and the detailed descriptions thereof can be separately Please refer to the description of steps S410 and S430, so it will not be repeated here. Then, if the difference is not greater than the difference threshold, the processor 130 may calculate depth information based on the intensity information of all phases corresponding to the pixel at the current time point (step S735). For example, the difference between 0 degrees and 180 degrees is taken as the real part, and the difference between 90 degrees and 270 degrees is taken as the imaginary part; the angle formed by the real part and the imaginary part can be taken as the phase difference φ, and the distance (that is, as depth information) is 1/2* c *φ/2π* f , where c is the constant of light speed and f is the sampling frequency.

On the other hand, if the difference is greater than the difference threshold, the processor 130 may discard/cancel/not use the intensity information of the four phases of the pixel at the current time point (step Step S780), that is, the intensity information in the time interval of the accumulated charge of the pixel is not used. The processor 130 may adjust the exposure time of the modulated light of which phase it detects when it is sensed through the modulated light receiving circuit 120 next time. Since shortening the exposure time can improve the motion blur phenomenon, the processor 130 can further notify the dimming light receiving circuit 120 to reduce the exposure time to re-shoot/sensing/receive the reflected dimming light REM (step S790), thereby obtaining the picture The intensity information of at least two phases obtained by responding to the modulated light REM at different time points.

為1g，則表示感測裝置100為靜止狀態且此差異是本域的動態模糊所造成(例如，目標物體TA移動)；若其值不為1g，則表示感測裝置100不為靜止狀態且此差異是全域的動態模糊所造成。需說明的是，依據姿態感測器170的類型，其判斷靜止狀態的條件可能不同，應用本發明實施例者可自行調整對應參數，非用以限制本發明的範疇。 8 is a flowchart of a processing method based on time-of-flight ranging according to a second embodiment of the present invention. Please refer to FIG. 8. For details of steps S810, S830, S835, S880 and S890, please refer to the description of steps S710, S730, S735, S780 and S790, which will not be repeated here. The difference from the first embodiment is that if the difference between the intensity information of the two phases is greater than the difference threshold, the processor 130 will determine the motion blur caused by the difference according to the pose information obtained by the pose sensor 170 Global or local (step S850). Taking the three-axis acceleration sensing values X _out , Y _out and Z _out as examples, if

Is 1g, it means that the sensing device 100 is at rest and this difference is caused by the motion blur in this domain (for example, the target object TA moves); if its value is not 1g, it means that the sensing device 100 is not at rest and This difference is caused by global motion blur. It should be noted that, according to the type of the posture sensor 170, the conditions for determining the static state may be different. Those applying the embodiments of the present invention may adjust the corresponding parameters by themselves, not to limit the scope of the present invention.

If it is judged as global motion blur, the processor 130 will give up directly The intensity information of the phases corresponding to the pixel at the current time point is captured by the modulation light receiving circuit 120 to regain the intensity information of at least two phases in the next charge accumulation time interval of the pixel (step S855). On the other hand, if it is determined to be motion blur in this domain, the processor 130 can further determine whether to regain at least two phases corresponding to this pixel at different time points through the dimming light receiving circuit 120 according to the number of blurred pixels Strength information. The number of blurred pixels is the amount accumulated in response to a pixel judged to have motion blur. In other words, if the difference between the intensity information corresponding to a certain pixel is greater than the difference threshold, the number of blurred pixels is accumulated, and the final number of blurred pixels can be obtained after evaluating all pixels.

It is worth noting that, according to experiments, different thresholds for differences will correspond to the number of different blur pixels. In other words, for different difference thresholds, the proportion of pixels that are judged to have motion blur in a sensing image may be different in all pixels. Taking 240×180 resolution as an example, Table (1) is the number and proportion of blurred pixels corresponding to different thresholds:

The number of fuzzy pixels obtained in the experiment in Table (1) can be used as a threshold value for comparison, but this threshold value may be adjusted according to different thresholds, different resolutions, or other conditions. The embodiments of the present invention do not Be restricted. Processor 130 may It is determined whether the number of blurred pixels obtained at the current time point (or within the sampling interval) is greater than the set threshold value (step S870). If the number of blurred pixels is greater than the set threshold value, the processor 130 can discard the intensity information of the phase corresponding to the pixel at the current time point (step S880), and shoot through the modulated light receiving circuit 120 to Retrieve the intensity information of at least two phases corresponding to different time points (for example, the next sampling time point or subsequent sampling interval) of the pixel (step S890). Conversely, if the number of blurred pixels obtained in the current time interval is not greater than the set threshold value, the processor 130 directly calculates the depth information based on the intensity information of the plurality of phases obtained at the current time point of the pixel (step S835) , That is, retain the intensity information corresponding to this pixel.

In one embodiment, the length of the exposure time adjusted in step S890 may be related to the difference between the number of blurred pixels and the threshold value. For example, the adjusted exposure time can be obtained according to formulas (1) and (2):

其中，exposure_time'為調整的曝光時間，exposure_time為原曝光時間，blur_pixels為模糊畫素數量，且threshold為數量門檻值。

Where, exposure_time' is the adjusted exposure time, exposure_time is the original exposure time, blur_pixels is the number of blur pixels, and threshold is the number threshold.

It should be noted that, in other embodiments, the modulated light receiving circuit 120 can also directly reduce the exposure time of a specific length or a random length.

9 is a flowchart of a processing method based on time-of-flight ranging according to a third embodiment of the present invention. Please refer to FIG. 9. For details of steps S910, S930, S935 and S880, please refer to the description of steps S710, S730, S735 and S780. Repeat. The difference from the first embodiment is that if the difference between the intensity information of at least two phases of any pixel is greater than the difference threshold, the processor 130 may only discard/cancel/not use the pixel at the current time The intensity information of the phases corresponding to the points (step 950), and the discarded pixel is judged to have motion blur (for example, the intensity difference between the two phases of this pixel is greater than the difference threshold). The processor 130 may record the position, index, or code of the discarded pixels in the sensing image. In step S935, the processor 130 will calculate the depth information according to the intensity information of the pixels that have not been discarded. It should be noted that the pixels remaining after excluding abandoned pixels among all pixels are undiscarded pixels. Excluding the intensity information of the discarded pixels can reduce the motion blur image, and the intensity information of the pixels that are not affected by the motion blur can still be used for subsequent depth information calculation. In this way, multiple re-shooting can be avoided and the efficiency can be improved.

10A and 10B are schematic diagrams illustrating an example of abandoning invalid data. Please refer to FIG. 10A first. The picture shows an image in which all pixels are retained. 10B, assuming that the difference threshold is 40, pixels with a difference exceeding 40 will be discarded, and the processor 130 may set the intensity of these discarded pixels to zero or neglect.

It should be noted that the steps in the foregoing three embodiments can be interchanged, added, or changed according to actual needs. For example, in step 730 of the first embodiment, the mechanism for determining the number of blurred pixels in step 870 is further increased.

In summary, the calculation device, the sensing device and the processing method based on the time-of-flight ranging in the embodiments of the present invention can be judged based on the difference between the intensity information of any two phases, the number of blurred pixels, the posture information or a combination thereof Whether the motion blur phenomenon occurs. If After evaluating the phenomenon of motion blur, you can re-shoot or discard the intensity information of some pixels with motion blur. In this way, the influence of the motion blur phenomenon on the subsequent depth information estimation can be reduced in a simple way.

Although the present invention has been disclosed as above by the embodiments, it is not intended to limit the present invention. Any person with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be subject to the scope defined in the appended patent application.

S410~S430: Steps

Claims (13)

A computing device based on time-of-flight ranging includes: a memory that records intensity information corresponding to at least two phases of at least one pixel and a program code corresponding to a processing method for the computing device, wherein the intensity information is utilized The at least two phases are delayed in time by sensing a dimming light; and a processor coupled to the memory and configured to execute the program code, the processing method includes: obtaining intensity information of the at least two phases And according to the difference between the intensity information of the at least two phases, determine whether to discard the intensity information corresponding to the at least two phases of the pixel, wherein the step of determining whether to discard the intensity information corresponding to the at least two phases of the pixel includes : Determine whether the difference is greater than a difference threshold; respond to the difference greater than the difference threshold, discard the intensity information corresponding to at least two phases of the pixel; respond to discard the intensity information corresponding to at least two phases of the pixel To determine whether to regain the intensity information of at least two phases corresponding to the pixel at different time points; and to calculate depth information based on the intensity information of different phases corresponding to the pixel. The computing device based on time-of-flight ranging as described in item 1 of the scope of the patent application, wherein the processing method further includes: adaptively adjusting an exposure time for detecting the at least two phases of modulated light to restart Obtain intensity information of at least two phases corresponding to the pixel at different time points. The computing device based on time-of-flight ranging as described in item 1 of the patent scope, wherein the memory further records a posture information corresponding to the device for sensing the at least two-phase dimming light, and the processing The method further includes: in response to the difference being greater than the difference threshold, judging from the posture information that the motion blur caused by the difference is global or local; in response to the global motion blur, discarding the pixel The intensity information of at least two phases should be obtained, and the intensity information of at least two phases corresponding to the pixel at different time points should be re-acquired; and the dynamic blur in the local area can be used to decide whether to re-acquire the image according to the number of fuzzy pixels Intensity information corresponding to at least two phases corresponding to different time points, wherein the number of blurred pixels is an amount that is accumulated in response to the pixel being judged to have motion blur. An arithmetic device based on time-of-flight ranging as described in item 3 of the patent application scope, wherein the processing method further includes: abandoning the intensity corresponding to at least two phases of the pixel in response to the number of blurred pixels being greater than a threshold value Information, and regain the intensity information of at least two phases corresponding to the pixel at different time points. The computing device based on time-of-flight ranging as described in item 1 of the patent application scope, wherein the processing method further comprises: in response to the difference being greater than the difference threshold, discarding at least one intensity information corresponding to different phases of the pixel, wherein The abandoned at least one pixel is judged to be dynamic Blur; and calculate the depth information according to the intensity information of different phases corresponding to the pixels that have not been abandoned. A sensing device based on time-of-flight ranging includes: a dimming transmitting circuit that emits a dimming light; a dimming receiving circuit that uses at least two phases to receive the dimming light on a time delay; a memory Body, recording the intensity information of at least two phases corresponding to at least one pixel and the program code corresponding to the processing method for the sensing device; and a processor coupled to the dimming light receiving circuit and the memory, and Configured to execute the program code, the processing method includes: obtaining intensity information of the at least two phases, wherein the intensity information is obtained by using the at least two phases to delay sense the dimming in time; and based on the at least two phases The difference between the phase intensity information determines whether to discard the intensity information corresponding to at least two phases of the pixel, and the operation to determine whether to discard the intensity information corresponding to at least two phases of the pixel includes: determining whether the difference is greater than A difference threshold; in response to the difference being greater than the difference threshold, the intensity information corresponding to at least two phases of the pixel is discarded; in response to discarding the intensity information corresponding to at least two phases of the pixel, it is judged whether to reacquire the Intensity information of at least two phases corresponding to pixels at different time points; and A depth information is calculated according to the intensity information of different phases corresponding to the pixels. The sensing device based on time-of-flight ranging as described in item 6 of the patent application scope, wherein the processing method further comprises: adaptively adjusting an exposure time of the modulated light receiving circuit to detect the modulated light of the at least two phases, to Regain the intensity information of at least two phases corresponding to the pixel at different time points. The sensing device based on time-of-flight ranging as described in item 6 of the scope of the patent application further includes: a posture sensor that senses the posture of the sensing device and generates a posture information accordingly, and the processing method also Including: in response to the difference being greater than a difference threshold, the motion blur caused by the difference is determined to be global or local based on the posture information; in response to the global motion blur, the intensity information corresponding to at least two phases of the pixel is discarded, And obtain the intensity information of at least two phases corresponding to the pixel at different time points through the dimming light receiving circuit; and reflect the dynamic blur in the local area, and decide whether to re-transmit the dimming light according to the number of a blur pixel The receiving circuit retrieves the intensity information of at least two phases corresponding to the pixel at different time points, wherein the number of blurred pixels is an amount accumulated in response to the determination of dynamic blur. A processing method based on time-of-flight ranging includes: obtaining intensity information of at least two phases corresponding to at least one pixel, wherein the intensity The information is obtained by using the at least two phases to delay sense a dimming light in time; and according to the difference between the intensity information of the at least two phases, it is determined whether to discard the intensity information corresponding to the at least two phases of the pixel, wherein The step of deciding whether to discard the intensity information corresponding to at least two phases of the pixel includes: determining whether the difference is greater than a difference threshold; in response to the difference being greater than the difference threshold, discarding the pixel corresponding to at least two phases Intensity information; in response to discarding the intensity information corresponding to at least two phases of the pixel, to determine whether to regain the intensity information of at least two phases corresponding to the pixel at different points in time; and according to the number of different phases corresponding to the pixel Strength information to calculate a depth of information. The processing method based on time-of-flight ranging as described in item 9 of the patent application scope, wherein after the step of deciding whether to discard the intensity information corresponding to at least two phases of the pixel, it further includes: adaptively adjusting the detection of the at least two phases Adjusting an exposure time of light to regain the intensity information of at least two phases corresponding to the pixel at different time points. The processing method based on time-of-flight ranging as described in item 9 of the scope of the patent application, wherein after the step of determining whether the difference is greater than the difference threshold, it further includes: obtaining posture information corresponding to sensing the at least Two-phase dimming device; In response to the difference being greater than the difference threshold, the motion blur caused by the difference is determined to be global or local based on the posture information; in response to the global motion blur, the intensity information corresponding to at least two phases of the pixel is discarded, and the Obtain the intensity information of at least two phases corresponding to the pixel at different time points; and respond to the dynamic blur in the local area, and determine whether to re-acquire at least two phases corresponding to the pixel at different time points according to the number of blurred pixels Intensity information, where the number of blurred pixels is an amount that is accumulated in response to the pixel being judged to have motion blur. The processing method based on time-of-flight ranging as described in item 11 of the patent application scope, wherein after the step of judging whether the difference is greater than the difference threshold, it further includes: in response to the number of blurred pixels being greater than a number threshold, abandoning the The pixel corresponds to intensity information of at least two phases, and retrieves intensity information of at least two phases corresponding to the pixel at different time points. The processing method based on time-of-flight ranging as described in item 9 of the patent application scope, wherein after the step of judging whether the difference is greater than the difference threshold, it further includes: abandoning at least one of the paintings in response to the difference being greater than the difference threshold Intensity information corresponding to different phases of the pixel, wherein the discarded at least one pixel is judged to have motion blur; and based on the intensity information of different phases corresponding to the plurality of pixels that have not been discarded Calculate depth information.

Images