JP2010074362A - Distance image generating device - Google Patents

Abstract

A distance image generation apparatus capable of high-speed processing is provided.
A light emitting source that irradiates light to a target space, a photoelectric conversion element that receives reflected light that is irradiated from the light source and reflected by an object in the target space, and converts the light into a charge, and the converted charge An image sensor including a plurality of pixels formed by a combination with a charge storage unit to store, a reading unit that reads the accumulated charges from the plurality of pixels, and a pixel value based on the charges read by the reading unit A distance image generating unit that generates a distance image having a distance value; a determination unit that determines whether or not the distance between the target object in the target space is approached; and a pixel that is to be read by the reading unit is thinned out Thinning means, wherein the thinning means is read by the reading means when the determining means determines that the distance to the object in the target space is approaching. Wherein the thinning pixels that are.
[Selection] Figure 4

Description

  The present invention relates to a distance image generation device, and more particularly to a distance image generation device capable of high-speed processing.

  2. Description of the Related Art Conventionally, an apparatus that generates a so-called distance image using a distance image sensor (there is a CCD type or a CMOS type) is known (for example, Patent Document 1).

  The distance image generating device described in Patent Document 1 emits light whose intensity is modulated at a predetermined light emission frequency into a space, receives reflected light of light emitted from the light source, and has a signal level corresponding to the received light intensity. An image sensor or the like as a distance image sensor including a plurality of photoelectric conversion units that output light reception signals is provided.

In the distance image generation device described in Patent Document 1, reflected light is received and converted into electric charges within a modulation period, and a predetermined calculation is performed based on the converted electric charges, whereby the levels of the modulated light and the reflected light are changed. A phase difference is calculated and a distance image is generated.
JP 2004-32682 A

  However, in the above-mentioned patent document 1, since all pixels are always read, there is a problem that the amount of information is large and a considerable processing time is required.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a distance image generation apparatus capable of high-speed processing.

  In order to solve the above-described problem, the invention according to claim 1 is directed to a light source that irradiates light to a target space, and reflected light that is irradiated from the light source and reflected by an object in the target space to receive charges. An imaging device including a plurality of pixels formed by a combination of a photoelectric conversion element for conversion and a charge storage unit for storing the converted charge, a reading unit for reading the accumulated charge from the plurality of pixels, and the reading unit A distance image generation unit that generates a distance image whose pixel value is a distance value based on the electric charge read out by the determination unit, and a determination unit that determines whether or not the distance between the object in the target space is approached, A thinning means for thinning out pixels to be read by the reading means, and the thinning means is determined by the determination means to be close to the object in the target space Characterized by thinning out pixels that are a read target by the reading means.

  According to the first aspect of the present invention, when it is determined that the distance to the object in the target space is close, the pixels to be read are thinned out by the reading unit, so that all the pixels are always read. The amount of information can be reduced and the processing speed can be increased as compared with the case where it is assumed.

  For example, when all the pixels of a distance image sensor (for example, 10,000 pixels, 15 fps) are to be read, an object in the surrounding environment that moves at a speed of 100 km / h (27.8 m / sec) is only about every 1.85 m. Although detection is not possible, if pixels to be read are thinned out by 3/4, detection can be performed at intervals of about 0.46 m (improvement of detection speed using a distance image).

  In addition, according to the first aspect of the present invention, only when it is determined that the distance from the object in the target space is close, pixels to be read out are thinned out, and the object in the target space is If it is not determined that the distance is close (for example, when the object is located far away), pixels to be read are not thinned out. For this reason, it is possible to prevent the pixels corresponding to the object existing in the distance from being thinned out so that the object cannot be recognized.

  According to a second aspect of the present invention, in the first aspect of the present invention, the thinning-out means includes a thinning-out consisting of a combination of a pulse having a first pulse width and a pulse having a second pulse width shorter than the first pulse width. Based on the pattern, the pixels to be read out are thinned out by the reading means.

  According to the second aspect of the present invention, since the thinning is performed based on a thinning pattern composed of a pulse train obtained by combining a pulse having a first pulse width and a pulse having a second pulse width shorter than the first pulse width, By appropriately combining the pulse and the pulse having the second pulse width, it is possible to thin out the pixels to be read out with various thinning patterns.

  According to a third aspect of the present invention, in the first or second aspect of the present invention, the counting means for counting the number of pixels of the distance image generated by the distance image generation unit for each distance range divided in advance. Comparing means for comparing the number of pixels belonging to the specific distance range counted by the counting means with a threshold value, wherein the judging means exceeds the number of pixels belonging to the specific distance range. In this case, it is determined that the distance from the object in the object space is close.

  According to the third aspect of the present invention, when the number of pixels belonging to the specific distance range exceeds the threshold without paying attention to the individual objects in the surrounding environment, the distance to the object in the target space. Therefore, it is possible to increase the processing speed compared to calculating the distance for each object in the surrounding environment.

  According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, the thinning means includes the plurality of pixels, a pixel group corresponding to a specific object among the plurality of pixels, or The pixels to be read out by the reading means are thinned out from the pixels other than the pixel group corresponding to a specific object among the plurality of pixels.

  According to the invention described in claim 4, a plurality of pixels (all pixels), a pixel group corresponding to a specific object among the plurality of pixels, or a pixel group corresponding to a specific object among the plurality of pixels. It is possible to thin out pixels to be read from other pixels.

  For example, when the distance image generating apparatus of the present invention is mounted on a moving body such as a vehicle and travels at a relatively slow speed in a city, a specific object (eg, bus, person, tree) among a plurality of pixels. It is preferable that pixels to be read are thinned out from the pixel group corresponding to. In this way, it is possible to pay attention to changes around the pixel group corresponding to a specific object (for example, bus, person, tree).

  In addition, when the distance image generating apparatus of the present invention is mounted on a moving body such as a vehicle and runs on a highway, a pixel group corresponding to a specific object (for example, a bus, a person, a tree) among a plurality of pixels. It is preferable to thin out pixels to be read from other pixels. In this way, it is possible to pay attention to changes in the pixel group corresponding to a specific object (for example, bus, person, tree).

  According to a fifth aspect of the present invention, in the invention according to any one of the first to third aspects, the detection means detects a size of a specific object included in the distance image generated by the distance image generation means. Collating means for collating the size and shape database of the specific object detected by the detecting means, and the thinning means is configured such that the specific object detected by the detecting means is the shape. When registered in the database, pixels to be read out by the reading unit are thinned out from a pixel group corresponding to the specific object among the plurality of pixels.

  According to the invention described in claim 5, since the shape database is used, it is possible to more accurately recognize a specific object (for example, bus, person, tree) among a plurality of pixels. It is possible to thin out pixels to be read out from a pixel group (for example, bus, person, tree) corresponding to the specified target object.

  The invention according to claim 6 is the invention according to any one of claims 1 to 3, wherein the specific object included in the distance image generated by the distance image generation means is generated at the next timing. The image processing apparatus further includes an estimation unit that estimates a pixel corresponding to a boundary line of the specific object in the range image, and the thinning unit is specified in the range image generated at the next timing estimated by the estimation unit. The pixels to be read out are thinned out by pixels other than the pixels corresponding to the boundary line of the target object.

  According to the sixth aspect of the present invention, the pixel to be read out is a pixel other than the pixel group corresponding to the boundary line of the specific object in the distance image generated at the next timing estimated by the estimation unit. In order to thin out pixels, for example, when a specific object is a vehicle such as a truck, pixels corresponding to the boundary line of the vehicle are always read out without being thinned out. For this reason, it is possible to detect a motorcycle or the like that suddenly appears from the shadow of the vehicle.

  According to the present invention, it is possible to provide a distance image generation device capable of high-speed processing.

[First embodiment]
Hereinafter, a distance image generating apparatus according to an embodiment of the present invention will be described with reference to the drawings.

[Configuration of Distance Image Generation Device]
First, the configuration of the distance image generation device of this embodiment will be described.

  1 and 4 are block diagrams of the distance image generation apparatus of the present embodiment.

  As shown in FIG. 1, the distance image generation device 1 includes a time-of-flight distance image sensor 10 (hereinafter referred to as a distance image sensor 10). The distance image sensor 10 includes a light source 11, an image sensor 15, a control unit 16, a distance image generation unit 17, a normal image generation unit 18, and the like.

  The light source 11 is a light source that irradiates the target space with light of a reference frequency fs (for example, infrared light or visible light modulated at high speed with a sine wave or a rectangular wave), and can perform high-speed modulation of LEDs, lasers, and the like. Devices are used.

  The image sensor 15 is an XY address CMOS image sensor, and receives reflected light that is irradiated from the light source 11 and reflected by an object (for example, a bus, a person, or a tree in FIG. 7) in the target space. A plurality of pixels (not shown) each including a combination of a photoelectric conversion element that converts the charge into a charge and a charge storage portion that stores the converted charge. Specifically, the imaging element 15 receives a plurality of photoelectric conversion elements that receive incident light 14 including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light 14 into charges. A plurality of charge storage units (in this embodiment, four charge storage units C1, C2, C3, and C4 are illustrated. In the following, when C1, C2, C3, and C4 are used in mathematical expressions, C1, C2, C3, and C4), and means for distributing charges converted by the photoelectric conversion elements to a plurality of charge storage units in synchronization with the modulation of the light source 11. (Neither shown).

  The control unit 16 controls the light source 11 and the image sensor 15 synchronously. The image sensor 15 performs charge distribution based on the reference frequency fs. Specifically, as illustrated in FIG. 2, the image sensor 15 is synchronized with the reference frequency fs according to the gate control signals DG 1, DG 2, DG 3, and DG 4 supplied from the control unit 16. Each time, the gate is opened and closed, and the charges converted by the photoelectric conversion elements are distributed to the charge storage portions C1, C2, C3, and C4, respectively. The imaging element 15 acquires one image by repeating the cycle of time T1 to T4 several thousand to several hundred thousand times. This time for acquiring one image is called one frame. Note that the number of charge storage units is not limited to four. The present invention can be similarly applied to two or more charge storage portions.

  The distance image generation unit 17 performs a predetermined calculation (using Expression 1 described later) based on the charges distributed as described above (charges stored in the charge storage units C1, C2, C3, and C4), and the pixel value A distance image in which is a distance value to the object to be measured is generated. The distance image becomes three-dimensional (XYZ direction) information including the distance information of the pixel in addition to the two-dimensional (XY direction) position information. According to the distance image, it is possible to detect the shape of the target object based on the distance information even when the normal image has little change in luminance value and the feature amount cannot be extracted by edge detection or the like.

  The normal image generation unit 18 performs a predetermined calculation (using formula 2 described later) based on the charges distributed as described above (charges stored in the charge storage units C1, C2, C3, and C4), and the pixel value A normal image having a luminance value is generated.

  As described above, in the present embodiment, both the distance image and the normal image can be acquired using one image sensor 15.

[Principle for generating range images]
Next, the principle of generating a distance image will be described.

  FIG. 3 is a diagram for explaining the principle of generating a distance image. In FIG. 3, a sine wave 21 represents light of the reference frequency fs emitted from the light source 11, and a sine wave 22 represents light incident on the image sensor 15. The phase difference φ between the sine wave 21 and the sine wave 22 represents a delay caused by the flight time of the light of the reference frequency fs emitted from the light source 11 to the object to be measured.

  In FIG. 3, one cycle of the light emitted from the light source 11 is divided into four periods, and the charges are distributed to the four charge storage units C1, C2, C3, and C4. If each period is T1, T2, T3, and T4, and the amount of charge accumulated in each period is C1, C2, C3, and C4, the phase difference φ is expressed by the following equation.

  Since the speed of light is known, by obtaining this phase difference φ, the distance to the object to be measured can be obtained, and a distance image whose pixel value is the distance value can be generated. For example, when the light source 11 is modulated at 10 MHz, it is possible to measure a distance of up to 15 m from the light emission side in a round trip.

  The charge amount average A used as general image data (normal image) is expressed by the following equation.

  Further, the amplitude amount B of the modulated light component reflected by the object to be measured is expressed by the following equation.

  In general, the modulation frequency of the light source is several tens of MHz, and therefore one period of modulation is about several tens of ns. Therefore, in order to obtain a distance image, a charge accumulation time of several hundred to several hundred thousand cycles is required.

[Principle of thinning]
Next, the principle of thinning will be described.

  4 and 5 are diagrams for explaining a thinning process for thinning out pixels (detection pixels) to be read.

  FIG. 5B shows an example of a thinning pattern composed of a pulse train in which a pulse having a first pulse width H1 and a pulse having a second pulse width H2 shorter than the first pulse width H1 are combined. When this pulse train is supplied by the horizontal counter 19 (H_count) (or vertical counter 20 (V_count)) shown in FIG. 4 and a pixel is selected, charge is read from the pixel selected by the pulse having the first pulse width H1. (Because the second pulse width H2 is shorter than the first pulse width H1, no charge is read from the pixel selected by the pulse having the second pulse width H2.)

  By appropriately combining the pulse having the first pulse width H1 and the pulse having the second pulse width H2, for example, a pulse train corresponding to the thinning pattern shown in FIGS. 10 to 12, 14, 16, 19, and 21. Can be created. In each figure, ■ represents a pixel to be detected (a pixel from which charges are read), and blanks other than ■ represent pixels from which charges are not to be read (pixels not to be detected and thinned).

  10 is a pattern that is thinned out from all pixels in a checkered pattern, FIG. 11 is a pattern that is thinned out from every pixel every other column, FIG. 12 is a pattern that is thinned out every other line from every pixel, and FIG. FIG. 14B is a pattern that is thinned out in a checkered pattern from an equidistant portion that is imprinted in the image, and FIG. 14B is a pattern that is thinned out in a checkered pattern from a portion other than the equidistant portion that is imprinted in the distance image. (A) is a pattern that thins out most of the equidistant portion imprinted in the distance image, and FIG. 16 (b) is a pattern that thins out all portions other than the equidistant portion imprinted in the distance image. (A) is a pattern which is imprinted on the distance image and thins out most of the part recognized as a specific object (for example, bus, person, tree in FIG. 7), and FIG. Imprinted, special This is a pattern in which all parts other than those recognized as objects (for example, buses, people, trees in FIG. 7) are thinned out, and FIG. 21A is imprinted on the distance image and recognized as a specific object. The boundary portion of the portion that is not thinned out is a pattern that is thinned out in a checkered pattern from the other portions, and FIG. 21B is the boundary of the portion that is imprinted in the distance image and recognized as a specific object This is a pattern in which all the other parts are thinned out without thinning out the parts.

  As described above, by using a thinning pattern composed of a pulse train that is a combination of a pulse having the first pulse width H1 and a pulse having the second pulse width H2, thinning processing for thinning pixels (detection pixels) to be read is performed. Is possible.

[Operation Example 1 of Distance Image Generating Device 1]
Next, an operation example 1 of the distance image generating apparatus 1 having the above configuration will be described with reference to the drawings.

  FIG. 6 is a flowchart for explaining an operation example 1 of the distance image generating apparatus 1. The following processing is mainly performed by the control unit 16 or a control unit (not shown) provided separately.

  First, the distance image generating apparatus 1 performs processing for capturing an image of the surrounding environment with the image sensor 15 and acquiring a distance image (distance data) (step S1). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation device 1 performs a process of reading charges from all pixels. Specifically, the following processing is performed.

  The distance image generation unit 17 sequentially selects pixels by a pulse train (see FIG. 5A) having the first pulse width H1 supplied by the horizontal counter 19 (H_count) (or the vertical counter 20 (V_count)), and Charge is read from the selected pixel.

  Then, the distance image generation unit 17 performs a predetermined calculation (calculation of the above formula 1) based on the read charge, calculates the phase difference φ with respect to the light source 11, and FIG. 8A and FIG. As shown in FIG. 5, a distance image having a pixel value as a distance value is generated. This generated distance image is stored in, for example, the memory 21 shown in FIG. FIG. 8A shows an example of a distance image generated at time t-dt, and FIG. 8B shows an example of a distance image generated at time t.

  Next, the distance image generation device 1 determines whether or not a short-distance signal exists in the distance image acquired in step S <b> 1 or more, that is, an object in the target space (for example, a bus, a person in FIG. 7). , Tree) is determined whether or not the distance is short (step S2). Specifically, the following processing is performed.

  As shown in FIGS. 9A and 9B, the distance image generating device 1 determines the number of pixels of the distance image acquired in step S1 by dividing the distance ranges A1 to A2, A2 to A3, and A3. It counts for every A4, A4-A5, A5-A6.

  Next, the comparison unit 22 compares the number of pixels belonging to a specific distance range (for example, the distance range A1 to A2) with a threshold value. As a result of this comparison, as shown in FIG. 9B, when it is determined that the number of pixels belonging to a specific distance range (for example, distance ranges A1 to A2) exceeds the threshold (step S2: Yes), the distance The image generation device 1 determines that the distance from the object in the target space (for example, a bus, a person, or a tree in FIG. 8) has approached, and determines a predetermined thinning pattern (for example, FIG. 10, FIG. 11 or FIG. Using the thinning pattern shown in FIG. 12, thinning processing is performed to thin out pixels to be read (detection target pixels). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation unit 17 performs a thinning pattern (for example, FIG. 10, FIG. 11, FIG. 10) including a pulse train that is a combination of a pulse having a first pulse width H 1 and a pulse having a second pulse width H 2 shorter than the first pulse width H 1. Based on the thinning pattern shown in FIG. 12, pixels to be read (detection target pixels) are thinned out from a plurality of pixels (all pixels). That is, the distance image generation unit 17 supplies a pulse train (for example, a pulse train corresponding to the thinning pattern shown in FIG. 10, FIG. 11 or FIG. 12) supplied by the horizontal counter 19 (H_count) (or the vertical counter 20 (V_count)). Then, the pixels are sequentially selected, and the charge is read from the pixel selected by the pulse having the first pulse width H1 (the charge is not read from the pixel selected by the pulse having the second pulse width H2).

  Then, the distance image generation unit 17 performs a predetermined calculation (the calculation of Equation 1 above) based on the read charge, calculates the phase difference φ with the light source 11, and generates a distance image whose pixel value is a distance value. To do. This generated distance image is stored in, for example, the memory 21 shown in FIG.

  As described above, according to the first operation example 1, when it is determined that the distance to the target object in the target space (for example, the bus, the person, and the tree in FIG. 8) has approached (Step S2: Yes). Since the pixels to be read are thinned out (step S3), the amount of information can be reduced and the processing speed can be increased compared to the case where all the pixels are always read.

  For example, when all the pixels of a distance image sensor (for example, 10,000 pixels, 15 fps) are to be read, an object in the surrounding environment that moves at a speed of 100 km / h (27.8 m / sec) is only about every 1.85 m. Although detection is not possible, if pixels to be read are thinned out by 3/4, detection can be performed at intervals of about 0.46 m (improvement of detection speed using a distance image).

  Further, according to the first operation example, because thinning is performed based on a thinning pattern composed of a pulse train in which a pulse having a first pulse width H1 and a pulse having a second pulse width H2 shorter than the first pulse width H1 are combined (step S3), By appropriately combining the first pulse width pulse and the second pulse width pulse, the pixels to be read out are thinned out in various thinning patterns (for example, thinning patterns shown in FIGS. 10, 11, and 12). Is possible.

  Further, according to this operation example 1, the number of pixels of the distance image generated in step S1 is determined without paying attention to the individual objects (for example, buses, people, trees in FIG. 8) in the surrounding environment. Counting for each pre-divided distance range, and when it is determined that the number of pixels belonging to a specific distance range (for example, A1 to A2) exceeds the threshold (step S2: Yes), the object in the target space Therefore, the processing speed can be increased compared to calculating the distance for each object in the surrounding environment.

  Moreover, according to this operation example 1, only when it is determined that the distance from the object in the target space (for example, the bus, the person, the tree in FIG. 8) is approaching (Step S2: Yes), the read target If the pixel to be read out is thinned out and it is not determined that the distance to the target object in the target space is short (for example, when the target object is in the distance), the pixel to be read out is not thinned out . For this reason, it is possible to prevent the pixels corresponding to the object existing in the distance from being thinned out so that the object cannot be recognized.

[Operation Example 2 of Distance Image Generating Device 1]
Next, an operation example 2 of the distance image generating apparatus 1 having the above configuration will be described with reference to the drawings.

  FIG. 13 is a flowchart for explaining an operation example 2 of the distance image generating apparatus 1. The following processing is mainly performed by the control unit 16 or a control unit (not shown) provided separately.

  First, the distance image generating apparatus 1 performs processing for capturing an image of the surrounding environment with the image sensor 15 and acquiring a distance image (distance data) (step S10). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation device 1 performs a process of reading charges from all pixels. Specifically, the following processing is performed.

  The distance image generation unit 17 sequentially selects pixels by a pulse train (see FIG. 5A) having the first pulse width H1 supplied by the horizontal counter 19 (H_count) (or the vertical counter 20 (V_count)), and Charge is read from the selected pixel.

  Then, the distance image generation unit 17 performs a predetermined calculation (calculation of the above formula 1) based on the read charge, calculates the phase difference φ with respect to the light source 11, and FIG. 8A and FIG. As shown in FIG. 5, a distance image having a pixel value as a distance value is generated. This generated distance image is stored in, for example, the memory 21 shown in FIG. FIG. 8A shows an example of a distance image generated at time t-dt, and FIG. 8B shows an example of a distance image generated at time t.

  Next, the distance image generation device 1 determines whether or not a short-distance signal exists in the distance image acquired in step S <b> 1 or more, that is, an object in the object space (for example, a bus, a person in FIG. 8). It is determined whether or not the distance from the tree has approached (step S11). Specifically, the following processing is performed.

  As shown in FIGS. 9A and 9B, the distance image generating device 1 determines the number of pixels of the distance image acquired in step S10 as distance ranges A1 to A2, A2 to A3, and A3. It counts for every A4, A4-A5, A5-A6.

  Next, the comparison unit 22 compares the number of pixels belonging to a specific distance range (for example, the distance range A1 to A2) with a threshold value. As a result of this comparison, as shown in FIG. 9B, when it is determined that the number of pixels belonging to a specific distance range (for example, distance ranges A1 to A2) exceeds the threshold (step S11: Yes), the distance The image generating apparatus 1 determines whether or not equidistant signals are continuously present in the distance image acquired in step S10 for a threshold or more (step S12).

  As a result of this determination, when it is determined that equidistant signals are continuously present over the threshold (step S12: Yes), the distance image generating device 1 uses the thinning pattern (for example, FIG. 14A or FIG. 14B). ) And a thinning process for thinning out pixels to be read (detection target pixels) is performed using the thinning pattern (step S13). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation unit 17 performs a thinning pattern (for example, FIG. 14A) including a pulse train obtained by combining a pulse having a first pulse width H1 and a pulse having a second pulse width H2 shorter than the first pulse width H1. Based on the thinning pattern shown in FIG. 14B, a pixel to be read (a pixel to be detected) from a pixel group determined to be equidistant in step S <b> 12 or a pixel other than the pixel group determined to be equidistant. ). That is, the pixels are sequentially detected by a pulse train (for example, a pulse train corresponding to the thinning pattern shown in FIG. 14A or FIG. 14B) supplied by the horizontal counter 19 (H_count) (or the vertical counter 20 (V_count)). Then, charge is read from the pixel selected by the pulse having the first pulse width H1 (charge is not read from the pixel selected by the pulse having the second pulse width H2).

  Then, the distance image generation unit 17 performs a predetermined calculation (the calculation of Equation 1 above) based on the read charge, calculates the phase difference φ with the light source 11, and generates a distance image whose pixel value is a distance value. To do. This generated distance image is stored in, for example, the memory 21 shown in FIG.

  As described above, according to the second operation example, when it is determined that the distance from the target object in the target space (for example, the bus, the person, and the tree in FIG. 8) has approached (Step S11: Yes). Since the pixels to be read are thinned out (step S13), the amount of information can be reduced and the processing speed can be increased compared to the case where all the pixels are always read.

  For example, when all the pixels of a distance image sensor (for example, 10,000 pixels, 15 fps) are to be read, an object in the surrounding environment that moves at a speed of 100 km / h (27.8 m / sec) is only about every 1.85 m. Although detection is not possible, if pixels to be read are thinned out by 3/4, detection can be performed at intervals of about 0.46 m (improvement of detection speed using a distance image).

  Further, according to the second operation example, in order to perform thinning based on a thinning pattern composed of a pulse train in which a pulse having the first pulse width H1 and a pulse having the second pulse width H2 shorter than the first pulse width H1 are combined (step S13), By appropriately combining the pulse having the first pulse width and the pulse having the second pulse width, the pixels to be read out are subjected to various thinning patterns (FIG. 14A, FIG. 14B, FIG. 16A, It can be thinned out in FIG.

  Further, according to the second operation example, the number of pixels of the distance image generated in step S10 is determined without paying attention to individual objects (for example, buses, people, trees in FIG. 8) in the surrounding environment. Counting for each pre-divided distance range, and when it is determined that the number of pixels belonging to a specific distance range (for example, A1 to A2) exceeds the threshold (step S11: Yes), the object in the target space Therefore, the processing speed can be increased compared to calculating the distance for each object in the surrounding environment.

  Further, according to the second operation example, only when it is determined that the distance to the target object in the target space (for example, the bus, the person, the tree in FIG. 8) is approaching (Step S11: Yes), the read target If the pixel to be read out is thinned out and it is not determined that the distance to the target object in the target space is short (for example, when the target object is in the distance), the pixel to be read out is not thinned out . For this reason, it is possible to prevent the pixels corresponding to the object existing in the distance from being thinned out so that the object cannot be recognized.

  Further, according to the second operation example, a pixel group corresponding to a specific object (for example, a bus, a person, a tree in FIG. 8) or a pixel group corresponding to the specific object among a plurality of pixels. It is possible to thin out pixels to be read out from the pixels (see FIGS. 14A and 14B).

  For example, when the distance image generating device 1 is mounted on a moving body such as a vehicle and travels at a relatively slow speed in a city, a specific object (for example, for example, using the thinning pattern shown in FIG. It is preferable to thin out pixels to be read out from a pixel group corresponding to a bus, a person, and a tree shown in FIG. In this way, it becomes possible to pay attention to changes around the pixel group corresponding to a specific object (for example, a bus, a person, or a tree shown in FIG. 8).

  Further, for example, when the distance image generating apparatus 1 is mounted on a moving body such as a vehicle and runs on a highway, a specific object (a plurality of pixels) is selected using a thinning pattern shown in FIG. For example, it is preferable to thin out pixels to be read from pixels other than the pixel group corresponding to the bus, person, and tree shown in FIG. In this way, it is possible to pay attention to changes in the pixel group corresponding to a specific object (for example, bus, person, tree).

  In addition, this operation example 2 can be implemented even if step S11 is deleted as shown in FIG.

[Operation Example 3 of Distance Image Generating Device 1]
Next, an operation example 3 of the distance image generating apparatus 1 having the above configuration will be described with reference to the drawings.

  FIG. 17 is a flowchart for explaining an operation example 3 of the distance image generating apparatus 1. The following processing is mainly performed by the control unit 16 or a control unit (not shown) provided separately.

  First, the distance image generation device 1 performs processing for capturing an image of the surrounding environment with the image sensor 15 and acquiring a distance image (distance data) (step S20). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation device 1 performs a process of reading charges from all pixels. Specifically, the following processing is performed.

  The distance image generation unit 17 sequentially selects pixels by a pulse train (see FIG. 5A) having the first pulse width H1 supplied by the horizontal counter 19 (H_count) (or the vertical counter 20 (V_count)), and Charge is read from the selected pixel.

  Then, the distance image generation unit 17 performs a predetermined calculation (calculation of the above formula 1) based on the read charge, calculates the phase difference φ with respect to the light source 11, and FIG. 8A and FIG. As shown in FIG. 5, a distance image having a pixel value as a distance value is generated. This generated distance image is stored in, for example, the memory 21 shown in FIG. FIG. 8A shows an example of a distance image generated at time t-dt, and FIG. 8B shows an example of a distance image generated at time t.

  Next, the distance image generation device 1 determines whether or not a short-distance signal exists in the distance image acquired in step S20 above a threshold value, that is, an object in the object space (for example, a bus, a person in FIG. 8). , Tree) is determined (step S21). Specifically, the following processing is performed.

  As shown in FIGS. 9A and 9B, the distance image generating device 1 determines the number of pixels of the distance image acquired in step S20 from the distance ranges A1 to A2, A2 to A3, and A3 that are preliminarily divided. It counts for every A4, A4-A5, A5-A6.

  Next, the comparison unit 22 compares the number of pixels belonging to a specific distance range (for example, the distance range A1 to A2) with a threshold value. As a result of this comparison, as shown in FIG. 9B, when it is determined that the number of pixels belonging to a specific distance range (for example, distance ranges A1 to A2) has exceeded the threshold (step S21: Yes), the distance The image generation device 1 determines whether or not equidistant signals are continuously present in the distance image acquired in step S20 for a threshold or more (step S22).

  As a result of this determination, when it is determined that equidistant signals are continuously present beyond the threshold (step S22: Yes), the distance image generating device 1 recognizes the pixel group determined to be the equidistant as one group. Then, the actual size (height, width, etc.) of the pixel group recognized as the equidistance is obtained by calculation (step S23).

  For example, as shown in FIG. 18, if the pixel value of a pixel recognized as equidistant is y, the focal length of the lens is f, and the size of one pixel is x, a group of pixels determined to be equidistant (FIG. 18). In this case, the actual size a (height or width) of the tree can be obtained by the equation a = y × x / f.

  Next, the distance image generation device 1 determines whether or not the group recognized in step S23 matches the pattern (step S24). Specifically, the actual size a (height and width) of the group obtained in step S23 is compared with a shape database (a database storing patterns such as sizes of people, cars, utility poles, etc.).

  As a result of this collation, when it is determined that the group recognized in step S23 matches the pattern (step S24: Yes), the distance image generating device 1 uses the thinning pattern (for example, the thinning pattern shown in FIG. 19A). ) And a thinning process for thinning out pixels to be read (detection target pixels) is performed using the thinning pattern (step S25). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation unit 17 performs a thinning pattern (for example, FIG. 19A) including a pulse train obtained by combining a pulse having a first pulse width H1 and a pulse having a second pulse width H2 shorter than the first pulse width H1. Based on the thinning pattern shown), pixels to be read (detection target pixels) are thinned out from the pixel group corresponding to the group matched in step S24. That is, the pixels are sequentially selected by the pulse train (pulse train corresponding to the thinning pattern shown in FIG. 19A) supplied by the horizontal counter 19 (H_count) (or the vertical counter 20 (V_count)), and the first pulse width H1 The charge is read from the pixel selected by the pulse No. (the charge is not read from the pixel selected by the pulse having the second pulse width H2).

  Then, the distance image generation unit 17 performs a predetermined calculation (the calculation of Equation 1 above) based on the read charge, calculates the phase difference φ with the light source 11, and generates a distance image whose pixel value is a distance value. To do. This generated distance image is stored in, for example, the memory 21 shown in FIG.

  On the other hand, as a result of the determination in step S24, when it is determined that the group recognized in step S23 does not match the pattern (step S24: No), the distance image generating device 1 displays the thinning pattern (for example, FIG. 19B). (Thinning pattern shown) is generated, and the thinning pattern is used to thin out the pixels to be read (detection target pixels) (step S26). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation unit 17 generates a plurality of pixels (all pixels) based on a thinning pattern composed of a pulse train obtained by combining a pulse having a first pulse width H1 and a pulse having a second pulse width H2 shorter than the first pulse width H1. Pixels to be read (detection target pixels) are thinned out from pixels other than the pixels corresponding to the group recognized in step S23. That is, pixels are sequentially selected by a pulse train (for example, a pulse train corresponding to the thinning pattern shown in FIG. 19B) supplied by the horizontal counter (H_count) (or vertical counter (V_count)), and the first pulse width H1 The charge is read from the pixel selected by the pulse No. (the charge is not read from the pixel selected by the pulse having the second pulse width H2).

  Then, the distance image generation unit 17 performs a predetermined calculation (the calculation of Equation 1 above) based on the read charge, calculates the phase difference φ with the light source 11, and generates a distance image whose pixel value is a distance value. To do. This generated distance image is stored in, for example, the memory 21 shown in FIG.

  As described above, according to the third operation example, when it is determined that the distance from the target object in the target space (for example, the bus, the person, and the tree in FIG. 8) has approached (Step S21: Yes). Since the pixels to be read are thinned out (step S25), the amount of information can be reduced and the processing speed can be increased compared to the case where all the pixels are always read.

  For example, when all the pixels of a distance image sensor (for example, 10,000 pixels, 15 fps) are to be read, an object in the surrounding environment that moves at a speed of 100 km / h (27.8 m / sec) is only about every 1.85 m. Although detection is not possible, if pixels to be read are thinned out by 3/4, detection can be performed at intervals of about 0.46 m (improvement of detection speed using a distance image).

  Further, according to the third operation example, because thinning is performed based on a thinning pattern composed of a pulse train in which a pulse having a first pulse width H1 and a pulse having a second pulse width H2 shorter than the first pulse width H1 are combined (step S25), By appropriately combining the first pulse width pulse and the second pulse width pulse, it is possible to thin out the pixels to be read out in various thinning patterns (FIGS. 19A, 19B, etc.). It becomes.

  Further, according to the third operation example, the number of pixels of the distance image generated in step S20 is determined without paying attention to individual objects (for example, buses, people, and trees in FIG. 8) in the surrounding environment. Counting for each pre-divided distance range, and when it is determined that the number of pixels belonging to a specific distance range (for example, A1 to A2) has exceeded the threshold (step S21: Yes), the object in the target space Therefore, the processing speed can be increased compared to calculating the distance for each object in the surrounding environment.

  Further, according to the third operation example, only when it is determined that the distance to the target object in the target space (for example, the bus, the person, the tree in FIG. 8) has approached (Step S21: Yes), If the pixel to be read out is thinned out and it is not determined that the distance to the target object in the target space is short (for example, when the target object is in the distance), the pixel to be read out is not thinned out . For this reason, it is possible to prevent the pixels corresponding to the object existing in the distance from being thinned out so that the object cannot be recognized.

  In addition, according to the third operation example, a pixel group corresponding to a specific object (for example, a bus, a person, a tree in FIG. 8) or a pixel group corresponding to a specific object among a plurality of pixels. It is possible to thin out the pixels to be read out from the pixels (see FIGS. 19A and 19B).

  In addition, in the third operation example, since the shape database is used, it is possible to more accurately recognize a specific target object (for example, a bus, a person, and a tree in FIG. 8) among a plurality of pixels. It is possible to thin out pixels to be read out from a pixel group (for example, a bus, a person, a tree in FIG. 8) corresponding to the recognized specific object or a pixel other than the pixel group.

[Operation Example 4 of Distance Image Generating Device 1]
Next, an operation example 4 of the distance image generating apparatus 1 having the above configuration will be described with reference to the drawings.

  FIG. 20 is a flowchart for explaining an operation example 4 of the distance image generating apparatus 1. The following processing is mainly performed by the control unit 16 or a control unit (not shown) provided separately.

  First, the distance image generating apparatus 1 performs processing for capturing an image of the surrounding environment with the image sensor 15 and acquiring a distance image (distance data) (step S30). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation device 1 performs a process of reading charges from all pixels. Specifically, the following processing is performed.

  The distance image generation unit 17 sequentially selects pixels by a pulse train (see FIG. 5A) having the first pulse width H1 supplied by the horizontal counter 19 (H_count) (or the vertical counter 20 (V_count)), and Charge is read from the selected pixel.

  Then, the distance image generation unit 17 performs a predetermined calculation (calculation of the above formula 1) based on the read charge, calculates the phase difference φ with respect to the light source 11, and FIG. 8A and FIG. As shown in FIG. 5, a distance image having a pixel value as a distance value is generated. This generated distance image is stored, for example, in the memory shown in FIG. FIG. 8A shows an example of a distance image generated at time t-dt, and FIG. 8B shows an example of a distance image generated at time t.

  Next, the distance image generation device 1 determines whether or not a short-distance signal exists in the distance image acquired in step S30 above a threshold value, that is, an object in the object space (for example, a bus, a person in FIG. , Tree) is determined (step S31). Specifically, the following processing is performed.

  As shown in FIGS. 9A and 9B, the distance image generating device 1 determines the number of pixels of the distance image acquired in step S30 as distance ranges A1 to A2, A2 to A3, and A3. It counts for every A4, A4-A5, A5-A6.

  Next, the comparison unit 22 compares the number of pixels belonging to a specific distance range (for example, the distance range A1 to A2) with a threshold value. As a result of this comparison, as shown in FIG. 9B, when it is determined that the number of pixels belonging to a specific distance range (for example, distance ranges A1 to A2) has exceeded the threshold (step S31: Yes), the distance The image generating apparatus 1 determines whether or not equidistant signals are continuously present in the distance image acquired in step S30 for a threshold or more (step S32).

  As a result of this determination, when it is determined that equidistant signals are continuously present beyond the threshold (step S32: Yes), the distance image generating device 1 recognizes the pixel group determined to be equidistant as one group. Then, the actual size (height, width, etc.) of the pixel group recognized as the equidistant is obtained by calculation (step S33).

  For example, as shown in FIG. 18, if the pixel value of a pixel recognized as equidistant is y, the focal length of the lens is f, and the size of one pixel is x, a group of pixels determined to be equidistant (FIG. 18). In this case, the actual size a (height or width) of the tree can be obtained by the equation a = y × x / f.

  Next, the distance image generation device 1 determines whether or not the group recognized in step S33 matches the pattern (step S34). Specifically, the actual size a (height and width) of the group obtained in step S33 is collated with a shape database (database storing patterns such as sizes of people, cars, utility poles, etc.).

  As a result of this collation, when it is determined that the group recognized in step S33 matches the pattern (step S34: Yes), the distance image generating device 1 uses the thinning pattern (for example, the thinning pattern shown in FIG. 21A). ) And a thinning process for thinning out pixels to be read (detection target pixels) using the thinning pattern (step S35). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation unit 17 performs a thinning pattern (for example, FIG. 21A) including a pulse train obtained by combining a pulse having a first pulse width H1 and a pulse having a second pulse width H2 shorter than the first pulse width H1. Based on the thinning pattern shown, pixels to be read (detection target pixels) are thinned out from pixels other than the pixel group corresponding to the boundary line of the specific target object (for example, the group matched in step S34). That is, pixels are sequentially selected by a pulse train (for example, a pulse train corresponding to the thinning pattern shown in FIG. 21A) supplied by the horizontal counter 19 (H_count) (or the vertical counter 20 (V_count)), and the first pulse The charge is read from the pixel selected by the pulse having the width H1 (the charge is not read from the pixel selected by the pulse having the second pulse width H2).

  In this case, in the distance image generated at the next timing based on the amount of change in the distance from the target object (for example, a vehicle such as a truck) included in the distance image generated in step S30. It is preferable to provide an estimation unit that estimates a pixel corresponding to the boundary line of the specific object, and to thin out pixels to be read out from pixels other than the pixel estimated by the estimation unit. In this way, for example, when a specific object is a vehicle such as a truck, pixels corresponding to the boundary line of the vehicle are always read out without being thinned out. It becomes possible to detect a motorcycle that suddenly appears.

  Then, the distance image generation unit 17 performs a predetermined calculation (the calculation of Equation 1 above) based on the read charge, calculates the phase difference φ with the light source 11, and generates a distance image whose pixel value is a distance value. To do. This generated distance image is stored in, for example, the memory 21 shown in FIG.

  On the other hand, when it is determined that the group recognized in step S33 does not match the pattern as a result of the collation in step S34 (step S34: No), the distance image generating device 1 displays the thinning pattern (for example, FIG. 21B). (Thinning pattern shown) is generated, and using the thinning pattern, thinning processing is performed to thin out pixels to be read (detection target pixels) (step S36). Specifically, the following processing is performed.

  The light source 11 irradiates the target space (for example, FIG. 7A and FIG. 7B) with modulated light according to the synchronization signal of the control unit 16. The image sensor 15 receives incident light including reflected light that is irradiated from the light source 11 and reflected by an object in the target space, and converts the incident light into charges by a plurality of photoelectric conversion elements. The converted charges are distributed to the plurality of charge storage units C1, C2, C3, and C4 according to the gate control signals DG1, DG2, DG3, and DG4 of the control unit 16, respectively. The above processing is continued until the light accumulation time ends (elapses).

  Next, the distance image generation unit 17 generates a plurality of pixels (all pixels) based on a thinning pattern composed of a pulse train obtained by combining a pulse having a first pulse width H1 and a pulse having a second pulse width H2 shorter than the first pulse width H1. Among pixels, pixels to be read (detection target pixels) are thinned out from pixels other than the pixel group corresponding to the boundary line of a specific target object (for example, the group recognized in step S33). That is, pixels are sequentially selected by a pulse train (pulse train corresponding to the thinning pattern shown in FIG. 21B) supplied by a horizontal counter (H_count) (or vertical counter (V_count)), and a pulse having a first pulse width H1. The charge is read out from the pixel selected by (the charge is not read out from the pixel selected by the pulse having the second pulse width H2).

  Then, the distance image generation unit 17 performs a predetermined calculation (the calculation of Equation 1 above) based on the read charge, calculates the phase difference φ with the light source 11, and generates a distance image whose pixel value is a distance value. To do. This generated distance image is stored, for example, in the memory shown in FIG.

  As described above, according to the fourth operation example, when it is determined that the distance from the target object in the target space (for example, the bus, the person, and the tree in FIG. 8) has approached (Step S31: Yes). Since the pixels to be read are thinned out (step S35), the amount of information can be reduced and the processing speed can be increased as compared with the case where all the pixels are always read.

  For example, when all the pixels of a distance image sensor (for example, 10,000 pixels, 15 fps) are to be read, an object in the surrounding environment that moves at a speed of 100 km / h (27.8 m / sec) is only about every 1.85 m. Although detection is not possible, if pixels to be read are thinned out by 3/4, detection can be performed at intervals of about 0.46 m (improvement of detection speed using a distance image).

  Further, according to the fourth operation example, because thinning is performed based on a thinning pattern composed of a pulse train in which a pulse having the first pulse width H1 and a pulse having the second pulse width H2 shorter than the first pulse width H1 are combined (step S35), By appropriately combining the first pulse width pulse and the second pulse width pulse, it is possible to thin out the pixels to be read out in various thinning patterns (FIG. 21A, FIG. 21B, etc.). It becomes.

  Further, according to the fourth operation example, the number of pixels of the distance image generated in step S30 is determined without paying attention to individual objects (for example, buses, people, trees in FIG. 8) in the surrounding environment. Counting for each pre-divided distance range, and when it is determined that the number of pixels belonging to a specific distance range (for example, A1 to A2) has exceeded the threshold (step S31: Yes), the object in the target space Therefore, the processing speed can be increased compared to calculating the distance for each object in the surrounding environment.

  Further, according to the fourth operation example, only when it is determined that the distance to the target object in the target space (for example, the bus, the person, the tree in FIG. 8) is approaching (step S31: Yes), the read target If the pixel to be read out is thinned out and it is not determined that the distance to the target object in the target space is short (for example, when the target object is in the distance), the pixel to be read out is not thinned out . For this reason, it is possible to prevent the pixels corresponding to the object existing in the distance from being thinned out so that the object cannot be recognized.

  Further, according to the fourth operation example, a pixel group corresponding to a specific object among the plurality of pixels or a pixel other than the pixel group corresponding to the specific object among the plurality of pixels is set as a reading target. Pixels can be thinned out (see FIGS. 21A and 21B).

  Further, according to the fourth operation example, since the shape database is used, it is possible to more accurately recognize a specific object (for example, a bus, a person, and a tree in FIG. 8) among a plurality of pixels. The pixels to be read can be thinned out from a pixel group (for example, a bus, a person, a tree in FIG. 8) corresponding to the recognized specific object or a pixel other than the pixel group.

  Further, according to the fourth operation example, pixels to be read out from pixels other than the pixel group corresponding to the boundary line of the specific object in the distance image generated at the next timing estimated by the estimation unit. For example, when a specific object is a vehicle such as a truck, pixels corresponding to the boundary line of the vehicle are always read out without being thinned out. For this reason, it is possible to detect a motorcycle or the like that suddenly appears from the shadow of the vehicle.

  Next, a modified example will be described.

  In each of the above operation examples, the number of pixels of the generated distance image is divided in advance without paying attention to the individual objects in the surrounding environment (for example, buses, people, trees in FIG. 8). In the above description, the number of pixels belonging to a specific distance range (for example, A1 to A2) is determined to have exceeded the threshold value, and it is determined that the distance to the target in the target space is approaching. However, the present invention is not limited to this.

  For example, focusing on individual objects in the surrounding environment, calculating the distance for each individual object using the generated distance image, and determining that the calculated distance exceeds the threshold value, It may be determined that the distance to the target object is close.

  The above embodiment is merely an example in all respects. The present invention is not construed as being limited to these descriptions. The present invention can be implemented in various other forms without departing from the spirit or main features thereof.

It is a block diagram of the distance image generation device of this embodiment. FIG. 10 is a diagram for explaining charge distribution in the image sensor 15. It is a figure for demonstrating the principle which produces | generates a distance image. It is a figure for demonstrating the thinning-out process which thins out the pixel (detection pixel) made into reading object. (A) This is an example of a pulse train used for reading all pixels, and (b) is an example of a pulse train (thinning pattern) used for thinning processing for thinning out pixels to be read (detection pixels). It is a flowchart of the operation example 1 of the distance image generation apparatus of this embodiment. It is an example of the surrounding environment of the distance image generation apparatus of this embodiment. (A) An example of a distance image generated at time t-dt, and (b) an example of a distance image generated at time t. (A) This is an example in which the number of pixels of the distance image shown in FIG. 8 (a) is counted for each distance range divided in advance, and (b) the number of pixels of the distance image shown in FIG. This is an example of counting for each distance range divided in advance. It is an example of the thinning pattern used in Operation Example 1. It is an example of the thinning pattern used in Operation Example 1. It is an example of the thinning pattern used in Operation Example 1. It is a flowchart of the operation example 2 of the distance image generation apparatus of this embodiment. It is an example of the thinning pattern used in the operation example 2. 10 is a flowchart of a modification of Operation Example 2. It is an example of the thinning pattern used in the operation example 2. It is a flowchart of the operation example 3 of the distance image generation apparatus of this embodiment. It is a figure for demonstrating the principle which calculates | requires the actual magnitude | size of the specific target object imprinted on the distance image. It is an example of the thinning pattern used in the operation example 3. It is a flowchart of the operation example 4 of the distance image generation apparatus of this embodiment. It is an example of the thinning pattern used in the operation example 4.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Distance image generation apparatus, 10 ... Light time-of-flight type distance image sensor (distance image sensor), 11 ... Light source, 14 ... Incident light, 15 ... Image sensor, 16 ... Control part, 17 ... Distance image generation part, 18 ... Normal image generation unit, 19 ... horizontal counter (H_count), 20 ... vertical counter, 21 ... memory, φ ... phase

Claims (6)

A light source that irradiates the target space with light;
A plurality of pixels comprising a combination of a photoelectric conversion element that receives reflected light irradiated from the light emitting source and reflected by an object in a target space and converts the reflected light into a charge, and a charge storage unit that stores the converted charge; An image sensor;
Reading means for reading the accumulated charge from the plurality of pixels;
A distance image generation unit that generates a distance image whose pixel value is a distance value based on the electric charge read by the reading unit;
Determining means for determining whether or not the distance to the object in the target space is approached;
Thinning means for thinning out pixels to be read by the reading means,
The distance image generating apparatus, wherein the thinning unit thins out pixels to be read by the reading unit when the determining unit determines that the distance to the object in the target space is close.   The thinning unit thins out pixels to be read by the reading unit based on a thinning pattern composed of a pulse train combining a pulse having a first pulse width and a pulse having a second pulse width shorter than the first pulse width. The distance image generating apparatus according to claim 1, wherein: Counting means for counting the number of pixels of the distance image generated by the distance image generation unit for each distance range divided in advance;
Comparing means for comparing the threshold value with the number of pixels belonging to the specific distance range counted by the counting means,
The said determination means determines with the distance with the target object in the said target space having approached, when the number of the pixels which belong to the said specific distance range exceeds a threshold value. Distance image generator.   The thinning-out means reads the pixels from the pixels other than the plurality of pixels, a pixel group corresponding to a specific object among the plurality of pixels, or a pixel group corresponding to a specific object among the plurality of pixels. 4. The distance image generating apparatus according to claim 1, wherein pixels to be read out are thinned out by means. Detecting means for detecting the size of a specific object included in the distance image generated by the distance image generating means;
Collating means for collating the size and shape database of the specific object detected by the detecting means,
When the specific object detected by the detecting means is registered in the shape database, the thinning-out means reads out from the pixel group corresponding to the specific object among the plurality of pixels. The distance image generating apparatus according to claim 1, wherein pixels to be read out are thinned out by the step. Estimation means for estimating a pixel corresponding to a boundary line of the specific object in the distance image generated at the next timing for the specific object included in the distance image generated by the distance image generation means; Prepared,
The thinning unit is a pixel to be read by the reading unit from pixels other than the pixel corresponding to the boundary line of the specific object in the distance image generated at the next timing estimated by the estimating unit. The distance image device according to claim 1, wherein the distance image device is thinned out.

Images