JP2012168049A - Distance image generation device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the accuracy of distance measurement independent of the condition within the photographing space when generating the distance image of the space using an optical flight time-type distance image sensor.SOLUTION: A method includes: using an irradiation light intensity image of a photographing space, the image generated at the same timing as a distance image to determine whether the photographing space has an area where the incident light (reflection light) is such great to influence a pixel value calculation of another pixel; if there is such area, controlling a light source so as to perform a division-irradiation, irradiating the area and other areas mutually independently; and performing the determination by comparing each pixel value of an irradiation intensity image with a predetermined threshold.

Description

  The present invention relates to a distance image generation technique, and more particularly to a distance image generation technique using an optical time-of-flight distance image sensor.

  There is a distance image generation device that uses a time-of-flight distance image sensor to generate a distance image of an object in an imaging space whose pixel value is the distance from the sensor. In the time-of-flight distance image sensor, the distance of the object is calculated for each pixel using the phase difference between the modulated light emitted from the light source and the reflected light of the modulated light from the object. The phase difference is calculated by converting the reflected light received through the lens by the imaging device prepared for each pixel into a charge amount corresponding to the light amount, and performing a predetermined calculation on the charge amount (for example, a patent) Reference 1).

JP 2008-241435 A

  A distance image generation apparatus using a time-of-flight distance image sensor calculates a distance value based on reflected light received for each pixel as described above. However, when an object with an extremely close distance is in the shooting target space, or when an object with extremely high reflectance is in the shooting target space, when strong reflected light is incident on the lens, depending on the performance of the lens such as resolving power, Reflected light may sneak into pixels other than the pixel that originally forms an image, and a phenomenon called flare may occur.

  When flare occurs, an accurate light amount for calculating the distance cannot be obtained, and the distance value calculated in accordance with the light amount is also inaccurate.

  The present invention has been made in view of the above circumstances, and a technique for improving distance measurement accuracy when generating a distance image of a shooting space using an optical time-of-flight distance image sensor regardless of the situation in the shooting space. The purpose is to provide.

  The present invention uses an irradiation light intensity image of the same imaging space generated at the same timing as the distance image, and there is an area where the incident light (reflected light) intensity is strong enough to affect the calculation of the pixel value of other pixels in the imaging space. It is discriminated whether or not there is, and if there is, divided irradiation for independently controlling irradiation to the region and irradiation to other regions is performed. The discrimination is performed by comparing each pixel value of the irradiation light intensity image with a predetermined threshold value.

  Specifically, a light source that irradiates the imaging space with modulated light, a plurality of photoelectric conversion elements that receive reflected light that is irradiated from the light source and reflected by an object in the imaging space, and converts the reflected light into charges, and the photoelectric conversion A plurality of charge storage units provided for each element, and a distribution unit that distributes the charges converted by the photoelectric conversion elements to the plurality of charge storage units in synchronization with the modulation of the light source. And distance image generating means for generating a distance image of the imaging space in which the pixel value is a distance value from the charges accumulated in the plurality of charge accumulation units at a predetermined time, and the pixel value is the irradiation light intensity An irradiation light intensity image generating means for generating an irradiation light intensity image of the imaging space, and whether the irradiation light intensity image has a pixel having a pixel value equal to or greater than a predetermined threshold value Judgment of whether or not And a light source control means for controlling the irradiation of the light source. When the light source control means determines that the determination means is present, the light source control means divides the imaging space into a plurality of unit irradiation areas. The distance image generation device is characterized in that the irradiation of the light source is controlled to irradiate the light.

  In addition, a light source that irradiates the imaging space with modulated light, a plurality of photoelectric conversion elements that receive reflected light that is irradiated from the light source and reflected by an object in the imaging space, and converts the reflected light into charges, for each of the photoelectric conversion elements A plurality of charge storage units provided, and a light distribution unit that distributes the charges converted by the photoelectric conversion elements to the plurality of charge storage units in synchronization with the modulation of the light source, A distance image generation method in a distance image generation apparatus comprising a distance image that generates a distance image of the imaging space in which a pixel value is a distance value from charges accumulated in the plurality of charge accumulation units at predetermined time intervals. A generation step, an irradiation light intensity image generation step for generating an irradiation light intensity image of the imaging space whose pixel value is the irradiation light intensity, and each image of the irradiation intensity image each time the irradiation light intensity image is generated. A determination step for determining whether or not there is a pixel having a pixel value equal to or greater than a predetermined threshold, and if the determination step determines that there is a pixel, the target space is divided into a plurality of irradiation regions for irradiation. And a light emission control step for controlling the light emission of the light source.

  According to the present invention, when a distance image of a shooting space is generated using a time-of-flight distance image sensor, the accuracy of distance measurement of an object in the entire shooting target space is increased regardless of the situation in the shooting space.

It is a block diagram of the distance image generation device of a first embodiment. It is explanatory drawing for demonstrating the principle of distance image generation. (A) And (b) is explanatory drawing for demonstrating the division | segmentation irradiation of 1st embodiment. It is explanatory drawing for demonstrating the light source control method of 1st embodiment. (A) is explanatory drawing for demonstrating the timing of the division | segmentation irradiation of 1st embodiment, (b) is the division | segmentation irradiation at the time of combining 1st embodiment and 2nd embodiment. It is explanatory drawing for demonstrating timing. It is explanatory drawing for demonstrating the image composition of 1st embodiment. It is a flowchart of the image generation process of 1st embodiment. It is a flowchart of the discrimination | determination process of 1st embodiment. It is a flowchart of the division | segmentation irradiation process of 1st embodiment. It is a flowchart of the image generation process of 2nd embodiment. (A) And (b) is a flowchart of the discrimination | determination process of 2nd embodiment.

<< First Embodiment >>
Hereinafter, embodiments to which the present invention is applied will be described. Hereinafter, in all the drawings for explaining the embodiments of the present invention, those having the same function are denoted by the same reference numerals, and repeated explanation thereof is omitted.

  First, the configuration of the distance image generation device used in the present embodiment will be described. FIG. 1 is a block diagram of a distance image generating apparatus 100 according to this embodiment.

  As shown in FIG. 1, the distance image generation device 100 uses an optical time-of-flight distance image sensor that generates a distance image using a phase difference between irradiation light and incident light, and includes a light source 110 and a light receiving unit 120. And an image generation unit 130 and a control unit 140.

  The light source 110 irradiates modulated light 111 (for example, infrared light or visible light modulated at high speed with a sine wave or rectangular wave; hereinafter referred to as modulated light) 111. As the light source 110, a device capable of high-speed modulation such as an LED or a laser is used. In addition, the light source 110 of the present embodiment includes a split irradiation function that divides and irradiates a shooting space for generating a distance image into a plurality of regions under the control of the control unit 140.

  The light receiving unit 120 is an image sensor that receives incident light 112 including reflected light, which is reflected from the object 200 in the imaging space, from the modulated light 111 emitted from the light source 110 and converts the incident light 112 into charges. In order to realize this, the light receiving unit 120 receives the incident light 112, converts a plurality of received light amounts into a charge amount, a charge accumulation unit that accumulates a charge amount obtained by the photoelectric conversion element, A distribution unit that distributes the charge to each charge storage unit. A lens (not shown) is disposed in front of the light receiving unit 120.

  A pixel is formed by this photoelectric conversion element. Therefore, each photoelectric conversion element is regularly arranged in association with each pixel of the distance image. In addition, the phase difference between the modulated light 111 and the incident light 112, which is distance information, is calculated using the charges accumulated in the charge accumulation unit. In order to calculate this phase difference, at least two pieces of phase information (charges accumulated in the charge accumulation unit) are necessary. For this reason, at least two charge storage units are provided for each photoelectric conversion element. Hereinafter, in the present embodiment, a case where four are used will be described as an example.

  The light source 110 and the light receiving unit 120 are synchronously controlled by the control unit 140. The light source 110 irradiates the target space with the modulated light 111 in accordance with the synchronization signal output from the control unit 140. In the light receiving unit 120, the photoelectric conversion element converts the incident light 112 including the reflected light reflected by the object 200 into an amount of electric charge according to the synchronization signal. The distribution unit distributes the obtained charge amount to each of the four charge storage units provided in association with the photoelectric conversion elements according to the synchronization signal. Here, every one period of modulation is divided into four equal parts, and these four charge accumulating units are distributed. The charge storage unit may store the charge amount itself, or may store data after AD conversion of this charge amount.

  The image generation unit 130 generates a distance image having a distance value as a pixel value and an irradiation light intensity image having an irradiation light intensity as a pixel value from the accumulated charge every time the light receiving unit 120 accumulates the charge for a predetermined period. To do. For this reason, the image generation unit 130 of the present embodiment includes a distance image generation unit 131 that generates a distance image of the imaging space and an irradiation light intensity image generation unit 132 that generates an irradiation light intensity image of the imaging space.

  The distance image generation unit 131 performs a predetermined calculation on the amount of charge distributed to each charge storage unit, and calculates the distance value of the imaging space corresponding to the position of each pixel from the phase difference between the modulated light 111 and the incident light 112. calculate.

  FIG. 2 is a diagram for explaining the principle of distance image generation. When the intensity of the modulated light 111 emitted from the light source 110 changes to draw a sinusoidal curve as shown in the figure, the intensity of the incident light 112 to the light receiving unit 120 also changes to draw a sinusoidal curve.

However, the modulated light 111 and the incident light 112 have a phase delay (phase difference φ) due to the flight time of the light traveling back and forth to the object 200. Since the speed c of light is known, the distance value D to the object can be obtained by the following formula (1) using the phase difference φ and the modulation frequency f.

Here, the phase difference φ between the modulated light 111 and the incident light 112 is defined as T1, T2, T3, T4 for each period obtained by dividing one period of the modulated light 111 into four, and the amount of charge accumulated in each period is defined as T1, T2, T3, T4. Assuming C1, C2, C3, and C4, they are represented by the following formula (2).

  Note that each period T1, T2, T3, T4 obtained by dividing one period into four is, for example, between 0 degrees and 90 degrees, between 90 degrees and 180 degrees, between 180 degrees and 270 degrees, and between 270 degrees and 0 Between degrees.

  The modulation frequency of the light source 110 is several tens of MHz. Therefore, one modulation period is about several tens of ns. For this reason, in order to obtain a distance image, a charge accumulation time of several hundred to several hundred thousand cycles is required. The distance image generation unit 131 of the present embodiment uses the respective charge amounts C1, C2, C3, and C4 accumulated in the charge accumulation unit at the charge accumulation time Δt interval, and for each pixel, the phase difference φ according to the equation (2) , And a distance value D to the object is determined according to the equation (1), and a distance image having the distance value as a pixel value is generated. The generated distance image is output from, for example, an external output terminal.

The irradiation light intensity image generation unit 132 performs a predetermined calculation on the charge amount distributed to each charge storage unit, and calculates the irradiation light intensity value of the imaging space corresponding to the position of each pixel. At this time, the respective charge amounts C1, C2, C3, and C4 used by the distance image generation unit 131 to calculate the distance value are used as they are. Using this, the illuminance light intensity image generation unit 132 calculates the irradiation light intensity value B for each charge accumulation time Δt according to the following equation (3) for each pixel, and uses the irradiation light intensity value as the pixel value. A light intensity image is generated.

In general, the average value A of the charges distributed to the four charge storage units according to the following formula (4) is used as the pixel value of the luminance image.

  The control unit 140 controls the operation of the entire distance image generation device 100. In the present embodiment, when an instruction to start processing (for example, power ON) is received from the user, modulated light is emitted from the light source 110 and incident light (reflected light) is received by the light receiving unit 120 and received at predetermined time intervals. A distance image and an irradiation light intensity image are generated from the incident light (reflected light). In addition to this control, the control unit 140 according to the present embodiment causes the light source 110 to perform divided irradiation when there is a region where incident light (reflected light) of a predetermined light amount or more is emitted in the imaging space. In order to realize this, a light source control unit 141, a determination unit 142, and a synthesis unit 143 are provided.

  Each time the irradiation light intensity image is generated, the determination unit 142 determines whether or not there is a pixel equal to or greater than a predetermined threshold in the irradiation light intensity image. In the present embodiment, when it is determined that there are pixels equal to or greater than the threshold, the division flag is validated.

  The light source control unit 141 controls the operation of the light source 110 according to the state of the division flag. When the division flag is invalidated, the light source 110 is controlled so as to continuously irradiate the entire imaging space with the modulated light. This is called normal irradiation. Further, when the division flag is validated, control is performed so as to perform divided irradiation in which the imaging space is divided into a plurality of areas (unit irradiation areas).

  FIG. 3A is an explanatory diagram for explaining divided irradiation according to the present embodiment. Here, as an example, a case where the imaging space 210 is divided into four unit irradiation areas 211, 212, 213, and 214 is illustrated. Note that a plane obtained by two-dimensionally projecting the imaging space (irradiation area) 210 corresponds to the image area of the distance image and the irradiation light intensity image.

  For example, when the light emitters used for the light source 110 are configured by LED elements arranged in an array, the light source control unit 141 realizes divided irradiation by driving each LED element independently. In this embodiment, irradiation is controlled for each LED element group corresponding to each unit irradiation region. In this case, not only the irradiation and non-irradiation control but also the irradiation intensity can be changed for each LED element group.

  In addition, when the light emitter used for the light source 110 is a point light source such as a laser, the light source control unit 141 causes the light source 110 to be dividedly irradiated by the scanning unit. FIG. 4 is a diagram for explaining the control at this time. As shown in the figure, the light source control unit 141 controls the scanning unit 320 to scan the light emitted from the point light source 110 in the x and y directions in the figure. As the scanning unit 320, a galvano mirror that rotates a mirror to scan the modulated light, a MEMS that vibrates the mirror and scans the modulated light, or the like is used.

  The scanning direction may be a uniaxial direction. In this case, as shown in FIG. 4, the light source control unit 141 controls the light source 110 using the diffusion plate 330 in addition to the scanning unit 320. The diffusing plate 330 is disposed so as to diffuse the light emitted from the light source 110 in a direction orthogonal to the scanning direction (FIG. 4: 340).

  Note that the division of the imaging space (irradiation region) 210 by divided irradiation is not limited to that shown in FIG. For example, as shown in FIG. 3B, the image may be divided in the x direction and the y direction. FIG. 3B illustrates an example in which the imaging space 210 is divided into four in the x direction and two in the y direction to form eight unit irradiation areas 221, 222, 223, 224, 225, 226, 227, and 228. To do.

  In the present embodiment, the light source control unit 141 controls the light source 110 to sequentially irradiate each unit irradiation region for each charge accumulation time Δt during the charge accumulation time Δt. FIG. 5A shows the timing of divided irradiation when the imaging space 210 is divided into four unit irradiation areas 211, 212, 213, and 214 for irradiation.

  The synthesizing unit 143 synthesizes a plurality of distance images obtained during the divided irradiation. FIG. 6 shows a state of image synthesis when the imaging space 210 is divided into four unit irradiation areas 211, 212, 213, and 214 for irradiation. Images obtained when the unit irradiation areas 211, 212, 213, and 214 are irradiated are denoted by 411, 412, 413, and 414, respectively. These are combined to generate an image 450.

  In image composition, pixel values of all images may be added, or pixel values of pixels corresponding to the irradiation area may be extracted from each image. The boundary pixel may be configured to use the larger pixel value of the irradiation light intensity image.

  The distance image generation apparatus 100 according to the present embodiment includes a CPU, a memory, and a storage device, and realizes the functions of the above-described units by executing a program stored in the storage device.

  Next, a flow of processing (image generation processing) at the time of image generation by the control unit 140 according to the present embodiment will be described.

  The control unit 140 controls the light source 110, the light receiving unit 120, and the image generation unit 130, and continues the process of generating a distance image and an irradiation light intensity image for each charge accumulation time Δt. In this embodiment, each time an irradiation light intensity image is generated, a determination process is performed to determine whether or not to perform divided irradiation, and the irradiation of the light source 110 is controlled according to the determination result.

  FIG. 7 is a processing flow of the image generation processing of this embodiment. When the control unit 140 receives an instruction to start processing (for example, power ON), the control unit 140 starts processing. First, the division flag indicating whether or not to divide is invalidated (step S1101).

  Then, the control unit 140 causes the light source 110 to emit modulated light, and causes the light receiving unit 120 to start receiving incident light (reflected light), photoelectric conversion, and charge accumulation (step S1102). At this time, the clock t for counting the charge accumulation time is initialized (t = 0).

  Next, the control unit 140 determines whether or not the division flag is valid (step S1103). If it is invalid, the light source control unit 141 performs normal irradiation (step S1104). In response to this instruction, the light source control unit 141 controls the light source 110 to perform normal irradiation. Then, the control unit 140 accumulates charges obtained by photoelectrically converting incident light (reflected light) in the light receiving unit 120 for a charge accumulation time Δt (steps S1105 and S1106), and causes the image generation unit 130 to move away from the charges. An image and an irradiation light intensity image are generated (step S1107).

  On the other hand, if the split flag is valid in step S1103, the control unit 140 performs split irradiation processing for causing the light source control unit 141 to perform split irradiation (step S1108). Details of the split irradiation process will be described later.

  When the irradiation light intensity image is generated, the control unit 140 causes the determination unit 142 to start determination processing for determining whether or not divided irradiation is necessary (step S1109). Details of the discrimination processing will also be described later.

  Thereafter, the control unit 140 repeats the processing from step S1103 until an end instruction (for example, power OFF) is received (step S1110).

  Next, the flow of determination processing by the determination unit 142 in step S1109 will be described. In the discrimination process, using the generated irradiation light intensity image, there is a region (high reflection region) in which the incident light (reflected light) intensity is strong enough to affect the calculation of the pixel value of other pixels in the imaging space 210. If there is, the division flag is validated so that the light source 110 is divided and irradiated. The determination is performed by comparing the pixel value with a predetermined threshold value for each pixel of the generated irradiation light intensity image.

  FIG. 8 is a processing flow of the discrimination processing of this embodiment. First, the determination unit 142 invalidates the division flag (step S1201). Then, for all the pixels, the pixel value is compared with a threshold value to determine whether there is a pixel having a pixel value equal to or greater than the threshold value (step S1202). If there is a pixel having a pixel value equal to or greater than the threshold, it is determined that split irradiation is necessary, and the split flag is validated (step S1203). Then, the process ends. On the other hand, if the pixel values of all the pixels are smaller than the threshold value, the division flag remains invalid and the process ends.

  As a result of the above determination processing, the division flag is validated when the irradiation light intensity image has a pixel value having a value equal to or greater than the threshold value, and the division flag is invalidated in other cases.

  Next, the flow of divided irradiation processing in step S1108 will be described. In the present embodiment, the control unit 140 causes the light source control unit 141 to perform divided irradiation when the division flag is valid. Upon receiving this instruction, the light source control unit 141 causes the light source 110 to perform divided irradiation.

  FIG. 9 is a process flow of the divided irradiation process of the present embodiment. Here, the imaging space (irradiation area) 210 is irradiated with N division (N is a natural number), and each unit irradiation area is represented as R (n) (n is a natural number satisfying 1 ≦ n ≦ N).

  When receiving the instruction to start the determination process, the determination unit 142 initializes n (n = 1) (step S1301). The light source control unit 141 controls the light source 110 to irradiate the nth unit irradiation region R (n) (step S1302). When the irradiation is started, the control unit 140 causes the light receiving unit 120 to receive incident light (reflected light) and accumulates charges after photoelectric conversion as usual. Then, for each charge accumulation time Δt, the image generation unit 130 is caused to generate a distance image and an irradiation light intensity image (steps S1303, S1304, and S1305).

  Then, the light source control unit 141 repeats the processing from step S1302 to S1305 until irradiation is completed for all the regions while changing the unit irradiation region R (n) in order (steps S1306 and S1307).

  When the irradiation of all unit irradiation regions R (n) is completed and N distance images and irradiation light intensity images are generated, the control unit 140 causes the combining unit 143 to combine them (image combining; step S1308). Then, an image in which the entire photographing space 210 is irradiated is generated.

  After the image synthesis, the process returns to the image generation process of FIG. 7, and the distance image is output from, for example, an external output terminal, and the irradiation light intensity image is used for the determination process in step S1109.

  As described above, according to the present embodiment, the presence / absence of the high reflection region is determined using the pixel value of the irradiation light intensity image in the same range as the imaging space in which the distance image is generated. When the irradiation light intensity image generated immediately before has a high reflection area, the imaging range (irradiation area) is divided, and modulated light is irradiated at different timings for each unit irradiation area. Then, a distance image is generated by combining images generated from the obtained incident light (reflected light).

  Therefore, even when there is a highly reflective region, the influence is limited to the pixel region corresponding to the unit irradiation region including the region, and it is possible to prevent all pixels from being affected. For this reason, if it sees as the whole image, the distance image from which the distance value was calculated with high precision can be obtained.

<< Second Embodiment >>
Next, a second embodiment to which the present invention is applied will be described. In the first embodiment, when there is incident light (reflected light) of a predetermined intensity or higher, the shooting range (irradiation area) is divided and irradiated at different timings, and the obtained images are combined to generate a distance image. is doing. In this embodiment, when there is incident light (reflected light) having a predetermined intensity or higher, the imaging range (irradiation area) is divided and irradiated, but the timing of irradiating each divided irradiation area is the same, and only the amount of irradiation light is emitted. To change.

  The configuration of the distance image generation device 100 of this embodiment is basically the same as that of the first embodiment. However, as described above, the configuration of the control unit 140 is different in order to change the amount of light irradiated to each divided region while simultaneously irradiating each divided region. In addition, the light source 110 can change the amount of irradiation light for each unit irradiation region.

  As the realizable light source 110, for example, it is assumed that light emitters such as LEDs are arranged in an array. It is sufficient that the individual light emitters can irradiate different irradiation regions and the light emission amount can be controlled independently.

  The determination unit 142 according to the present embodiment determines whether or not the irradiation light intensity image includes a pixel that is equal to or greater than a predetermined threshold every time an irradiation light intensity image is generated, and also determines whether or not there is a pixel that exceeds the threshold. The corresponding unit irradiation area is specified. In the present embodiment, when the determination unit 142 determines that there is a pixel that is equal to or greater than the threshold value, in the present embodiment, the information for specifying the unit irradiation region corresponding to the region where the pixel is present is changed without changing the light amount. Output as an instruction.

  The light source control unit 141 of this embodiment controls the operation of the light source 110 according to the presence or absence of a light amount change instruction. When the light amount change instruction is not output, the entire imaging space is controlled to be irradiated with the same light amount. When the light amount change instruction is output, the light source 110 is controlled to reduce the irradiation light amount to the unit irradiation region. In this embodiment, what controls the light source 110 to irradiate the entire photographing space with the same light amount is called normal irradiation, and control is performed so that at least one unit irradiation region is irradiated with a light amount different from other unit irradiation regions. This is called split irradiation.

  The reduction in the amount of light is realized by reducing the intensity of the modulated light emitted from the light source 110. The amount of light to be reduced is determined in advance.

  In the present embodiment, since the imaging space is irradiated at the same timing even during divided irradiation, the combining unit 143 is not provided.

  Hereinafter, a flow of image generation processing by the control unit 140 of the present embodiment will be described. FIG. 10 is a processing flow of the image generation processing of the present embodiment.

  When control unit 140 receives a processing start instruction (for example, power ON), it starts processing. The control unit 140 causes the light source 110 to emit modulated light, and causes the light receiving unit 120 to start receiving incident light (reflected light), photoelectric conversion, and charge accumulation (step S2101). At this time, the clock t for counting the charge accumulation time is initialized (t = 0). At this time, the control unit 140 causes the light source control unit 141 to perform normal irradiation control. In response to this, the light source control unit 141 controls the light source 110 to irradiate the entire imaging space (all unit irradiation areas) with the same amount of light.

  Then, the control unit 140 determines whether or not there is a light amount change instruction (step S2102). If there is a light amount change instruction, the control unit 140 notifies the light source control unit 141 to that effect. In response to this, the light source control unit 141 changes (reduces) the light amount of the unit irradiation region for which the light amount change instruction is output (step S2103). On the other hand, when the light amount change instruction is not output, the irradiation state of the light source 110 is not changed.

  When charge is accumulated for the charge accumulation time Δt (step S2104), the control unit 140 initializes the time counter t (step S2105), and causes the image generation unit 130 to generate a distance image and an irradiation light intensity image (step S2104). S2106).

  When the irradiation light intensity image is generated, the control unit 140 causes the determination unit 142 to start determination processing for determining whether or not divided irradiation is necessary (step S2107). Details of the discrimination processing of this embodiment will be described later.

  Thereafter, the control unit 140 repeats the processing from step S2102 until an end instruction (for example, power OFF) is received (step S2108).

  Next, the determination process by the determination unit 142 in step S2107 of the present embodiment will be described. FIG. 11A is a processing flow of the discrimination processing of the present embodiment. In the present embodiment, it is determined whether or not there is a highly reflective region in the pixel region corresponding to each unit irradiation region of the irradiation light intensity image. If there is, the corresponding unit irradiation region is specified as a light amount change instruction. Output information.

  Here, the imaging range (irradiation region) 210 is irradiated with N division (N is a natural number), and each unit irradiation region is represented as R (n) (n is a natural number satisfying 1 ≦ n ≦ N).

  When receiving the instruction to start the determination process, the determination unit 142 initializes n (n = 1) (step S2201). And it is discriminate | determined whether the pixel value Ps is more than the predetermined threshold value Ts in the pixel area of the irradiation light intensity image corresponding to area | region R (n) (step S2202). Only when there is, the unit irradiation region R (n) is output as a light amount change instruction for the unit irradiation region R (n) (step S2203).

  The processes in steps S2202 and S2203 are repeated for all areas (steps S2204 and S2205).

  As a result of the determination process described above, when a light amount change instruction is output for one or more divided irradiation regions R (n), the light source control unit 141 receives the instruction and controls the irradiation of the light source 110. The light receiving unit 120 receives incident light (reflected light) due to controlled modulated light irradiation, and the image generation unit 130 generates a distance image and an irradiation light intensity image from the accumulated charges.

  As described above, according to the present embodiment, the imaging range (irradiation region) is divided into a plurality of unit irradiation regions, and the high reflection region is irradiated with modulated light while reducing the amount of light. The amount of light in each unit irradiation region is controlled using the pixel value of the irradiation light intensity image in the same range as the shooting range in which the distance image is generated. The control determines the presence or absence of a pixel that has received incident light (reflected light) that is so intense as to affect the calculation of surrounding pixel values in the irradiation light intensity image. The amount of light applied to the unit irradiation area corresponding to the included area is reduced.

  Therefore, according to the present embodiment, even when there is a highly reflective area within the imaging range, the modulated light is irradiated with the light amount reduced only in the area. Accordingly, the light amount can be adjusted to an optimum value in accordance with the imaging range (irradiation region), so that the distance value calculation accuracy is increased and a highly accurate distance image can be obtained. For this reason, according to the present embodiment, a highly accurate distance image can be obtained regardless of the shooting environment.

  In the above embodiment, the light amount change instruction is output so as to reduce the light amount only in a certain region having pixels having pixel values equal to or greater than the threshold value, but the light amount change instruction is not limited to this. For example, when the total pixel value of the pixel region corresponding to the unit irradiation region R (n) to be determined is smaller than the threshold value, the irradiation light amount to the region may be increased stepwise up to the maximum value.

  FIG. 11B shows a processing flow of discrimination processing by the discrimination unit 142 in this case. Similarly to FIG. 11A, when the determination unit 142 receives an instruction to start the determination process, n is initialized (n = 1) (step S2201). And it is discriminate | determined whether the pixel value Ps is more than the predetermined threshold value Ts in the pixel area of the irradiation light intensity image corresponding to area | region R (n) (step S2202). If there is, an instruction to reduce the light amount of the unit irradiation region R (n) and the unit irradiation region is output as a light amount change instruction for the unit irradiation region R (n) (step S2203).

  On the other hand, if there is no pixel having a pixel value equal to or greater than the threshold value Ts in the irradiation light intensity image in step S2201, the irradiation light intensity to the region R (n) at that time is different from that in FIG. It is determined whether or not it is the maximum (step S2206), and if it is the maximum, as it is, if not the maximum, an instruction to increase the unit irradiation region R (n) and the light amount of the unit irradiation region is output (step S2207). ).

  In this case, not only the information specifying the corresponding unit irradiation area but also information indicating whether to decrease or increase is simultaneously output as the light quantity change instruction. The reduction amount and the increase amount are determined in advance.

  By configuring in this way, as described above, the SN ratio is improved by the amount of light increased, and the accuracy of the distance image is improved without receiving strong incident light (reflected light) that affects others. Can be improved.

  Note that the method of the present embodiment and the method of the first embodiment may be combined. That is, when the irradiation light intensity image generated immediately before has a high reflection area, the imaging range (irradiation area) is divided, and the modulated light is irradiated at different timings for each unit irradiation area as in the first embodiment. As in the second embodiment, the unit irradiation region corresponding to the high reflection region is irradiated with modulated light with a reduced amount of light.

  The image generation process in this case is basically the same procedure as the image generation process of the first embodiment shown in FIG.

  On the other hand, the discrimination process is basically the same procedure as the discrimination process of the second embodiment shown in FIG. 11 (a) or FIG. 11 (b). However, as in the discrimination process of the first embodiment shown in FIG. 8, the division flag is invalidated at the start of the discrimination process. If it is determined in step S2202 that there is a pixel equal to or greater than the threshold Ts, the division flag is validated. To do.

  The divided irradiation process is basically the same procedure as the divided irradiation process of the first embodiment shown in FIG. However, the presence or absence of a light quantity change instruction is confirmed for each unit irradiation region R (n) before step S1302, and if there is, the light quantity is changed (reduced or increased).

  The light source control unit 141 may realize the reduction or increase of the light amount by reducing or increasing the intensity of the modulated light from the light source 110 that irradiates the corresponding region, or by shortening or extending the irradiation time. May be. FIG. 5B shows a state of divided irradiation when it is realized by changing the irradiation time.

  By comprising in this way, the incident light (reflected light) from a highly reflective area | region can be reduced. Therefore, even if there is a high reflection region, not only can the influence on other regions be prevented, but also the strong reflected light itself in the high reflection region can be reduced. For this reason, flare can be suppressed even in the region, and the accuracy of the distance value of the obtained distance image is improved.

  In addition to the above-described effects, if the light intensity is sequentially increased up to the maximum value for regions other than the region where the irradiation light intensity value is greater than or equal to the threshold value, a sufficient SN ratio can be obtained and further calculated. The accuracy of the distance value is improved.

  In each of the above embodiments, the irradiation light intensity image is generated by performing the calculation of Expression (3) using the charge used when generating the distance image, but the irradiation light intensity image is generated. The generation method is not limited to this. For example, the pixel value may be obtained by taking the difference in the amount of incident light (reflected light) between light irradiation and non-irradiation.

  100: distance image generation device, 110: light source, 110: point light source, 111: modulated light, 112: incident light, 120: light receiving unit, 130: image generation unit, 131: distance image generation unit, 132: irradiation light intensity image Generation unit 140: Control unit 141: Light source control unit 142: Discrimination unit 143: Composition unit 200: Object 210: Shooting space 211: Unit irradiation area 212: Unit irradiation area 213: Unit irradiation area 214: unit irradiation region, 221: unit irradiation region, 222: unit irradiation region, 223: unit irradiation region, 224: unit irradiation region, 225: unit irradiation region, 226: unit irradiation region, 227: unit irradiation region, 228 : Unit irradiation area, 320: Scanning means, 330: Diffusion plate, 411: Image 412: Image 413: Image 414: Image 450: Image

Claims (7)

A light source that irradiates the imaging space with modulated light, a plurality of photoelectric conversion elements that receive reflected light that is irradiated from the light source and reflected by an object in the imaging space, and converts the reflected light into charges. A plurality of charge storage units, and a distribution unit that distributes the charges converted by the photoelectric conversion elements to the plurality of charge storage units in synchronization with the modulation of the light source, and a predetermined time. Distance image generation means for generating a distance image of the imaging space whose pixel value is a distance value from the charges accumulated in the plurality of charge storage units, and irradiation of the imaging space whose pixel value is an irradiation light intensity An irradiation light intensity image generating means for generating a light intensity image, and a distance image generating device comprising:
A discriminating means for discriminating whether or not there is a pixel having a pixel value equal to or greater than a predetermined threshold in the irradiation light intensity image;
A light source control means for controlling the irradiation of the light source,
The light source control unit controls the irradiation of the light source so as to divide the imaging space into a plurality of unit irradiation areas and irradiate the modulated light when it is determined that the determination unit is present. Generator. The distance image generating device according to claim 1,
The said light source control means controls irradiation of the said light source so that each said unit irradiation area may be irradiated in order for every predetermined irradiation time. The distance image generation apparatus characterized by the above-mentioned. The distance image generating device according to claim 2,
Further comprising a combining means for combining a plurality of distance images,
The distance image generation means generates a distance image every time the light source irradiates the unit irradiation region for the predetermined time,
The synthesizing unit synthesizes a distance image obtained by irradiating the unit irradiation area for the entire photographing space. The distance image generating device according to claim 1,
The determination unit specifies a unit area corresponding to a pixel having a pixel value equal to or greater than a predetermined threshold as a light amount reduction area,
The light source control means irradiates each unit irradiation region with modulated light simultaneously, and makes the intensity of the modulated light irradiated to the unit irradiation region smaller than the intensity irradiated to other unit regions. A distance image generation device characterized by controlling irradiation. The distance image generating device according to any one of claims 1 to 3,
The determination unit specifies a unit area corresponding to a pixel having a pixel value equal to or greater than a predetermined threshold as a light amount reduction area,
The light source control means makes the light amount of the modulated light irradiated to the light amount reduction region smaller than the light amount irradiated to other unit regions. The distance image generating device according to claim 2 or 3,
The determination unit specifies a unit area corresponding to a pixel having a pixel value equal to or greater than a predetermined threshold as a light amount reduction area,
The light source control means makes the irradiation time for irradiating the light quantity reduction region shorter than the irradiation time for irradiating other unit regions. A light source that irradiates the imaging space with modulated light, a plurality of photoelectric conversion elements that receive reflected light that is irradiated from the light source and reflected by an object in the imaging space, and converts the reflected light into charges. A distance comprising: a plurality of charge storage units; and a distribution unit that distributes the charges converted by the photoelectric conversion elements to the plurality of charge storage units in synchronization with the modulation of the light source. A distance image generation method in an image generation apparatus,
A distance image generation step of generating a distance image of the imaging space in which a pixel value is a distance value from charges accumulated in the plurality of charge accumulation units at predetermined time intervals;
An irradiation light intensity image generation step for generating an irradiation light intensity image of the imaging space in which the pixel value is the irradiation light intensity;
A determination step of determining whether each pixel of the irradiation intensity image includes a pixel having a pixel value equal to or higher than a predetermined threshold every time the irradiation light intensity image is generated;
A light emission control step of controlling the light emission of the light source so as to divide and irradiate the target space into a plurality of irradiation areas when the determination step determines that there is a distance image generating method. .

Images