JP2000241131A - Range finder device and imaging device - Google Patents

Abstract

(57) [Problem] To provide a range finder device capable of stably obtaining highly accurate three-dimensional position information. SOLUTION: A light source unit 10 projects light onto a subject 1 for three-dimensional measurement. The camera section 20 receives the reflected light of the projection light from the light source section 10 on the subject 1. Distance measuring sensor 1
01 measures the approximate distance to the subject 1, and the exposure control unit 1
Reference numeral 02 controls the light output of the light source unit 10 and the opening / closing operation of the shutter unit 104 based on the approximate distance measured by the distance measurement sensor 101. Thereby, for example, even if the subject 1 moves, the intensity of the projection light is controlled in accordance with the movement, so that highly accurate three-dimensional position information can always be obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a range finder device for measuring three-dimensional position information of a subject and an imaging device.

[0002]

2. Description of the Related Art FIG. 22 is a diagram showing a configuration example of a conventional range finder device. In FIG. 22, a light source unit 10 includes first and second light sources 11A and 11B, and filters 12A and 1B are provided on the light projection side of each of the light sources 11A and 11B.
2B are provided. Output lights from the first and second light sources 11A and 11B are combined by a half mirror 13, and the combined light is
5 is projected onto the subject 1. Each light source 11A, 11
The output wavelength of B is set in the infrared region.

FIG. 23 shows a filter 12A and a filter 1
It is a figure showing an example of the characteristic of 2B. As shown in FIG. 23A, the filters 12A and 12B are configured to select and pass light having different wavelengths from each other. Alternatively, as shown in FIG. 23B, light is separated according to the level of the wavelength.

The camera section 20 has first and second image pickup devices 22A and 22B for distance measurement, and a light-receiving side of each of the image pickup devices 22A and 22B has a filter 12A and a filter provided in the light source section. A filter 23A and a filter 23B having the same characteristics as those of the filter 12B are provided. Thereby, each of the image sensors 22A, 22B
Can receive the output light of the first light source 11A and the output light of the second light source 11B separately from the reflected light of the subject 1. Further, the camera unit 20 includes a third image sensor 22C that receives light in the visible region, and a color signal processing unit 27 generates a texture image (color image) of the subject 1 from an output signal of the image sensor 22C. .

FIG. 24A shows the relationship between the light intensity of the projection light and the projection angle θ. As shown in FIG. 24A, the light source controller 16 adjusts each of the light sources 11A and 11B in accordance with a change in the projection angle of the combined light by the rotating mirror 15.
Are controlled. Accordingly, the light intensity ratio IA / IB changes as shown in FIG. FIG.
As can be seen from (b), there is a one-to-one correspondence between the light intensity ratio IA / IB and the projection angle θ, and the light intensity ratio IA / IB
Is obtained, the projection angle θ at that time can be uniquely specified. When the projection angle θ is specified, the distance Z to the subject can be obtained as shown in FIG.

The operation of the conventional range finder shown in FIG. 22 will be described below.

First, in the light source unit 10, the first and second light sources 11A and 11B output light. These output lights are combined by the half mirror 13 through the filters 12A and 12B, and the combined light is processed by the slit 14 into a slit light elongated in the vertical direction. The slit light is reflected by the rotating mirror 15 controlled by the rotation control unit 17 and is projected on the subject 1 side.

[0008] The reflected light of the light projected on the subject 1 is incident on the camera unit 20, and the image pickup devices 22A, 22B, 22C
Receives reflected light via the lens 21 and the half mirrors 24A and 24B. At this time, the light received by the first and second imaging elements 22A and 22B becomes light of a single wavelength separated from the combined light by the filters 23A and 23B provided on the light receiving side.

The first light source signal processing unit 25 receives the output of the first image pickup device 22A, outputs an image signal based on a reflected light component of the light output from the first light source 11A, and outputs a second light source signal processing unit. 26 receives the output of the second image sensor 22B,
An image signal based on a reflected light component of the light output from the second light source 11B is output. The distance calculation unit 30 calculates the light intensity ratio for each pixel using the image signals output from the first light source signal processing unit 25 and the second light source signal processing unit 26.
Then, the projection angle θ is specified for each pixel based on the correspondence as shown in FIG.

Here, each pixel position of the image sensor and the lens 2
The line-of-sight angle φ (see FIG. 22) determined by the line of sight formed by the center of 1 and the optical axis of the lens 21 is 1 pixel
It is a known value corresponding to one to one. The distance D between the lens 21 and the center of rotation of the rotating mirror 15 (see FIG. 22) is also known.

Therefore, the distance calculation unit 30 substitutes the projection angle θ, the line-of-sight angle φ, and the distance D into the following equation for each pixel according to the principle of triangulation, and calculates the distance on the subject 1 corresponding to each pixel. The distance Z between the point and the camera unit 20 can be calculated.

Z = (tan θ · tan φ / tan θ−tan φ) · D (1) In this manner, three-dimensional position information of each point on the subject 1 can be obtained.

Further, a texture image (color image) of the subject 1 is obtained by the color signal processing section 27 from the output of the third image sensor 22C together with the three-dimensional position information of the subject 1.

[0014]

However, the conventional range finder has the following problems.

First, in a conventional range finder device,
Adjustment of the light output of the light source unit 10 is performed by the user, and in practice, appropriate settings have been made based on the experience of the user or by trial and error. For this reason, for example, when a beginner uses a range finder device, it is difficult to appropriately set the optical output, so that highly accurate distance information cannot always be obtained. Also,
When the subject moves, the light output needs to be readjusted each time, which causes a problem that it takes time and labor. Furthermore, if the subject is too close,
If the output light of the light source unit is excessively strong, the subject may be adversely affected.

In a conventional range finder,
There were other problems as well. FIG. 25 is a diagram illustrating signal levels of video signals when performing three-dimensional measurement. In FIG.
L is the signal level of the entire video signal, LA is the signal level of the reflected light component, and LB is the signal level of the image component (background light) of the subject. In order to improve the accuracy of three-dimensional position information,
The light output of the light source unit 10 may be set high so that the S / N ratio of the signal level LA of the reflected light component becomes higher. However, since the dynamic range of the camera section 20 is determined, there is a limit in increasing the signal level LA of the reflected light component. On the other hand, in order to obtain a normal two-dimensional image, the signal level LB of the image component of the subject also needs to have a certain level. Therefore, the S / N ratio of the reflected light component of the subject cannot be sufficiently improved, which has hindered the improvement of the accuracy of the three-dimensional position information.

Furthermore, in the field of computer vision, for example, a technique is generally known in which a background image and a foreground image are distinguished from subject distance information, and only the foreground image is cut out using the result. However, in a system such as a videophone, although a function of cutting out a person's face (foreground) from the background is desired, it is not always necessary to obtain distance information of a subject. For this reason, it is preferable that a function capable of distinguishing a background image from a foreground image can be realized without using distance information of a subject.

In view of the above problems, an object of the present invention is to provide a range finder device which can stably obtain high-accuracy three-dimensional position information and has high security for a subject.

Another object of the present invention is to provide an image pickup apparatus capable of distinguishing a background image from a foreground image without using distance information of a subject.

[0020]

Means for Solving the Problems In order to solve the above-mentioned problems, a solution taken by the invention of claim 1 is to measure three-dimensional position information of the object by receiving reflected light of light projected on the object. A light source unit that projects the light, a camera unit that receives reflected light from a subject, and a light output unit of the light source unit and an exposure condition of the camera unit based on distance information of the subject. And a control unit for controlling at least one of the above.

According to the first aspect of the present invention, at least one of the light output of the light source unit and the exposure condition of the camera unit is controlled based on distance information of the subject. For this reason, even if the subject moves, for example, the intensity of the projection light or the level of the light reception signal is controlled in accordance with the movement, so that highly accurate three-dimensional position information can always be obtained. Also,
Even when the subject is too close, it is possible to operate so that the projection light does not adversely affect the subject.

According to a second aspect of the present invention, the range finder device of the first aspect further includes a distance measuring sensor for measuring a distance to the object, and the control unit outputs an output of the distance measuring sensor to the distance of the object. Control shall be performed using the information.

According to a third aspect of the present invention, the range finder device of the first aspect further includes a distance calculating unit that obtains a distance image from a video signal output from the camera unit, and the control unit controls the distance calculating unit. Control is performed using the obtained distance image as the distance information of the subject.

According to a fourth aspect of the present invention, there is provided a range finder for measuring three-dimensional position information of a subject by receiving reflected light of the light projected on the subject, wherein the light is projected. A light source unit, a camera unit that captures reflected light from a subject, and at least one of a light output of the light source unit and an exposure condition of the camera unit based on level information of a video signal output from the camera unit. And a control unit for controlling the

According to the present invention, at least one of the light output of the light source unit and the exposure condition of the camera unit is controlled based on the level information of the video signal output from the camera unit. For this reason, even if the subject moves, for example, the intensity of the projection light or the level of the light reception signal is controlled in accordance with the movement, so that highly accurate three-dimensional position information can always be obtained.

According to a fifth aspect of the present invention, the control unit in the range finder device according to the first or fourth aspect determines from the distance information or the level information that the distance to the subject is greater than or equal to a first threshold. In this case, the light output of the light source unit is set to be relatively large, while when it is determined that the light output is equal to or less than the second threshold value, the light output of the light source unit is set to be relatively small.

According to a sixth aspect of the present invention, in the range finder according to the first or fourth aspect, the exposure condition of the camera section is set by at least one of the aperture, the sensitivity of the image sensor, and the shutter speed. I do.

According to a seventh aspect of the present invention, the range finder device of the first or fourth aspect is configured to be openable and closable, and includes a shutter unit for blocking the light emitted from the light source unit when the range finder device is in a closed state. The switching of the open / close state of the shutter unit is controlled.

The invention according to claim 8 is a light source for projecting the light as a range finder device which receives reflected light of the light projected on the subject and measures three-dimensional position information of the subject. And the camera unit capable of receiving a reflected light of the projection light from the light source unit on the subject and capturing a two-dimensional image, and when the light source unit projects the light for three-dimensional measurement, The signal level of the image component of the subject is set to 2 so that the signal level of the reflected light is sufficiently high.
And a control unit that suppresses the temperature to a lower level than when capturing a two-dimensional image.

According to the eighth aspect of the present invention, the signal level of the image component of the object is suppressed to be lower in three-dimensional measurement than in two-dimensional image pickup. Therefore, the S / N of the signal level of the light reflected by the subject of the light projected from the light source unit
The ratio is improved, and the image quality of the two-dimensional image is not deteriorated. Therefore, the accuracy of the three-dimensional position information is improved.

According to a ninth aspect of the present invention, in the range finder device according to the eighth aspect, the camera section includes a filter section for adjusting an amount of light incident on the camera section, and the control section includes a light section of the filter section. The transmittance is relatively low when performing three-dimensional measurement, and relatively high when capturing two-dimensional images.

According to a tenth aspect of the present invention, in the range finder device according to the ninth aspect, the filter section has a liquid crystal element, and the light transmittance can be controlled by a voltage applied to the liquid crystal element. I do.

According to an eleventh aspect of the present invention, the control unit in the range finder device of the eighth aspect controls the exposure condition of the camera unit.

In the twelfth aspect of the present invention, the exposure condition of the camera section in the range finder device of the eleventh aspect is
It is set by at least one of the aperture, the sensitivity of the image sensor, and the shutter speed.

According to a thirteenth aspect of the present invention, there is provided an image pickup apparatus, comprising: a light source section for projecting light and changing a light characteristic of the projected light in accordance with a projection direction; A camera unit that receives the reflected light of the projection light from the light source unit at the subject, and a foreground / background distinguishing unit that distinguishes between the foreground and the background in the two-dimensional image based on the light characteristics of the reflected light at the subject. It is provided with.

According to the thirteenth aspect, in the two-dimensional image, the foreground and the background are distinguished based on the light characteristics of the light projected from the light source and reflected by the subject. Therefore, the foreground and the background can be distinguished without using the distance information.

According to a fourteenth aspect of the present invention, the imaging device according to the thirteenth aspect includes a cutout unit that cuts out a foreground or a background from the two-dimensional image using a result of the discrimination by the foreground / background discriminating unit. .

According to a fifteenth aspect of the present invention, in the light source section of the imaging device according to the thirteenth aspect, the light intensity characteristic varies according to the projection direction, and the variation patterns thereof are different from each other.
And the second light is projected, and the foreground / background distinguishing unit distinguishes between the foreground and the background by using a light intensity ratio of the reflected light of the first light and the reflected light of the second light. Shall be performed.

According to a sixteenth aspect of the present invention, the light source unit in the imaging device according to the thirteenth aspect projects light whose light intensity characteristic changes according to a projection direction, and the foreground / background discriminating unit includes the light source. The foreground and the background are distinguished by using the light intensity of the reflected light.

According to a seventeenth aspect of the present invention, the imaging apparatus according to the thirteenth aspect includes a threshold value determining unit for determining a threshold value as a reference for distinguishing between the foreground and the background in accordance with the subject.

In the eighteenth aspect of the present invention, the threshold value deciding unit in the imaging device according to the seventeenth aspect determines the threshold value based on a distribution of light characteristics of each pixel of the reflected light from the subject.

According to a nineteenth aspect of the present invention, the threshold value determining unit in the imaging device of the seventeenth aspect determines the threshold value using the surface reflectance of the subject.

According to a twentieth aspect of the present invention, the imaging device according to the nineteenth aspect includes a distance measuring sensor for measuring a distance to the subject, and the threshold value determining unit uses the measurement result of the distance measuring sensor to detect the subject. The surface reflectance shall be determined.

[0044]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 is a diagram showing a configuration of a range finder device according to a first embodiment of the present invention. In FIG. 1, components common to those in the conventional configuration shown in FIG. 22 are denoted by the same reference numerals as those in FIG. 22, and detailed description thereof will be omitted.

In FIG. 1, the distance measuring sensor 101 is
The approximate distance (average distance) between the present rangefinder device and the subject 1 is measured. This distance measuring sensor 10
1 is realized by a distance sensor or the like using reflection of ultrasonic waves. The exposure control unit 102 as a control unit controls the operations of the light source controller 16 and the shutter unit 104 according to the distance information output from the distance measurement sensor 101. The light source controller 16 adjusts the light output (light amount) of the light source unit 10. The shutter unit 104 is configured to be openable and closable, and is arranged on the front surface of the light source unit 10 on the light projection side. When it is closed according to a command from the exposure control unit 102, the light irradiation path is shut off and the projection light of the light source unit 10 is shut off.

The operation of the range finder shown in FIG. 1 will be described with reference to FIG. 2A is a diagram showing the relationship between the distance L to the subject 1 and the light output of the light source unit 10, and FIG. 2B is a diagram showing the relationship between the distance L to the subject 1 and the shutter unit 1.
It is a figure which shows the relationship with the operation of 04.

First, the distance measuring sensor 101 measures the distance to the subject 1, and outputs the distance measurement result L to the exposure control unit 1.
Send to 02. Here, light projection from the light source unit 10 is not performed. The exposure control unit 102 determines whether the subject 1 exists in the first area (close distance), the second area (intermediate distance), or the third area (long distance) based on the transmitted distance measurement result L. With the first and second reference values l
The determination is made based on 1,12. That is, the determination here is performed as follows. (1) First area when 0 ≦ L ≦ l1 (2) Second area when l1 <L <l2 (3) Third area when l2 ≦ L

The exposure control section 102 operates as follows according to the area where the subject 1 exists.

(1) When the subject 1 exists in the first area In this case, since the subject 1 is too close to the present rangefinder device, irradiation of the subject 1 with light is avoided. Accordingly, the exposure control unit 102 sets the light output of the light source unit 10 to be relatively small as shown in FIG. 2A, and closes the shutter unit 104 as shown in FIG. The light is not applied to the subject 1. However, at this time, it is preferable not to completely turn off the light output of the light source unit 10. In this case, the light output of the light source unit 10 can be started up more quickly than in the case where the light source unit 10 is completely turned off.

Here, the reason why the irradiation of the subject 1 with light is avoided is to protect the subject 1 from the irradiated light power. In particular, when the light from the light source unit 10 is a laser beam, when the subject 1 is at a close distance, the applied energy density increases, and there is a possibility that the subject 1 may have some adverse effect.

In this case, since no light is projected on the subject 1, measurement of three-dimensional position information is not executed. In order to execute the measurement of the three-dimensional position information, it is necessary to move the subject 1 away from the range finder device and to an appropriate position.

(2) When Subject 1 Exists in Second Region In this case, the exposure control unit 102 opens the shutter unit 104 as shown in FIG.
As shown in (a), the light source unit 1 is controlled so that the level of the image signal output from the camera unit 20 falls within an appropriate range.
0 light output is controlled. This makes it possible to prevent a light quantity shortage or an excessive light quantity, and to always maintain a high S / N ratio for the reflected light component, so that it is possible to measure highly accurate three-dimensional position information.

(3) When Subject 1 Exists in Third Region In this case, since the subject 1 is far away from the present rangefinder device, it is necessary to project as strong light from the light source unit 10 as possible. Therefore, the exposure control unit 102
As shown in FIG. 2B, the shutter unit 104 is opened, and as shown in FIG. 2A, the light output of the light source unit 10 is relatively strong, and preferably maximized.

When the subject 1 exists in the second or third area, after the operations described in the above (2) and (3), the same three-dimensional position information measuring operation as in the related art is performed.

Here, the distance to the subject 1 is set to 3
Although the light output of the light source unit 10 is adjusted for each type of area, the method of adjusting the light output is not limited to this. For example, a first reference value l1 and a second reference value l2
May be set to the same value, and light output may be performed in two types of regions. Alternatively, the light output may be adjusted more precisely by further dividing the area.

Further, instead of adjusting the light output of the light source unit 10, or in addition to adjusting the light output of the light source unit 10, the exposure condition of the camera unit 20 may be adjusted. The control of the exposure condition of the camera unit 20 is realized by controlling, for example, the aperture, the sensitivity of the image sensor, or the shutter speed. For example, in the second area, the light output of the light source unit 10 is set to 100%, and the exposure condition of the camera unit 20 is controlled so that the level of the image signal output from the camera unit 20 falls within an appropriate range. Is also good.

The shutter unit 104 may be omitted, and only the light output adjustment of the light source unit 10 and the adjustment of the exposure condition of the camera unit 20 may be performed.

FIG. 3 is a diagram showing another example of the configuration of the light source unit 10. As shown in FIG. In the configuration shown in FIG. 3A, each of the light sources 11A, 11A
Variable light transmittance filters 41A and 41B are provided before B, and are configured to project pattern light instead of sweeping light. As shown in FIG.
The light transmittance of the light transmittance variable filters 41A and 41B differs depending on the light transmission position. Further, the light sources 11A, 1
When using a monochromatic light source such as a laser as 1B,
Filters 12A and 12B are not required.

FIG. 4 is a diagram showing a case where the exposure conditions of the camera section are controlled by the shutter speed. In the case where exposure control is performed based on the shutter speed, the light source unit 10 needs to be of a type that projects pattern light as shown in FIG. 3 instead of a type that sweeps projection light. In this case, the light emission period T of the light source unit 10 is fixed as shown in FIG. 4A, and the exposure time T0 of the camera unit 20 in one vertical period is controlled by controlling the shutter speed as shown in FIG. 4B. You can change it. Of course, the exposure time T0 of the camera unit 20 may be fixed, and the light emission period T of the light source unit 10 may be variable. In this case, T0 ≧ T.

In the present embodiment, the distance measuring sensor 101 is provided as a means for obtaining distance information of a subject, but such a means is not necessarily required. In this case, the light output of the light source unit 10 and the exposure condition of the camera unit 20 may be controlled using the distance image obtained by the distance calculation unit 30 as the distance information of the subject. However, since the distance information is first required to confirm the area where the subject exists, it is necessary to first irradiate the subject with light. At this time, since the distance to the subject has not yet been determined, it is desirable that the light output of the light source unit 10 be suppressed to a necessary minimum.

In this specification, the distance image is
An image in which a distance from a camera or a depth value in a three-dimensional coordinate system is indicated for each pixel. The former corresponds to r in the spherical coordinate system (r, θ, φ), and the latter corresponds to z in the rectangular coordinate system (x, y, z).

The distance image obtained by the distance calculator 30 may include a large error when the signal level of the image signal output from the camera 20 is not appropriate. For example, when the subject 1 is very far away, only a small signal can be obtained for the reflected light of the projected light, so that the S / N ratio of the distance image is low. Such S /
Performing control using a distance image with a low N ratio as distance information of a subject becomes a cause of system instability.
In other words, as in the present embodiment, it is more stable to provide a means such as the distance measurement sensor 101 separately from the configuration for obtaining the distance image and use the measurement result obtained by this means as the distance information of the subject. Can be realized.

The light output of the light source unit 10 or the exposure condition of the camera unit 20 may be controlled based on the level information of the video signal output from the camera unit 20 instead of the subject distance information. .

In this embodiment, the range finder apparatus for performing three-dimensional measurement using the light intensity ratio has been described as an example. However, a configuration using other light characteristics such as light wavelength may be used. In the case where an optical wavelength is used, FIG.
A camera that can measure the light wavelength of the reflected light is required instead of the camera unit 20 shown in FIG. Further, the present invention is not limited to the configuration using the correspondence between the optical characteristics of the projection light and the projection angle.
A configuration in which the elapsed time from the start of light sweep to the arrival of light at each photosensor may be measured (A.Gruss, S.Tada, T.Kanade, “A VLSI Smart Sen
sor for Fast Range Imaging ”, in Proceedings of the
1992 IEEE / RSJ International Conference on Intelli
gent Robots and Systems, pp. 349-358, July 1992).

(Second Embodiment) FIG. 5 is a configuration diagram of a range finder device according to a second embodiment of the present invention.
5, the same elements as those of the conventional configuration shown in FIG. 22 are denoted by the same reference numerals, and detailed description thereof will be omitted. The range finder device shown in FIG.
Of the light projected from the camera 1 on the subject 1
, Thereby measuring the three-dimensional position information of the subject 1. The camera unit 310 can capture a color image (two-dimensional image). The operation of measuring the three-dimensional position information is the same as the conventional one.

The camera section 310 includes a lens 312, an aperture 3
13 and an image pickup device (CCD) 314, and an ND (Neutral Deflector) as a filter unit before the lens
nsity) Filter element 311 is provided. The ND filter element 311 has a liquid crystal element, and the light transmittance can be controlled by a voltage applied to the liquid crystal element. The exposure control unit 301 as a control unit includes the camera unit 31
0 based on the image signal output from the
Control of the aperture 313 or the ND filter element 311 of the light source unit 10 or control for adjusting the light output (light amount) of the light source unit 10 is performed.

FIG. 6 is a graph showing the relationship between the operation of the range finder device of FIG. 5 and the light transmittance of the ND filter element 311. As shown in FIG. 6, here, a first period T1 in which three-dimensional measurement is performed, that is, a distance image is captured,
The second period T2 for capturing a color image is provided alternately. The light transmittance of the ND filter element 311 is set by the exposure control unit 301 to be relatively low in the first period T1 and to be relatively high in the second period T2. The light source unit 10 includes an exposure control unit 301 and a light source control unit 16.
Thus, control is performed in the first period T1 in the same manner as in the case of the conventional three-dimensional measurement, and control is performed in the OFF state in the second period T2.

FIG. 7 is a graph showing the relationship between the sensitivity of the image sensor 314 of the camera unit 310 and the wavelength of light. Light source unit 1
The wavelength of the projection light of 0 is set in the near infrared region. The sensitivity of the image sensor 314 of the camera unit 310 is adjusted as shown in FIG. 7, and among the light incident on the camera unit 310, light in the visible region is used for capturing a color image, and light in the near-infrared region is used. It is used for three-dimensional measurement, that is, imaging of a distance image.

The technical features of the range finder device according to the present embodiment will be described with reference to FIG. In the figure, (a) shows the signal level of the video signal in the first period T1 in which three-dimensional measurement is performed, and (b) shows the signal level of the video signal in the second period T2 in which two-dimensional imaging is performed.

In this embodiment, in order to realize highly accurate three-dimensional measurement without deteriorating the image quality of the two-dimensional image, the diaphragm 31 is used in the first period T1 and the second period T2.
3 and the setting of the ND filter element 311 and the like are switched. That is, at the time of three-dimensional measurement, the signal level LB of the image component of the subject is changed so that the signal level LB of the reflected light of the projection light of the light source unit 10 becomes sufficiently high. Keep it lower than level L2.

For example, specifically, in the second period T 2, the exposure control section 301 sets the light transmittance of the ND filter element 311 to be relatively high, preferably to the maximum. Then, the exposure condition of the camera unit 310, for example, the aperture, the sensitivity of the image sensor, or the shutter speed is controlled so that the signal level L2 of the color image component does not reach the saturation level. The control here is that the color image is properly exposed,
For example, the operation may be performed such that the average value of the pixel values is equal to or higher than a predetermined reference level or the peak value of the pixel values has the maximum luminance.

In the first period T1, the exposure controller 30
Numeral 1 sets the light transmittance of the ND filter element 311 relatively low, and suppresses the background light, that is, the signal level LB of the image component of the subject to a low level. Then, the signal level L of the distance image
The exposure condition of the camera section 310 is controlled so that 1 becomes larger than the range width of the image sensor of the camera section 310. At this time, by setting the light transmittance of the ND filter element 311 low, the signal level LA of the reflected light of the projected light from the light source unit 10 which is a substantial distance image component is obtained.
Is also smaller. Therefore, the light output (light amount) of the light source unit 10
Is set high, and the substantial distance image component LA is preferably increased. Thereby, a range image having a high S / N ratio can be obtained.

Further, at the end of the second period T2,
A third period T3 in which the light of 0 is turned off may be provided, and in this third period T3, the light transmittance of the ND filter element 311 may be controlled so that the signal level of the color image component is suppressed as much as possible. . Also in this case, if the light output of the light source unit 10 is set high, it is possible to avoid the disadvantage that the range width L1 of the range image is reduced in the first period T1.

In addition to the control of the transmittance of the ND filter element 311, the level of the light output of the light source 10 is increased.
Control for down may be performed. Here, the control of the light output is a control of increasing or decreasing the level of the entire light amount while maintaining the light intensity change pattern as shown in FIG.

Instead of controlling the light transmittance of the ND filter element 311, as shown in FIG.
The incident light amount may be controlled by changing the shutter time of 0 (for example, using the electronic shutter function of the CCD).
In this case, the light source unit 10 needs to project the pattern light.

In this embodiment, both the light transmittance of the ND filter element and the exposure condition of the camera unit are controlled. However, the present invention is not limited to this, and a configuration in which either one is controlled may be used.

Here, the first and second periods T1,
Although the case where T2 is alternated has been described, the present invention is not limited to this. For example, one of the periods T1 and T2 may be continuous. Further, it is not necessary to make the sampling cycle the same between the three-dimensional measurement and the two-dimensional image capturing, and for example, the sampling cycle of the three-dimensional measurement may be set longer.

FIG. 9 is a diagram showing another configuration example of the light source section and the camera section in the range finder device according to the present embodiment. As shown in FIG. 9, the light source unit 10A can be configured using one light source 51. In this case, FIG.
As shown in FIG. 0, in the first period T1 in which three-dimensional measurement is performed, two types of pattern lights may be projected in a time-division manner.

(Third Embodiment) FIG. 11 shows a third embodiment of the present invention.
1 is a configuration diagram of a videophone system as an imaging device according to an embodiment. In FIG. 11, the same elements as those in the configuration shown in FIG. 5 are denoted by the same reference numerals, and detailed description thereof will be omitted. Here, the subject 711 is a human face.

FIG. 12A is a diagram showing the time-division processing of the imaging apparatus shown in FIG. As shown in FIG.
In the present embodiment, a first period T1 in which the infrared flash lamp 701a is turned on to distinguish the foreground from the background using the reflected light, and a second period in which a color image (two-dimensional image) is captured and only the foreground image is cut out And T2 are executed alternately.

As shown in FIG. 11, the light source section 701 includes an infrared flash lamp 701a and a transmission type liquid crystal display element 7.
01b. In response to a command from the light source control unit 702, the infrared flash lamp 701a is turned on twice (tA, tB) in the first period T1, and irradiates the subject 711 with infrared light. The transmission type liquid crystal display element 701b is a filter element having a different light transmittance depending on a light passing position, and two types of patterns can be set as a light transmittance distribution pattern. This light transmittance distribution pattern is
2, the infrared flash lamp 701a
Is switched every time is turned on. As a result, the light intensity pattern of the light projected on the subject 711 is as shown in FIG.
In FIG. 12B, the light intensity pattern IA corresponds to the light intensity characteristic of the light at the timing tA, and the light intensity pattern IB corresponds to the light intensity characteristic of the light at the timing tB. The light intensity patterns IA and IB change according to the projection angle θ.

The color signal processing section 703 processes the color image picked up in the second period T2. A foreground determining unit 704 as a foreground / background distinguishing unit distinguishes a foreground from a background based on an output signal from the camera unit 310. Cut-out part 705
Extracts a foreground image from the output image of the color signal processing unit 703 using the discrimination result by the foreground determination unit 704. The videophone unit 706 sends the foreground image output from the cutout unit 705 and the audio signal from the receiver 707 to the other party of the telephone via the telephone line 708. Control unit 709
Controls the ND filter element 311, the aperture 313, and the light source control unit 702 in the same manner as the exposure control unit 301 in the second embodiment, and additionally performs a color signal processing unit 703 and a foreground determination unit 704. , The timing of the processing is controlled.

The operation of the videophone system shown in FIG. 11 will be described with reference to FIG. FIG. 13A is the same as FIG. 12B, and shows the light intensity patterns IA and IB of the light projected on the subject 711. FIG. 13B shows the relationship between the light intensity ratio IA / IB and the projection angle θ. There is a one-to-one correspondence between the light intensity ratio IA / IB and the projection angle θ, and if the light intensity ratio IA / IB is obtained, the projection angle θ of the light can be uniquely specified.

The image pickup device 314 of the camera unit 310
In the period T1, the infrared flash lamp 701a
Light reflected by the second light irradiation is received. Foreground judgment unit 704
Calculates the light intensity ratio IA0 / IB0 for each pixel using the image signal from the image sensor 314. And FIG.
As shown in (c), it is determined from the light intensity ratio IA0 / IB0 whether each pixel is a foreground or a background. That is, when the obtained light intensity ratio IA0 / IB0 is a value larger than the predetermined criterion R _TH , the pixel is determined to correspond to the foreground (human face), while the criterion R
_{If the} value is smaller than _TH, it is determined to correspond to the background. Such a determination is made for all pixels,
This determination result is output to the cutout unit 705.

The reason why the above determination is possible will be described with reference to FIG.

As shown in FIG. 11, in the case of the videophone system, the center position P of the subject 711 existing in front of the camera unit 310 can be predetermined as an average position. Therefore, the projection angle θ _TH corresponding to the center position P thus determined may be used as the reference angle. As can be seen from FIG. 11, the projection angle is the reference angle θ _TH
If the projection angle is smaller than the reference angle θ _TH , the subject 711 is farther than the center position P when the projection angle is smaller than the reference angle θ _TH . Then, as described above, the projection angle and the light intensity ratio are 1
Since there is a one-to-one relationship, by using the light intensity ratio R _TH corresponding to the reference angle θ _TH as a reference, it is possible to directly determine the background or the foreground from the obtained light intensity ratio.

Next, in the second period T2, the camera unit 31
0 captures a color image of the subject 711. The color signal processing unit 703 obtains an imaging signal from the camera unit 310, performs predetermined processing, and outputs a color image signal to the cutout unit 705. The cutout unit 705 cuts out only the foreground image from the color image signal sent from the color signal processing unit 703 using the determination result of the foreground determination unit 704, and outputs it to the videophone unit 706. The videophone unit 706 converts the foreground image and the audio signal from the receiver 707 into a telephone line 708.
Output via.

As shown in FIGS. 14A and 14B, the criterion R _TH may have a hysteresis characteristic. This makes it possible to make a determination that is strong against noise.

In this case, the light intensity ratio is calculated to determine the background
The case where the determination of the foreground is performed has been described. However, the present invention is not limited to this. For example, the determination may be performed based on the light intensity itself. In this case, as shown in FIG. 15A, a predetermined reference light intensity I _TH is calculated from the observed value (light intensity IB) of the reflected light of the light emitted at the timing t2 (or t1) by the image sensor 314. This can be used to directly determine the background / foreground. This is because there is a one-to-one relationship between the projection angle θ and the light intensity, as shown in FIG. However, unlike the case where the light intensity ratio is used, the light intensity itself differs depending on the color of the surface of the subject 711. For this reason, in order to distinguish between the foreground and the background with the same level of accuracy as when the light intensity ratio is used, it is necessary to change the reference light intensity _ITH according to the color of the surface of the subject 711.

Although the infrared flash lamp is used as the light source here, a lamp that emits light continuously may be used. In this case, it is necessary to provide the camera unit 310 with an image sensor that captures only visible light and an image sensor that captures only infrared light. Further, as shown in FIG. 24, the light intensity may be temporally modulated to sweep the projection direction.

It is also possible to easily cut out the background instead of cutting out the foreground.

<First Modification> FIG. 16 is a diagram showing another configuration of the videophone system according to the present embodiment. FIG.
Here, the same reference numerals are given to the same components as those in FIG. 11 is different from FIG. 11 in that a threshold value determining unit 1201 that determines a threshold value as a reference for distinguishing a foreground from a background according to a subject is provided. The threshold determination unit 1201
The threshold is determined based on the distribution of the light intensity ratio of the reflected light at each pixel. More specifically, based on an output signal from the camera unit 310, the threshold determination unit 1201 determines a threshold used as a criterion for distinguishing between foreground and background by a method such as a mode method, a P-tile method, or a discrimination reference method. Is determined using

FIG. 17A is a diagram showing the threshold value determination by the mode method. In the mode method, a value in a valley of a histogram representing the distribution of the light characteristics of the reflected light in each pixel is determined as a threshold. Here, it is assumed that a histogram as shown in FIG. 17A is obtained from the brightness ratio (light intensity ratio shown in FIG. 13B) of each pixel output from the image sensor 314. The threshold value determination unit 1201 recognizes a valley of the histogram, and determines the light intensity ratio R _TH in this valley as a threshold. As a result, the threshold value changes according to the subject, and a more accurate determination can be made. The threshold value determined by the threshold value determination unit 1201 is sent to the foreground determination unit 704.

Further, the distribution of the light intensity itself may be used instead of the distribution of the light intensity ratio. In this case, FIG.
The reference light intensity I _TH shown in FIG. 5 (a) is determined. In the case where the light intensity itself is used for distinguishing the background and the foreground, since the influence of the color of the subject 711 is liable as described above, the provision of such a threshold value determination unit 1201 is necessary for improving the determination accuracy. Extremely effective.

FIG. 17B is a diagram showing how the threshold value is determined by the P-tile method. In the P-tile method, a value when the cumulative value in the cumulative histogram becomes a predetermined value is determined as a threshold. Here, it is assumed that a cumulative histogram as shown in FIG. 17B is obtained from the light intensity ratio of each pixel output from the image sensor 314. The threshold value determination unit 1201 determines the light intensity ratio _RTH when the cumulative frequency becomes P% from the cumulative histogram as a threshold value. In the case of a videophone system, the foreground (ie, human face) for the entire camera image area
Since the area ratio is expected to be almost a predetermined ratio,
The value P% of the cumulative frequency for determining the threshold can be determined in advance.

In the discriminant criterion method, the threshold value is determined so that the inter-class variance between the regions divided by the threshold value is maximized. The entire area of the camera image is defined as area R1 and area R2.
And divided into Regions R1, R2 are each omega ₁ the ratio of the area occupied, and _{ω 2 (ω 1 + ω 2} = 1), the average brightness mu _T in all areas, dispersed and sigma _T ^2, regions R1, R
The average brightness of _{No. 2} is μ ₁ and μ ₂ , respectively. At this time, the inter-class variance σ _B ² (t) is expressed by the following equation. σ _B ² (t) = ω ₁ (μ ₁ −μ _T ) ² + ω ₂ (μ ₂ −μ _T ) ² = ω ₁ · ω ₂ (μ ₁ −μ ₂ ) ² ... (2) Criteria η (t ) Is represented by the following equation. η (t) = σ _B ² (t) / σ _T ² (3) The value of t when η (t) is maximized is determined, and this is determined as a threshold value.

The above three methods are examples of the binarization method, and the threshold value may be determined by using another binarization method.

<Second Modification> FIG. 18 is a diagram showing another configuration of the videophone system according to the present embodiment. FIG.
Here, like FIG. 16, the same reference numerals are given to the same components as those in FIG. The difference from FIG. 11 is that a distance measuring sensor 1401 such as an ultrasonic sensor is provided. Then, the threshold value determination unit 1402 determines a threshold value according to the subject 711 using the distance information from the distance measurement sensor 1401 together with the signal from the image sensor 314.

More specifically, the threshold determination unit 1402 obtains the surface reflectance R of the subject 711 from the distance information to the subject 711 measured by the distance measuring sensor 1401 and the output signal of the image sensor 314, Reflectivity R
, A threshold is determined as a criterion for distinguishing between the foreground and the background.

Hereinafter, a method for determining the threshold value will be specifically described.

The distance measuring sensor 1401 detects the approximate distance r to the subject 711 and outputs it to the threshold value determining unit 1402. The threshold value determining unit 1402 determines the approximate distance r and the image sensor 3
From the light intensity IB (or IA) output from 14,
The following equation is used to calculate the surface reflectance R of the subject 711 for each pixel. R = I _B · r ² / (K · A) (4) where K is the brightness of the light source and A is the sensitivity of the image sensor 314.

Then, a threshold value of the light intensity IB, that is, a reference light intensity I _TH shown in FIG. 15 is determined from the obtained surface reflectance R. First, the threshold value IW _TH when the foreground (the surface of the subject 711) is white is determined. The determination of the threshold value IW _TH is performed while the operator looks at a monitor (not shown).
This is performed by adjusting a threshold value so that an optimal cutout image can be obtained. This determination is made at the design or manufacturing stage of the device or as a calibration prior to first use of the device.

[0104] Equation (4), the reference light intensity IW _TH when the foreground is white, with a surface reflectance R _W white as a reference, can be expressed by the following equation. IW _TH = R _W · S (5) where S = K · A / r ² Therefore, the reference light intensity I _TH for an arbitrary surface reflectance R of the foreground is equal to the predetermined reference light intensity I _{W TH} and white. by using the surface reflectance R _W of, it is expressed by the following equation. I _TH = IW _TH (R / R _W ) (6) In this way, the surface reflectance R is obtained by the equation (4) according to the subject 711, and the reference light intensity I _TH is obtained by the equation (6).
Can be determined.

When the background / foreground is distinguished from the light intensity itself, the influence of the color of the subject 711 is liable to be exerted as described above. Therefore, providing such a threshold value determination unit 1402 can improve the determination accuracy. It is extremely effective.

Further, the distance measuring sensor 1401 is provided as a means for giving the distance information to the threshold value determining unit 1402. However, instead of the distance measuring sensor 1401, a distance calculating unit 30 as shown in FIG. It doesn't matter.
In this case, for example, the subject 711 is
The average value of the distances in the vicinity of the center may be obtained, and this average value may be given to the threshold value determination unit 1402 as the approximate distance r.

FIG. 19 is a diagram showing another example of the configuration of the light source section 701. In the example shown in FIG. 19, two infrared LED arrays 1501a and 1501b are arranged up and down so that each irradiation light is directed toward the subject 711, and before that, a filter whose light transmittance can be changed. 1502a, 1
502b are provided. Filter 1502a,
The light transmittance of 1502b is the light intensity IA, I
B is adjusted so that it changes in position as shown in FIG.

As shown in FIG. 20, the threshold value for distinguishing between the foreground and the background may be changed according to the position of the pixel of the image sensor. This makes it possible to correct a deviation in the direction of the subject viewed from each pixel of the image sensor, and to more accurately distinguish between the foreground and the background.

Also, as shown in FIG. 21, the reciprocal of the threshold 1
The threshold value _RTH may be determined so that the relationship between / _RTH and Φ which is the reciprocal of the horizontal position of the pixel satisfies the following expression. 1 / R _TH = k1 · Φ + k2 (7) (where k1 and k2 are predetermined coefficients) In FIG.
It is approximate that the threshold value R _TH is determined according to the depth position of the subject 711 when the angle θP at which the subject 711 is seen from 01 is small. Further, the reciprocal of the angle φ may be used as Φ. Further, although a linear function is used in equation (7), any function may be used as long as the function changes monotonically.

[0110]

As described above, according to the present invention, even when a subject moves, for example, the intensity of the projected light and the level of the reflected light signal are controlled in accordance therewith, so that highly accurate three-dimensional position information is always obtained. Obtainable. Further, the S / N ratio of the reflected light signal can be improved without deteriorating the image quality of the two-dimensional image, and thus the accuracy of the three-dimensional position information is improved. In addition, by using the optical characteristics of the reflected light, the foreground and the background can be distinguished without using distance information.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a range finder device according to a first embodiment of the present invention.

2A and 2B are diagrams showing an operation of the apparatus of FIG. 1, wherein FIG. 2A shows a relationship between a distance to a subject and a light output of a light source unit;
FIG. 4B is a diagram illustrating a relationship between a distance to a subject and an operation of a shutter unit.

FIGS. 3A and 3B are diagrams illustrating another configuration example of a light source unit.

FIG. 4 is a diagram illustrating a case where exposure conditions of a camera unit are controlled by a shutter speed.

FIG. 5 is a configuration diagram of a range finder device according to a second embodiment of the present invention.

6 is a diagram showing the relationship between the operation of the device of FIG. 5 and the light transmittance of the ND filter element.

FIG. 7 is a graph showing the relationship between the sensitivity of the CCD of the camera unit of the apparatus of FIG. 5 and the wavelength of light.

FIGS. 8A and 8B are diagrams for explaining technical features of the second embodiment of the present invention, in which FIG. 8A illustrates a signal level of a video signal in a first period T1 in which three-dimensional measurement is performed; b)
FIG. 4 is a diagram illustrating a signal level of a video signal in a second period T2 in which a two-dimensional image is captured.

FIG. 9 is a diagram illustrating another configuration example of the light source unit and the camera unit in the range finder device.

FIG. 10 is a diagram showing timing when two types of pattern light are projected in a time-division manner.

FIG. 11 is a configuration diagram of a videophone system using an imaging device according to a third embodiment of the present invention.

12A is a diagram showing the operation of the device shown in FIG. 11,
(B) is a figure which shows two types of light intensity patterns.

FIGS. 13A to 13D are diagrams showing a method for distinguishing foreground and background from light intensity ratios.

FIGS. 14A and 14B are diagrams showing a case where a hysteresis characteristic is given to a reference for distinguishing between foreground and background.

15 (a) and (b) show the light intensity itself as the background
It is a figure showing the method of distinguishing foreground.

FIG. 16 is a diagram illustrating a configuration according to a third embodiment of the present invention, in which a threshold determination unit is provided.

FIGS. 17A and 17B are diagrams illustrating determination of a threshold using a mode method, and FIG. 17B is a diagram illustrating determination of a threshold using a P-tile method.

FIG. 18 is a diagram illustrating a configuration according to a third embodiment of the present invention, in which a distance sensor and a threshold value determination unit are provided.

FIG. 19 is a diagram illustrating another configuration example of the light source unit.

FIG. 20 is a diagram illustrating an example in which a threshold is changed according to a pixel position.

FIG. 21 is a diagram illustrating an example in which a threshold is changed according to a pixel position.

FIG. 22 is a configuration diagram of a conventional range finder device.

FIGS. 23A and 23B are diagrams showing characteristics of an optical filter provided in the apparatus of FIG.

24A is a diagram showing the relationship between the light intensity of the projection light and the projection angle, FIG. 24B is a diagram showing the relationship between the light intensity ratio and the projection angle, and FIG. It is a figure showing the relation with distance.

FIG. 25 is a diagram showing signal levels of video signals at the time of three-dimensional measurement in a conventional range finder device.

[Description of Signs] 1,711 subject 10,701 light source unit 20,310 camera unit 30 distance image 101 distance measurement sensor 102,301 exposure control unit (control unit) 104 shutter unit 311 ND filter element (filter unit) 704 foreground judgment Unit (foreground / background distinction unit) 705 cutout unit 1201, 1402 threshold value determination unit 1401 distance measurement sensor

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G03B 7/28 H04N 5/225 Z H04N 5/225 7/14 // H04N 7/14 13/02 13 / 02 G03B 3/00 A (72) Inventor Issei No. 1006 Kadoma Kadoma, Osaka Prefecture Inside Matsushita Electric Industrial Co., Ltd.

Claims (20)

[Claims] 1. Receiving reflected light of light projected on a subject,
A range finder device for measuring three-dimensional position information of the subject, a light source unit that projects the light, a camera unit that receives reflected light of the projection light from the light source unit on the subject, and A range finder device comprising: a control unit that controls at least one of a light output of the light source unit and an exposure condition of the camera unit based on distance information. 2. The range finder device according to claim 1, further comprising: a distance measuring sensor that measures a distance to the subject, wherein the control unit uses an output of the distance measuring sensor as distance information of the subject. A range finder device for performing control. 3. The range finder device according to claim 1, further comprising: a distance calculation unit that obtains a distance image from a video signal output from the camera unit, wherein the control unit calculates a distance obtained by the distance calculation unit. A range finder device, wherein control is performed using an image as distance information of the subject. 4. Receiving reflected light of light projected on a subject,
A range finder device for measuring three-dimensional position information of the subject, a light source unit that projects the light, a camera unit that receives reflected light of the projection light from the light source unit on the subject, and a camera unit. A control unit for controlling at least one of a light output of the light source unit and an exposure condition of the camera unit based on level information of a video signal output from the camera. 5. The range finder device according to claim 1, wherein the control unit determines, based on the distance information or the level information, that a distance to the subject is equal to or greater than a first threshold. While the light output of the light source unit is set to be relatively large, the second
A range finder device for setting the light output of the light source unit to a relatively small value when it is determined that the light output value is equal to or less than the threshold value. 6. The range finder according to claim 1, wherein an exposure condition of the camera unit is set by at least one of an aperture, a sensitivity of an image sensor, and a shutter speed. Range finder device. 7. The range finder device according to claim 1, further comprising: a shutter configured to be openable and closable, and blocking a projection light of the light source unit in a closed state, wherein the control unit includes a shutter unit. A range finder device that switches between open and closed states. 8. Receiving reflected light of light projected on a subject,
A range finder device for measuring three-dimensional position information of the subject, comprising: a light source unit for projecting the light; a reflected light of the projected light from the light source unit on the subject; and capturing a two-dimensional image A possible camera unit, and when the light source unit projects the light for three-dimensional measurement, the signal level of the image component of the subject is adjusted so that the signal level of the reflected light is sufficiently high when capturing a two-dimensional image. A range finder device comprising: a control unit that keeps the range finder low. 9. The range finder device according to claim 8, wherein the camera unit includes a filter unit for adjusting an amount of light incident on the camera unit, and the control unit sets the light transmittance of the filter unit to 3 A range finder device, which is relatively low during dimension measurement and relatively high during two-dimensional image capturing. 10. The range finder device according to claim 9, wherein the filter section has a liquid crystal element, and the light transmittance can be controlled by a voltage applied to the liquid crystal element. Range finder device. 11. The range finder device according to claim 8, wherein said control unit controls an exposure condition of said camera unit. 12. The range finder according to claim 11, wherein an exposure condition of said camera unit is set by at least one of an aperture, a sensitivity of an image sensor, and a shutter speed. apparatus. 13. A light source unit for projecting light and changing a light characteristic of the projection light in accordance with a projection direction, capturing a two-dimensional image, and reflecting a projection light from the light source unit on a subject An imaging apparatus, comprising: a camera unit that receives light; and a foreground / background distinguishing unit that distinguishes between a foreground and a background in the two-dimensional image based on light characteristics of light reflected by the subject. 14. The image pickup apparatus according to claim 13, wherein the foreground / background discriminating unit uses a result of the discrimination to obtain the two-dimensional image.
An imaging apparatus comprising: a cutout unit that cuts out a foreground or a background from a two-dimensional image. 15. The imaging device according to claim 13, wherein the light source unit has a light intensity characteristic that changes in accordance with a projection direction.
And the first and second lights whose change patterns are different from each other are projected, and the foreground / background discriminating unit is configured to project the reflected light of the first light and the reflected light of the second light. An imaging apparatus characterized in that a foreground and a background are distinguished by using a light intensity ratio. 16. The imaging device according to claim 13, wherein the light source unit projects light whose light intensity characteristic changes according to a projection direction, and the foreground / background distinction unit reflects the light. An imaging apparatus characterized in that a foreground and a background are distinguished by using light intensity of light. 17. The imaging apparatus according to claim 13, further comprising a threshold value determination unit that determines a threshold value, which is a reference for distinguishing between the foreground and the background, according to a subject. 18. The imaging apparatus according to claim 17, wherein the threshold value determining unit determines the threshold value based on a distribution of light characteristics of each pixel of reflected light from the subject. Imaging device. 19. The imaging apparatus according to claim 17, wherein the threshold value determination unit determines the threshold value using a surface reflectance of a subject. 20. The imaging device according to claim 19, further comprising a distance measurement sensor that measures a distance to the subject, wherein the threshold value determination unit determines a surface reflectance of the subject by using a measurement result of the distance measurement sensor. An imaging device characterized by what is required.