JP2015148565A - Information acquisition device and object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information acquisition device and an object detection device, capable of properly detecting an object, while removing background light.SOLUTION: An information acquisition unit 2 includes a projection part 100 projecting light to a target area, an imaging part 200 receiving reflected light, to image the target area, a light source control part 21a controlling the light source 110 of the projection part 100, and imaging information acquisition part 21b controlling the CMOS image sensor 240 of the imaging part 200, to acquire a background light removal image referred to for object detection on the basis of a light emission image and a light-off image. The imaging information acquisition part 21b sequentially performs exposure control for each line of the CMOS image sensor 240. The light source control part 21a controls the light source 110, so that a period in which the light is emitted in the exposure period of a line group used for the object detection of the CMOS image sensor 240 is longer than that in the exposure period of the line group not used for the object detection.

Description

  The present invention relates to an information acquisition device that acquires information in a target area and an object detection device that includes the information acquisition device.

  Conventionally, object detection devices using light have been developed in various fields. In this type of object detection apparatus, light is projected onto a target area by a projection unit, and the target area is imaged by an imaging unit including an image sensor (for example, Patent Document 1).

  In addition, in this type of object detection, an object detection device is known that removes background light from a captured image in which background light such as a fluorescent lamp in a target area is reflected (for example, Patent Document 2). This object detection apparatus uses a rolling shutter type CMOS image sensor that controls exposure of light receiving elements arranged on a two-dimensional plane for each line.

JP 2008-225985 A JP 2011-169701 A

  However, when the above CMOS image sensor is used, there arises a problem that the exposure start timing is delayed for each line. If the exposure start timing is delayed for each line, the pixel acquired when the light source emits light and the pixel acquired when the light source is turned off are mixed in the captured image. In this case, the background light cannot be properly removed, and there is a possibility of adversely affecting the object detection.

  In view of the above problems, an object of the present invention is to provide an information acquisition apparatus and an object detection apparatus that can appropriately detect an object while removing background light.

  A first aspect of the present invention relates to an information acquisition device. The information acquisition apparatus according to this aspect includes a projection unit that projects light of a predetermined wavelength onto a target region, an imaging unit that receives reflected light from the target region and images the target region, and a light source of the projection unit A light source control unit for controlling the image sensor, an imaging control unit for controlling an image sensor of the imaging unit, and information for acquiring information referred to for object detection in the target area based on an imaging signal acquired by the imaging unit An acquisition unit. The imaging control unit sequentially performs exposure control for each line of the image sensor, and the light source control unit uses a line group that is used for object detection on the image sensor for an exposure period that is not used for object detection. The light source is controlled so that the light emission period is longer than the group exposure period.

  A 2nd aspect of this invention is related with an object detection apparatus. An object detection device according to this aspect detects an object present in the target area based on the information acquisition device according to the first aspect and information acquired by an information acquisition unit of the information acquisition device. A section.

  According to the present invention, it is possible to provide an information acquisition apparatus and an object detection apparatus that can appropriately detect an object while removing background light.

  The effects and significance of the present invention will become more apparent from the following description of embodiments. However, the embodiment described below is merely an example when the present invention is implemented, and the present invention is not limited to the following embodiment.

It is a figure which shows the external appearance structure of the personal computer which incorporated the object detection apparatus which concerns on embodiment. It is a figure which shows the structure of the information acquisition apparatus and object detection apparatus which concern on embodiment. It is a figure which shows the structure of the information acquisition apparatus which concerns on embodiment. It is a figure which shows the structure of the information acquisition apparatus which concerns on embodiment. It is a figure explaining the method to remove the background light which concerns on embodiment. It is a figure explaining the reading method of the signal value of the CMOS image sensor concerning an embodiment. It is a figure which shows the imaging timing which concerns on embodiment, the light emission timing of a light source, and the exposure timing of a CMOS image sensor. It is a figure which shows the example of the background light removal which concerns on embodiment. It is a figure which shows the imaging timing which concerns on a comparative example, the light emission timing of a light source, and the exposure timing of a CMOS image sensor. It is a figure which shows the example of the background light removal which concerns on the example of a change. It is a figure which shows the imaging timing which concerns on the example of a change, the light emission timing of a light source, and the exposure timing of a CMOS image sensor. It is a flowchart which shows the process of exposure control which concerns on this Embodiment, light emission control, and imaging control. It is a figure which shows the imaging timing which concerns on the example of a change, the light emission timing of a light source, and the exposure timing of a CMOS image sensor. It is a figure which shows the example of the background light removal which concerns on the example of a change. It is a figure which shows the structure of the information acquisition apparatus which concerns on the example of a change. It is a figure which shows the sensitivity of the image sensor which concerns on the example of a change, and the waveform of the distance conversion function which prescribes | regulates the relationship between a luminance value and distance.

  Embodiments of the present invention will be described below with reference to the drawings.

  In this embodiment, the object detection apparatus according to the present invention is applied to a notebook personal computer. In addition, the object detection apparatus according to the present invention can be appropriately applied to other devices such as a desktop personal computer and a television. It should be noted that the information acquisition device and the object detection device according to the present invention do not necessarily have to be mounted integrally with other devices, and may constitute a single device alone.

  In the embodiment described below, the configuration including the opening 11a, the window 11b, and the information acquisition unit 2 provided in the main body 11 of the personal computer 1 corresponds to the “information acquisition device” described in the claims. .

  FIG. 1A is a diagram showing a schematic configuration of a personal computer 1 according to the present embodiment. FIG. 1B is a partially enlarged view around the information acquisition unit 2. In FIG. 1B, the inside of the main body portion 11 is shown to be seen through.

  As shown in FIG. 1A, the personal computer 1 includes a main body unit 11 and a monitor unit 12. The main body 11 includes an information acquisition unit 2 and an information processing unit 3. In addition, the main body 11 includes a keyboard 4 and an operation pad 5. The monitor unit 12 includes a monitor 6.

  The information acquisition unit 2 projects light (infrared light) in an infrared wavelength band longer than the visible light wavelength band over the entire target region, and receives the reflected light with a CMOS image sensor, thereby achieving the target. Imaging information of an object existing in the region is acquired. As shown in FIG. 1B, the information acquisition unit 2 is attached to the inside of the main body 11 with the front-rear direction tilted from the Z-axis direction to the in-plane direction of the YZ plane. The information processing unit 3 detects a predetermined object existing in the target area based on the imaging information acquired by the information acquisition unit 2, and further detects the movement of the object. Then, the information processing unit 3 controls the function of the personal computer 1 according to the movement of the object.

  For example, when the user performs a predetermined gesture using his / her hand, imaging information corresponding to the gesture is transmitted from the information acquisition unit 2 to the information processing unit 3. Based on this information, the information processing unit 3 detects the user's hand as a detection target object, and functions associated with the movement of the hand (screen enlargement / reduction, screen brightness adjustment, page turning, etc.) Execute.

  FIG. 2 is a diagram showing the configuration of the information acquisition unit 2 and the information processing unit 3.

  The information acquisition unit 2 includes a projection unit 100 and an imaging unit 200 as the configuration of the optical unit. The projection unit 100 and the imaging unit 200 are arranged so as to be aligned in the X-axis direction.

  The projection unit 100 includes four light sources 110 that emit light in the infrared wavelength band.

  The imaging unit 200 includes an aperture 210, an imaging lens 220, a filter 230, and a CMOS image sensor 240. In addition, the information acquisition unit 2 includes a CPU (Central Processing Unit) 21, an infrared light source driving circuit 22, an imaging signal processing circuit 23, an input / output circuit 24, and a memory 25 as a circuit unit. Yes.

  Light projected from the four light sources 110 onto the target area is reflected by an object existing in the target area, and enters the imaging lens 220 via the aperture 210.

  The aperture 210 limits light from the outside so as to match the F number of the imaging lens 220. The imaging lens 220 collects the light incident through the aperture 210 on the CMOS image sensor 240. The filter 230 is a bandpass filter that transmits light in the wavelength band including the emission wavelength of the light source 110 and cuts light in the visible wavelength band.

  The CMOS image sensor 240 is a color image sensor having sensitivity to the wavelength band of visible light and the wavelength band of infrared light emitted from the light source 110. The CMOS image sensor 240 has light receiving elements arranged on a two-dimensional plane. The CMOS image sensor 240 receives the light collected by the imaging lens 220 and outputs a signal corresponding to the amount of received light to the imaging signal processing circuit 23 for each pixel. In the CMOS image sensor 240, exposure control is performed by a so-called rolling shutter system in which exposure control is performed for each light receiving element arranged on one line among light receiving elements arranged on a two-dimensional plane. Here, in the CMOS image sensor 240, the output speed of the signal is increased so that the signal of the pixel can be output to the imaging signal processing circuit 23 with high response from the light reception in each pixel. By using a rolling shutter type CMOS image sensor 240 as the image sensor of the imaging unit 200, the target area can be imaged at high speed while suppressing power consumption.

  The CPU 21 controls each unit according to a control program stored in the memory 25. With this control program, the functions of the light source control unit 21a and the imaging information acquisition unit 21b are given to the CPU 21.

  The light source control unit 21a controls the infrared light source driving circuit 22 so that the light emission period and the light extinction period are repeated at predetermined time intervals.

  The imaging information acquisition unit 21b acquires imaging information related to a detection target object such as luminance information based on a signal output from the CMOS image sensor 240. The imaging information acquisition unit 21 b sets the exposure time and gain of the CMOS image sensor 240 in the imaging signal processing circuit 23.

  The infrared light source driving circuit 22 drives the four light sources 110 according to control signals from the CPU 21.

  The imaging signal processing circuit 23 drives the CMOS image sensor 240 under the control of the CPU 21 and sequentially acquires signals output from the CMOS image sensor 240 for each line. The imaging signal processing circuit 23 acquires the luminance signal of each pixel and outputs the acquired signal to the CPU 21. The imaging signal processing circuit 23 applies the exposure time set by the CPU 21 to each pixel of the CMOS image sensor 240, and further applies the gain set by the CPU 21 to the signal output from the CMOS image sensor 240. A luminance signal is acquired for each pixel.

  The input / output circuit 24 controls data communication with the information processing unit 3.

  The memory 25 is used not only as a control program executed by the CPU 21 but also as a work area during processing in the CPU 21.

  The information processing unit 3 includes a CPU 31, an input / output circuit 32, and a memory 33. In addition to the configuration shown in FIG. 2, the information processing unit 3 is provided with a configuration for driving and controlling each unit of the personal computer 1. For convenience, the configuration of these peripheral circuits is not shown.

  The CPU 31 controls each unit according to a control program stored in the memory 33. With this control program, the function of the function detection unit 31b for controlling the function of the personal computer 1 is given to the CPU 31 in accordance with the function of the object detection unit 31a and the signal from the object detection unit 31a.

  The object detection unit 31a acquires a difference between the luminance signal of the imaging information acquired by the imaging information acquisition unit 21b when the light source 110 emits light and the luminance signal of the imaging information acquired by the imaging information acquisition unit 21b when the light source 110 is turned off. Then, background light that does not affect the light emission of the light source 110 is removed. The object detection unit 31a extracts the shape of the object from the imaging information from which the background light is removed, and further detects the movement of the extracted object shape. The function control unit 31b determines whether the movement of the object detected by the object detection unit 31a matches a predetermined movement pattern. If the movement of the object matches the predetermined movement pattern, the movement pattern The control corresponding to is executed.

  FIG. 3 is a schematic diagram showing the configuration of the information acquisition unit 2.

  With reference to FIG. 3, the information acquisition unit 2 includes a projection unit 100, an imaging unit 200, a circuit board 300, and a circuit unit 400.

  The projection unit 100 includes the four light sources 110 as described above. The four light sources 110 are made up of light emitting diodes (LEDs). A light emitting element (not shown) that emits infrared light is accommodated in a housing. Infrared light emitted from the light emitting element is emitted to the outside at a predetermined radiation angle from the emission portion on the upper surface. As shown in FIG. 3, the four light sources 110 are mounted on the circuit board 300.

  The imaging unit 200 includes an aperture 210, a lens barrel 250 that houses the imaging lens 220, a filter 230, and an imaging lens holder 260 that houses the CMOS image sensor 240. The CMOS image sensor 240 is mounted on the circuit board 300. The imaging lens 220 is attached to the lens barrel 250, and the lens barrel 250 is attached to the imaging lens holder 260 while holding the imaging lens 220. The imaging lens holder 260 has a recess on the lower surface, and the filter 230 is attached to the recess. In this way, the imaging lens holder 260 is installed on the circuit board 300 so as to cover the CMOS image sensor 240 while holding the imaging lens 220 and the filter 230.

  In addition to the projection unit 100 and the imaging unit 200, the circuit unit 400 constituting the information acquisition unit 2 is mounted on the circuit board 300. The CPU 21, the infrared light source driving circuit 22, the imaging signal processing circuit 23, the input / output circuit 24 and the memory 25 shown in FIG. 2 are included in the circuit unit 400.

  FIG. 4 is a perspective view showing a state in which the information acquisition unit 2 is assembled to the main body 11. As shown in FIG. 4, the information acquisition unit 2 is attached to the main body 11 so that the front-rear direction is slightly inclined from the Z-axis direction to the in-plane direction of the YZ plane. When light is projected onto the upper portion of the information acquisition unit 2 by the four light sources 110, the light is transmitted through the window portion 11b and projected toward the upper periphery of the keyboard 4 (see FIG. 1A).

  5A to 5C are diagrams for explaining a method of removing background light.

  As shown in FIG. 5A, when a fluorescent lamp LD is included in the imaging region, when the imaging unit 200 images the imaging region while irradiating infrared light from the light source 110, FIG. A captured image as shown in FIG. A captured image that is captured in a state where the light source 110 is caused to emit light is referred to as a “light-emitting image”.

  Further, when an imaging region is imaged by the imaging unit 200 without irradiating infrared light from the light source 110, a captured image as illustrated in FIG. 5C is acquired. In this manner, a captured image captured with the light source 110 turned off is referred to as a “light-off image”.

  When the luminance value of the extinguished image of FIG. 5C is subtracted from the luminance value of the light emitting image of FIG. 5B, a captured image as shown in FIG. 5D is acquired. In this way, by subtracting the luminance value of the extinguished image acquired when the light source 110 is extinguished from the luminance value of the luminous image acquired when the light source 110 is emitting light, light other than infrared light emitted from the light source 110 is used. Noise components (for example, fluorescent lamp LD) can be removed. Such a captured image from which noise components due to light other than infrared light irradiation are removed is referred to as a “background light removed image”.

  FIG. 6 is a diagram for explaining a method of reading a signal value of the CMOS image sensor 240.

  As shown in FIG. 6, the CMOS image sensor 240 includes a photodiode 240a corresponding to a pixel. When the photodiode 240a receives light, the photodiode 240a accumulates electric charge according to the amount of light. The accumulated electric charge is converted into a voltage by the amplifier 240b and amplified. The amplified voltage is transmitted to the vertical signal line 240d for each line L when the switch 240c is turned on. The transmitted voltage is temporarily stored by the column circuit 240e arranged for each vertical signal line 240d. The stored voltage is sent to the horizontal signal line 240g when the column selection switch 240f is turned on. Then, the voltage sent to the horizontal signal line 240g is sent to the imaging signal processing circuit 23.

  Thus, the CMOS image sensor 240 transmits a voltage signal for each line L.

  FIG. 7 is a diagram illustrating the light emission timing of the light source 110 by the light source control unit 21a and the exposure timing for each of the lines L0 to Ln of the CMOS image sensor 240 by the imaging information acquisition unit 21b.

  The vertical synchronization signal repeats ON / OFF at a predetermined imaging interval ΔF1 and is synchronized with the exposure start timing of the line L0 of the CMOS image sensor 240. The light emission timing indicates the timing at which the light source 110 emits light and the timing at which the light source 110 is turned off. The light emission period ΔL1 and the turn-off period ΔL2 are set to the same time interval as the imaging interval ΔF1. The exposure timing indicates the timing at which the photodiodes 240a of the lines L0 to Ln of the CMOS image sensor 240 corresponding to a predetermined imaging timing are exposed. The exposure timings of the lines L0 to Ln corresponding to the predetermined imaging timing are indicated by the same hatching, and the exposure timings of the lines L0 to Ln corresponding to the next imaging timing are indicated by different hatching.

  The CMOS image sensor 240 starts exposure of the line L0 at a predetermined imaging interval ΔF1 by the vertical synchronization signal. As a result, the CMOS image sensor 240 starts accumulating charges according to the amount of light incident on each photodiode 240a arranged on the line L0. The CMOS image sensor 240 continues to accumulate charges for a predetermined exposure period ΔP1. In the line L0, the exposure is continued during the light emission period ΔL1.

  In the CMOS image sensor 240, since exposure control is performed for each line, the exposure start timing is slightly delayed each time the line advances. For example, in the line L1, the first exposure start timing is slightly delayed compared to the line L0. As a result, the first exposure end timing of the line L1 is slightly delayed from the light emission end timing. That is, the exposure is performed in the extinction period ΔL2 of the light source 110 just before the first exposure end timing of the line L1. Therefore, at the first exposure timing of the line L1, the amount of charge accumulated in the photodiode 240a is slightly smaller than that of the line L0. In the line L1, since the exposure period that overlaps the extinguishing period ΔL2 is very short, the amount of attenuation of the amount of light received by the photodiode 240a is also very small.

  Similarly, as the lines L2 and L3 progress, the exposure end timing is delayed, and in the last line Ln, most of the first exposure period overlaps the extinguishing period ΔL2. Therefore, the last line Ln and the line close to the line Ln cannot be exposed substantially during the light emission period ΔL1 of the light source 110, and the amount of light when the light source 110 emits light cannot be obtained appropriately.

  Thus, the interval ΔPe1 from the exposure start timing of the line L0 to the exposure end timing of the last line Ln at the first imaging timing is significantly longer than the imaging interval ΔF1 and the light emission period ΔL1.

  After the exposure at the first imaging timing of the last line Ln is started, when the second imaging timing comes, the exposure of the first line L0 is started even if the exposure of the last line Ln is continuing. The

  At the second imaging timing, the line L0 can be appropriately exposed in the extinguishing period ΔL2, but in the last line Ln, most of the exposure period overlaps the light emission period ΔL1. Therefore, the last line Ln and the line close to the line Ln cannot be exposed substantially during the light extinction period ΔL2 of the light source 110, and the luminance value when the light source 110 is extinguished cannot be obtained appropriately.

  FIG. 8A is a schematic diagram illustrating an example of a light emission image. FIG. 8B is a schematic diagram illustrating an example of a light-off image. FIG. 8C is a schematic diagram illustrating an example of a background light removed image. FIG. 8D is a schematic diagram illustrating an example of a light emission image on which noise light is incident. In FIGS. 8A to 8D, the higher the luminance value, the brighter the hatching, and the lower the luminance value, the darker the hatching. 8A to 8D, the uppermost portion of the image corresponds to the first line L0 of the CMOS image sensor 240, and the lowermost portion of the image corresponds to the last line Ln of the CMOS image sensor 240.

  As shown in FIG. 8A, the lower region D1 of the light emission image captured when the light source 110 emits light is dark. This is because the exposure start timing is delayed in the line of the CMOS image sensor 240 corresponding to the lower region D1, and the infrared light emitted from the light source 110 cannot be received sufficiently.

  Further, as shown in FIG. 8B, the lower region D2 of the extinguished image captured when the light source 110 is extinguished is bright. This is because the exposure start timing is delayed in the CMOS image sensor 240 line corresponding to the lower region D2, and infrared light emitted from the light source 110 is received at the next light emission timing.

  For example, when the luminance value of the extinguished image shown in FIG. 8B is subtracted from the luminance value of the light emitting image shown in FIG. 8A, a background light removed image as shown in FIG. 8C is acquired. . In the lower region D3 shown in FIG. 8C, the luminance value of the pixel has a negative gradation by subtracting the low luminance value by the high luminance value. When the luminance value has a negative gradation, it is shown in black as in the case where the luminance value is 0.

  As described above, when lines exposed in the light emission period ΔL1 and the extinguishing period ΔL2 coexist in the same captured image, the background light cannot be appropriately removed from that portion. Such an area where background light cannot be removed appropriately is referred to as an “exposure delay area”. If the exposure delay region D3 overlaps the detection target object, there is a possibility that the object cannot be detected properly.

  In order to prevent a line exposed in the light emission period ΔL1 and the light extinction period ΔL2 of the light source 110 from being mixed in the same captured image, for example, as in the comparative example shown in FIG. A method of lengthening the period ΔL4 can be adopted.

  FIG. 9 is a diagram illustrating the light emission timing of the light source 110 by the light source control unit 21a according to the comparative example and the exposure timing for each of the lines L0 to Ln of the CMOS image sensor 240 by the imaging information acquisition unit 21b.

  As shown in FIG. 9, the imaging interval ΔF2 according to the comparative example is set to twice the imaging interval ΔF1 in the case shown in FIG. Further, the light emission period ΔL3 and the light extinction period ΔL4 according to the comparative example are set to be twice the light emission period ΔL1 and the light extinction period ΔL2 in the case shown in FIG. The exposure period ΔP1 according to the comparative example is set to the same value as that shown in FIG.

  Thus, the interval ΔPe1 between the exposure start timing of the line L0 and the exposure end timing of the last line Ln at the first imaging timing is smaller than the imaging interval ΔF2 and the light emission period ΔL3. Therefore, in the case of the comparative example, since the lines exposed in the light emission period ΔL3 and the light extinction period ΔL4 of the light source 110 are not mixed in the same captured image, the background light can be appropriately removed.

  However, in the case of the comparative example, there arises a problem that the number of captured images (frame rate) that can be acquired in time units is lower than in the case of FIG. For example, in the case of the comparative example shown in FIG. 9, the imaging interval ΔF2 required when the light source 110 emits light is twice the imaging interval ΔF1 in the case of FIG. 7, and the frame rate according to the comparative example is as shown in FIG. Is ½ times the frame rate.

  Here, in the object detection process, the reference point (marker) of the detection target object is set from the captured image, and the movement of the object (gesture) is detected by detecting the change in the position of the marker over time. For example, the marker is set at the tip of the finger, the center of gravity of the finger, or the like. In the object detection process, at least if such a marker can be set in the captured image, the movement of the object can be detected appropriately. That is, the area other than the area necessary for setting the marker is not so important in the object detection process.

  Therefore, in the present embodiment, the light source 110 is controlled to emit light so that the exposure delay area D3 is positioned in an area that does not need attention in object detection.

  For example, as shown in FIGS. 8A to 8C, when the detection target object is a hand, if the exposure delay region D3 is not overlapped with the tip of the finger or the base of the finger, the finger moves properly. Can be detected. As shown in FIGS. 8A to 8C, when the tip of a finger important for object detection processing is positioned on the first line L0 side of the CMOS image sensor 240, the last line of the CMOS image sensor 240 is used. The light emission control of the light source 110 may be performed so that the exposure delay region D3 is positioned on the Ln side. That is, the light source 110 may be controlled to emit light as shown in FIG.

  Thereby, it is possible to remove the influence of background light while suppressing a decrease in the frame rate, and to perform object detection appropriately.

  In the case of the comparative example, as shown in FIG. 8D, when the noise light is reflected in the lower region only in the luminescent image and the noise light is not reflected in the extinguished image, There is a possibility that the light portion is misrecognized as an object to be detected.

  In the present embodiment, since the exposure delay area D3 is positioned in the lower area that does not need attention in object detection, the luminance value of the light emission image is determined from the luminance value of the pixel included in the bright area D2 below the unlit image. The luminance value of the pixel included in the lower dark area D1 is subtracted. Therefore, as shown in FIG. 8D, even if noise light is reflected in the lower area of the luminescent image, the luminance value amplified by the noise light of the luminescent image is displayed in the bright area D2 below the extinguished image. Subtracted by the luminance value of the included pixel. As a result, the luminance value of the noise light is greatly reduced, and it can be prevented that the area where the noise light is reflected is erroneously recognized as the detection target object.

  FIG. 10A is a schematic diagram illustrating an example of a light emission image according to the modified example. FIG. 10B is a schematic diagram illustrating an example of a light-off image according to the modification example. FIG. 10C is a schematic diagram illustrating an example of the background light removal image according to the modification example. 10A to 10C, the uppermost portion of the image corresponds to the first line L0 of the CMOS image sensor 240, and the lowermost portion of the image corresponds to the last line Ln of the CMOS image sensor 240.

  In the example shown in FIGS. 8A to 8D, the tip of the finger that is important for the object detection process is positioned on the first line L0 side of the CMOS image sensor 240. However, FIGS. ), The tip of the finger important for the object detection processing is positioned on the last line Ln side of the CMOS image sensor 240 by changing the arrangement, orientation, and the like of the information acquisition unit 2. In this case, the light source 110 may be controlled to emit light so that the exposure delay region D3 is positioned on the uppermost line L0 side of the CMOS image sensor 240.

  FIG. 11 is a diagram illustrating the light emission timing of the light source 110 by the light source control unit 21a according to the modification and the exposure timing for each of the lines L0 to Ln of the CMOS image sensor 240 by the imaging information acquisition unit 21b.

  As shown in FIG. 11, the imaging interval ΔF1 and the exposure period ΔP1 are set in the same manner as in FIG. In the present modification, the light emission period ΔL1 includes the completion timing of the exposure period ΔP1 of the last line Ln of the CMOS image sensor 240. In the present modification, the extinguishing period ΔL2 includes the completion timing of the exposure period ΔP1 of the last line Ln of the CMOS image sensor 240.

  In this way, by controlling the light source 110, a light emission image as shown in FIG. 10A and a light-off image as shown in FIG. 10B are acquired. Then, the luminance value of the extinguished image is subtracted from the light emitting image, and a background light removed image as shown in FIG.

  Also in this modified example, since the exposure delay region D3 is positioned in the upper region that does not need to be focused on in the object detection process, the influence of the background light can be removed while suppressing the decrease in the frame rate. And object detection can be performed properly. In addition, as described above, it is possible to suppress the influence of noise light that appears in the upper region that does not need attention in such object detection processing.

  FIG. 12A is a flowchart showing exposure control processing of the CMOS image sensor 240. FIG. 12B is a flowchart showing the light emission control process of the light source 110. FIG. 12C is a flowchart illustrating imaging control processing. The processes in FIGS. 12A and 12C are executed by the function of the imaging information acquisition unit 21b among the functions of the CPU 21 shown in FIG. The process of FIG.12 (b) is performed by the function of the light source control part 21a among the functions of CPU21 shown in FIG.

  Referring to FIG. 12A, the CPU 21 exposes the CMOS image sensor 240 so as to continuously expose each line during the exposure period ΔP1 at the imaging interval ΔF1 in order to image the target area. The operation is started (S101). Then, the CPU 21 determines whether or not the exposure operation of the CMOS image sensor 240 is stopped (S102). When the exposure operation is not stopped (S102: NO), the CPU 21 continues the exposure operation for each line of the CMOS image sensor 240. When the exposure operation is stopped (S102: YES), the CPU 21 stops the exposure operation of the CMOS image sensor 240 (S103).

  Referring to FIG. 12B, the CPU 21 sets the light source 110 so that an important area in the object detection process is positioned at an appropriate light emission / extinction timing in the exposure period ΔP1 of the lines L0 to Ln of the CMOS image sensor 240. The light emission / extinguishing operation is started (S201). The CPU 21 determines whether or not the light emission / extinguishing operation of the light source 110 is stopped (S202). When the light emission / light-off operation is not stopped (S202: NO), the CPU 21 continues the light emission / light-off operation of the light source 110. When the light emission / extinction operation is stopped (S202: YES), the CPU 21 stops the light emission / extinction operation of the light source 110 (S203).

  Thereby, as shown in FIG. 7 or FIG. 11, the CMOS image sensor 240 is controlled to be exposed and the light source 110 is controlled to emit light.

  With reference to FIG.12 (c), CPU21 acquires a light emission image based on the luminance signal of each pixel acquired for every line by the exposure control corresponding to predetermined | prescribed light emission timing (S301). Since the CMOS image sensor 240 is controlled to be exposed as shown in FIG. 12A, for example, as shown in FIG. 8A, the lower region D1 that does not need to be noticed in object detection becomes dark. The emitted light image is acquired.

  The CPU 21 acquires a light-off image based on the luminance signal of each pixel acquired for each line by exposure control corresponding to a predetermined light-off timing (S302). As a result, for example, as shown in FIG. 8B, an extinguished image in which the lower region D <b> 2 that does not need attention in object detection becomes bright is acquired.

  The CPU 21 subtracts the luminance value of the extinguished image for each pixel from the luminance value of the light emitting image (S303). The CPU 21 acquires a background light removed image based on the subtraction result (S304). As a result, for example, as shown in FIG. 8C, a background light removed image in which the exposure delay region D3 is positioned in the lower region that does not need attention in object detection is acquired.

  Then, the CPU 21 determines whether or not to end the imaging process (S305). When the imaging process is not finished (S305: NO), the CPU 21 returns the process to S301 and repeats the imaging process (S301 to S304). When the imaging process ends (S305: YES), the CPU 21 completes the process.

  By controlling in this way, as shown in FIG. 8A to FIG. 8C or FIG. 10A to FIG. 10C, an exposure delay is applied to an area that does not need attention in the object detection process. Region D3 can be positioned.

<Effect of embodiment>
As described above, according to the present embodiment, it is possible to position the exposure delay area D3 in an area that does not need attention in the object detection process. Therefore, the influence of background light can be removed while suppressing a decrease in the frame rate. Therefore, object detection can be performed appropriately.

  Further, according to the present embodiment, it is possible to suppress the influence of noise that appears in an area that does not need attention in object detection processing.

  Further, according to the present embodiment, since the imaging interval is not extended as in the comparative example shown in FIG. 9, it is possible to suppress a decrease in the frame rate of the captured image.

<Example of change>
While the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made to the configuration example of the present invention.

  For example, in the above embodiment, the exposure period ΔP1 of the CMOS image sensor 240 is set to substantially the same time as the imaging interval ΔF1 as shown in FIG. 7, but the exposure period ΔP1 of the CMOS image sensor 240 is shown in FIG. As shown, it may be set shorter than the imaging interval ΔF1.

  FIG. 13 is a diagram illustrating the light emission timing of the light source 110 by the light source control unit 21a according to the modification and the exposure timing for each of the lines L0 to Ln of the CMOS image sensor 240 by the imaging information acquisition unit 21b.

  As shown in FIG. 13, the imaging interval ΔF1, the light emission period ΔL1, and the extinguishing period ΔL2 are set similarly to the case shown in FIG. In this modification, the exposure period ΔP2 is set shorter than the exposure period ΔP1 in the case shown in FIG. The exposure period ΔP2 is shorter than the imaging interval ΔF1.

  Thus, by setting the exposure period ΔP2 to be short, the interval ΔPe2 between the exposure start timing of the line L0 and the exposure end timing of the last line Ln at the first imaging timing becomes shorter than the interval ΔPe1 shown in FIG. .

  FIG. 14A is a schematic diagram illustrating an example of a light emission image according to the modified example. FIG. 14B is a schematic diagram illustrating an example of a light-off image according to the modification example. FIG. 14C is a schematic diagram illustrating an example of the background light removal image according to the modification example. 14A to 14C, the uppermost portion of the image corresponds to the first line L0 of the CMOS image sensor 240, and the lowermost portion of the image corresponds to the last line Ln of the CMOS image sensor 240.

  As shown in FIGS. 14A to 14C, by setting the exposure period ΔP2 to be shorter than the imaging interval ΔF1, the width of the exposure delay region D3 can be kept small. Thereby, the influence of the delay of the exposure start timing of the CMOS image sensor 240 can be suppressed. In addition, since the width of the exposure delay region D3 can be reduced in this manner, object detection can be performed more appropriately.

  When the exposure period ΔP2 is shortened, the time for accumulating charges in the photodiode 240a of the CMOS image sensor 240 is shortened, so that the luminance value of the output captured image tends to be small. Therefore, when the exposure period ΔP2 is shortened, it is preferable that the amplification factor (gain) of the luminance value is set larger than that in the above embodiment.

  In this way, the width of the exposure delay area D3 can be kept small without reducing the luminance value of the captured image.

  In the configuration in which the exposure period ΔP2 of the CMOS image sensor 240 is set to be shorter than the imaging interval ΔF1 and the gain of the luminance value is increased as in this modification, the width of the exposure delay region D3 is Since it is small, it becomes difficult for the exposure delay region D3 to overlap the region important for object detection processing (such as the tip of a finger). For this reason, the exposure delay area D3 does not necessarily have to be positioned at an edge position far away from an area important for object detection processing (such as the tip of a finger). You may approach an area important for processing.

  In this modified example as well, it is desirable that the light source 110 is controlled to emit light so that the exposure delay region D3 is positioned in the edge region that does not need attention in the object detection process, as in the above embodiment. This makes it difficult for the exposure delay region D3 to overlap the region important for object detection processing (such as the tip of a finger), thereby improving the object detection accuracy.

  In the above embodiment, the imaging information is acquired by the information acquisition unit 2 and the shape and movement of the detection target object are detected by the information processing unit 3, but the distance to the detection target object is further detected by the information acquisition unit 2. Information may be acquired.

  FIG. 15 is a diagram illustrating configurations of the information acquisition unit 2 and the information processing unit 3 according to the modification.

  In this modified example, the function of the distance acquisition unit 21c is given to the CPU 21 together with the function of the imaging information acquisition unit 21b.

  The distance acquisition unit 21c acquires distance information based on a signal output from the CMOS image sensor 240. The memory 25 holds a distance conversion function used for acquiring distance information.

  FIG. 15A is a diagram schematically showing the sensitivity of each pixel on the CMOS image sensor 240.

  In the present embodiment, a color sensor is used as the CMOS image sensor 240. Therefore, the CMOS image sensor 240 includes three types of pixels that detect red, green, and blue, respectively.

  In FIG. 15A, R, G, and B indicate the sensitivity of red, green, and blue pixels included in the CMOS image sensor 240, respectively. As shown in FIG. 15A, the red, green, and blue pixels have substantially the same sensitivity in the infrared wavelength band of 800 nm or more (the hatched portion in FIG. 15A is shown). reference). Therefore, when the visible light wavelength band is removed by the filter 230 shown in FIG. 15B, the sensitivity of the red, green, and blue pixels of the CMOS image sensor 240 becomes substantially equal to each other. For this reason, when the same amount of infrared light is incident on the red, green, and blue pixels, the values of the signals output from the pixels of the respective colors are substantially equal. Therefore, there is no need to adjust the signal from each pixel between the pixels, and the signal from each pixel can be used as it is for obtaining distance information.

  FIG. 15B is a diagram schematically showing the waveform of the distance conversion function held in the memory 25.

  As shown in FIG. 15B, the distance conversion function defines the relationship between the luminance value acquired via the CMOS image sensor 240 and the distance corresponding to the luminance value. In general, the amount of light traveling straight is attenuated in inverse proportion to the square of the distance. Accordingly, the infrared light emitted from the projection unit 100 is attenuated to one-quarter of the distance obtained by adding the distance from the projection unit 100 to the target area and the distance from the target area to the imaging unit 200. The light is received by the imaging unit 200. For this reason, as shown in FIG. 15B, the luminance value acquired via the CMOS image sensor 240 decreases as the distance to the object increases, and increases as the distance to the object decreases. Therefore, the distance conversion function that defines the relationship between the distance and the luminance has a curved waveform as shown in FIG.

  The distance acquisition unit 21c converts a luminance value acquired for each pixel through the CMOS image sensor 240 into distance information using a distance conversion function. In this way, distance information from the information acquisition unit 2 to each part of the detection target object is acquired.

  In the case of this modified example, since distance information is acquired according to the luminance value, it is possible to accurately detect the movement in the depth direction as well as the vertical and horizontal movements of the object. As described above, in this modified example, since the motion of the object in the three-dimensional direction can be detected, the function of the personal computer 1 can be controlled according to various gestures.

  Moreover, in the said embodiment, although CPU21 had the function which acquires imaging information, acquisition of imaging information may be implement | achieved by the hardware process by a circuit.

  Further, in the above embodiment, light in the wavelength band of infrared light is used, but light in a wavelength band other than infrared light can also be used for distance acquisition.

  Moreover, in the said embodiment and modification, although the information acquisition unit 2 was assembled | attached to the main-body part 11 of the personal computer 1, and the information processing apparatus was comprised, an information processing apparatus may be comprised as a single-piece | unit. In this case, the information acquisition unit 2 is housed in a housing of the information processing apparatus, and infrared light transmitting portions (opening 11a and window portion 11b) are provided in the housing.

  In addition, the embodiment of the present invention can be variously modified as appropriate within the scope of the technical idea shown in the claims.

1 ... Personal computer (object detection device)
2 ... Information acquisition unit (information acquisition device)
21a: Light source control unit 21b: Imaging information acquisition unit (imaging control unit, information acquisition unit)
22 ... Infrared light source drive circuit (light source control unit)
23 ... Imaging signal processing circuit (imaging controller)
DESCRIPTION OF SYMBOLS 100 ... Projection part 110 ... Light source 200 ... Imaging part 240 ... CMOS image sensor (image sensor)
L ... Line L0-Ln ... Line

Claims (6)

A projection unit that projects light of a predetermined wavelength onto the target area;
An imaging unit that receives reflected light from the target area and images the target area;
A light source control unit for controlling the light source of the projection unit;
An imaging control unit that controls an image sensor of the imaging unit;
An information acquisition unit that acquires information referred to object detection in the target area based on an imaging signal acquired by the imaging unit;
The imaging control unit performs exposure control in order for each line of the image sensor,
The light source control unit is configured so that the exposure period of the line group used for object detection on the image sensor takes a longer period of light emission than the exposure period of the line group not used for object detection. Control the light source;
An information acquisition apparatus characterized by that. The information acquisition device according to claim 1,
The light source control unit controls the light source so that the light emission period and the light extinction period are assigned to two adjacent imaging periods, respectively.
The information acquisition unit subtracts the third imaging information obtained by subtracting the first imaging information acquired by the exposure control corresponding to the light emission period and the second imaging information acquired by the exposure control corresponding to the extinguishing period, Obtained as information referenced for object detection within the target area,
An information acquisition apparatus characterized by that. The information acquisition device according to claim 2,
The light source control unit controls the light source so that a timing at which exposure control is started for a predetermined line of the line group used for object detection and a start timing of the light emission period are synchronized.
An information acquisition apparatus characterized by that. In the information acquisition device according to claim 3,
The light source control unit controls the light source so that a timing at which exposure control is started for the predetermined line of the line group used for object detection and a start timing of the extinguishing period are synchronized.
An information acquisition apparatus characterized by that. In the information acquisition device according to any one of claims 1 to 4,
The imaging control unit sets the exposure period for each line shorter than the imaging period;
An information acquisition apparatus characterized by that. An information acquisition device according to any one of claims 1 to 5,
An object detection unit that detects an object present in the target region based on information acquired by an information acquisition unit of the information acquisition device;
An object detection apparatus characterized by that.

Images