JP2006099749A - Gesture switch - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a user-friendly gesture switch capable of extracting a specific portion of a person without being affected by a background of a detection target area. <P>SOLUTION: This gesture switch has: a distance image sensor DS continuously generating a distance image with a distance value to an object Ob present in the detection target area as a pixel value; a specific portion extraction part 5 extracting the specific portion of the person present in the detection target area on the basis of the distance image generated by the distance image sensor DS; a gesture recognition part 6 recognizing a prescribed gesture inside a specific space on the basis of time-series data on shape of the specific portion extracted by the specific portion extraction part 5; and a control part 7 imparting control output previously associated to the gesture recognized by the gesture recognition part 6 to control target equipment. The specific portion extraction part 5 binarizes the distance image with a value obtaining by adding a prescribed value to the minimum distance value in each distance image as a threshold value, and extracts an area wherein it becomes the threshold value or below as the specific portion. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a gesture switch.

  Conventionally, as an example of a man-machine interface for controlling a control target device without directly operating (touching) the control target device or a control switch of the control target device to the installation site, the tip of a person's wrist There has been proposed a gesture switch that detects the movement of the hand and gives a control output to the device to be controlled based on the detected hand movement (gesture) (see, for example, Patent Documents 1 and 2).

Here, Patent Documents 1 and 2 describe a method of using a human myoelectric signal as a method of detecting a gesture, and Patent Document 2 discloses one or a plurality of methods as a method of detecting a gesture. A method for recognizing a gesture by performing appropriate image processing on an image captured by a camera and extracting a specific part of a person is described.
JP 2000-138858 A JP-A-7-248873

  By the way, in the gesture switch that employs the method of detecting a gesture using the human myoelectric signal disclosed in Patent Documents 1 and 2, it is necessary for the person to wear a detection device for detecting the myoelectric signal. Yes, it was not easy to use.

  On the other hand, the gesture switch adopting the method for detecting a gesture by image processing disclosed in Patent Document 2 has an advantage that it is easy to use because it is not necessary for a person to wear anything.

  However, in the gesture switch that employs the method of detecting a gesture by the image processing disclosed in Patent Document 2, the image obtained by imaging the detection target area by the camera is a grayscale image. There was a defect that it was easily affected by the background.

  The present invention has been made in view of the above-mentioned reasons, and the purpose thereof is not necessary to go to the installation location by touching the control target device or the control switch of the control target device, and is easy to use. Another object of the present invention is to provide a gesture switch capable of extracting a specific part of a person without being affected by the background of the detection target area.

  According to the first aspect of the present invention, a distance image sensor that continuously generates a distance image having a pixel value as a distance value to an object existing in the detection target area, and a detection target based on the distance image generated by the distance image sensor A specific part extraction unit for extracting a specific part of a person existing in the area, and a gesture for recognizing a predetermined gesture in the specific space based on time-series data of the shape of the specific part extracted by the specific part extraction unit It comprises a recognition unit, and a control unit that gives a control output associated in advance with the gesture recognized by the gesture recognition unit to the device to be controlled. In addition, the object in which the detection target area exists includes a human body.

  According to this invention, it exists in a detection object area based on the distance image produced | generated by one distance image sensor which produces | generates the distance image which uses the distance value to the object which exists in a detection object area as a pixel value continuously. Since the specific part of the person is continuously extracted by the specific part extraction unit, and the predetermined gesture of the person in the specific space is recognized based on the data output from the specific part extraction unit in the gesture recognition unit, the control There is no need to go to the installation location and touch the target device or the control switch for the control target device, and there is no need for the user to wear anything, and it is easy to use and is affected by the background of the detection target area. Therefore, it becomes possible to extract a specific part of a person.

  According to a second aspect of the present invention, in the first aspect of the invention, the control unit outputs the control output when a ratio of the number of pixels of the specific portion to the number of pixels of the specific space in the distance image exceeds a specified value. Is provided to the device to be controlled.

  According to the present invention, it is possible to prevent the control output from being given to the control target device even if a motion of the person not intending to control the control target device in a space other than the specific space is recognized as a gesture. it can.

  According to a third aspect of the present invention, in the first or second aspect of the invention, the specific part extraction unit binarizes the distance image with a predetermined threshold value, and extracts an area that is equal to or less than the threshold value as the specific part. It is characterized by.

  According to the present invention, by appropriately setting the threshold, for example, the whole body or upper body of a person can be extracted as the specific part, and a gesture using the whole body or upper body of the person can be recognized.

  According to a fourth aspect of the present invention, in the first or second aspect of the invention, the specific part extracting unit binarizes the distance image using a value obtained by adding a predetermined value to the minimum distance value for each distance image as a threshold value. And a region that is equal to or less than a threshold value is extracted as the specific part.

  According to the present invention, by appropriately setting a predetermined value, for example, when using the distance image sensor by holding a hand and performing a gesture at a portion ahead of the wrist, it is more than the wrist of a human hand. The previous part can be easily extracted as the specific part.

  According to a fifth aspect of the invention, in the fourth aspect of the invention, the specific part extraction unit sets the predetermined value assuming that the specific part is a part ahead of the wrist of the hand directed to the distance image sensor. The gesture recognizing unit recognizes a gesture based on time-series data of feature values using the 3D centroids of the fingers and palms of the hand that are the specific parts extracted by the specific part extracting unit as feature quantities. It is characterized by.

  According to this invention, when the hand is held over the distance image sensor and a gesture is performed at a portion ahead of the wrist, the gesture recognition performance in the gesture recognition unit can be improved.

  According to a sixth aspect of the present invention, in the first or second aspect of the present invention, the specific part extracting unit is configured such that the number of pixels that are equal to or smaller than a threshold value with respect to a peripheral region of the pixel that is the minimum distance value in the distance image is a constant value. The distance image is adjusted so that the distance image is binarized with the threshold value, and a region that is equal to or less than the threshold value is extracted as the specific part.

  According to the present invention, for example, when using the distance image sensor by holding a hand and performing a gesture at a portion ahead of the wrist, the portion ahead of the wrist of a human hand is stably used as the specific portion. And can be extracted.

  A seventh aspect of the invention is characterized in that, in the third to sixth aspects of the invention, the specific part extracting unit binarizes only the input determination region in the distance image.

  According to the present invention, it is possible to increase the processing speed of the specific part extraction unit as compared with the case where binarization is performed on all the pixels in the distance image.

  In the invention of claim 1, it is not necessary to go to the installation location and touch the control target device or the control switch of the control target device, and it is not necessary for a person to wear anything, and it is easy to use, There is an effect that a specific part of a person can be extracted without being influenced by the background of the detection target area.

(Embodiment 1)
The gesture switch of the present embodiment is configured as shown in FIG. 1, and a distance image sensor DS that continuously generates a distance image having a pixel value as a distance value to an object Ob existing in a detection target area; Based on the distance image generated by the distance image sensor DS, the specific part extraction unit 5 that extracts a specific part of a person existing in the detection target area, and the shape of the specific part extracted by the specific part extraction unit 5 in time series A gesture recognition unit 6 for recognizing a predetermined gesture in a specific space based on various data, and a control output associated in advance with the gesture recognized by the gesture recognition unit 6 to a control target device (not shown) And a control unit 7. Here, the distance image sensor DS is installed, for example, on the ceiling 131 of the room R as shown in FIG. 2A, or at the corner on the ceiling 131 side of the room R as shown in FIG. However, the installation location of the distance image sensor DS is not particularly limited, as shown in FIG. 5C, and the specific site extraction unit 5 may select a specific site to be extracted. (Examples of specific parts will be described later with reference to FIG. 3). Note that a space E surrounded by a one-dot chain line in FIGS. 2A, 2 </ b> B, and 2 </ b> C schematically illustrates a detection target area of the distance image sensor DS.

  The distance image sensor DS includes a light emitting source 2 that irradiates light to a detection target area (target space) E, and receives a light from the target space and obtains an electrical output having an output value that reflects the received light amount. 1 is provided. The distance to the object Ob existing in the target space is the time from when the light is irradiated from the light source 2 to the target space until the reflected light from the object Ob enters (arrives) the light detection element 1 (“flight time”). "). However, since the flight time is a very short time at the nanosecond level, the intensity modulated light that was modulated so that the intensity of the light irradiating the target space periodically changes at a constant period was received. The time of flight is obtained using the phase of time.

That is, as shown in FIG. 5A, the intensity of light radiated from the light source 2 into the space changes as shown by curve A, and the amount of received light received by the light detecting element 1 changes as shown by curve B. For example, since the phase difference ψ corresponds to the flight time, the distance to the object Ob can be obtained by obtaining the phase difference ψ. Further, the phase difference ψ can be calculated using the received light amount of the curve B obtained at a plurality of timings of the curve A. For example, it is assumed that the received light amounts of curve B obtained at the timings of curve 0 at a phase of 0 degree, 90 degrees, 180 degrees, and 270 degrees are A0, A1, A2, and A3 (received light quantities A0, A1, A2,. A3 is indicated by hatching). However, the received light amount A0, A1, A2, A3 in each phase uses the received light amount integrated at a predetermined time Tw instead of the instantaneous value. Now, while obtaining the received light amounts A0, A1, A2, A3, the phase difference ψ does not change (that is, the distance to the object Ob does not change), and the reflectance of the object Ob does not change. To do. Further, it is assumed that the intensity of light emitted from the light emitting source 2 is modulated by a sine wave, and the intensity of light received by the light detection element 1 at time t is represented by A · sin (ωt + δ) + B. Here, A is the amplitude, B is the external light component, ω is the angular frequency, and δ is the phase. If the received light amounts A0, A1, A2, and A3 received by the light detecting element 1 are instantaneous values instead of the integrated values of the time Tw, the received light amounts A0, A1, A2, and A3 can be expressed as follows.
A0 = A · sin (δ) + B
A1 = A · sin (π / 2 + δ) + B
A2 = A · sin (π + δ) + B
A3 = A · sin (3π / 2 + δ) + B
Since δ = −ψ, A0 = −A · sin (ψ) + B, A1 = A · cos (ψ) + B, A2 = A · sin (ψ) + B, A3 = −A · cos (ψ ) + B, and as a result, the relationship between the received light amounts A0, A1, A2, A3 and the phase difference ψ is expressed as follows.
ψ = tan ⁻¹ {(A2−A0) / (A1−A3)} (Formula 1)
Although the instantaneous values of the received light amounts A0, A1, A2, and A3 are used in Equation 1, the phase difference ψ can be obtained by Equation 1 even if the integrated values at the time Tw are used as the received light amounts A0, A1, A2, and A3. it can.

  As described above, in order to modulate the intensity of light applied to the target space, the light source 2 includes, for example, a structure in which a large number of light emitting diodes are arranged on one plane, a combination of a semiconductor laser and a diverging lens, or the like. Is used. The light source 2 is driven by a modulation signal having a predetermined modulation frequency output from the control circuit unit 3, and the intensity of the light emitted from the light source 2 is modulated by the modulation signal. In the control circuit unit 3, for example, the intensity of light emitted from the light source 2 is modulated by a sine wave of 20 MHz. The intensity of the light emitted from the light source 2 may be modulated by a triangular wave, a sawtooth wave or the like in addition to the modulation by a sine wave. In short, any configuration is acceptable as long as the intensity is modulated at a constant period. May be adopted.

  The light detection element 1 includes a plurality of photosensitive portions 11 regularly arranged. A light receiving optical system 8 is disposed in the light incident path to the photosensitive portion 11. The photosensitive portion 11 is a portion where light from the target space is incident through the light receiving optical system 8 in the light detection element 1, and the photosensitive portion 11 generates an amount of charge corresponding to the amount of received light. In addition, the photosensitive portions 11 are arranged on the lattice points of the planar lattice, and are arranged in a matrix in which, for example, a plurality are arranged at equal intervals in the vertical direction (ie, the vertical direction) and the horizontal direction (ie, the horizontal direction). Is done.

  The light receiving optical system 8 associates the line-of-sight direction when viewing the target space from the light detection element 1 with each photosensitive portion 11. That is, the range in which light enters each photosensitive portion 11 through the light receiving optical system 8 can be regarded as a conical field of view having a small apex angle set for each photosensitive portion 11 with the center of the light receiving optical system 8 as the apex. . Therefore, if the reflected light emitted from the light emitting source 2 and reflected by the object Ob existing in the target space is incident on the photosensitive portion 11, the optical axis of the light receiving optical system 8 is changed depending on the position of the photosensitive portion 11 that has received the reflected light. The direction in which the object Ob exists can be known as the reference direction.

  Since the light receiving optical system 8 is generally arranged so that the optical axis is orthogonal to the plane on which the photosensitive portion 11 is arranged, the center of the light receiving optical system 8 is the origin, and the vertical and horizontal directions of the plane on which the photosensitive portion 11 is arranged If an orthogonal coordinate system is set in which the optical axis of the light receiving optical system 8 is in the direction of three axes, the angles (so-called azimuth and elevation) when the position of the object Ob existing in the target space is expressed in spherical coordinates are each This corresponds to the photosensitive portion 11. The light receiving optical system 8 can also be arranged so that the optical axis intersects at an angle other than 90 degrees with respect to the plane on which the photosensitive portions 11 are arranged.

  In the present embodiment, as described above, in order to obtain the distance to the object Ob, the received light amounts A0, A1, A2, and the like at the timing of four points synchronized with the intensity change of the light irradiated from the light source 2 to the target space. We are seeking A3. Therefore, it is necessary to control the timing to obtain the desired received light amount A0, A1, A2, A3. In addition, since the amount of charge generated in the photosensitive portion 11 is small in one cycle of intensity change of light irradiated from the light source 2 to the target space, it is desirable to accumulate the charge over a plurality of cycles. Therefore, as shown in FIG. 1, a plurality of charge accumulating units 13 for accumulating the charges generated in the respective photosensitive units 11 are provided, and the areas of the regions for generating the charges that can be used in the respective photosensitive units 11 are changed to change the respective areas. A plurality of sensitivity control units 12 for adjusting the sensitivity of the photosensitive unit 11 are provided.

  In each sensitivity control unit 12, the sensitivity of the photosensitive unit 11 corresponding to the sensitivity control unit 12 is increased at any one of the four points described above, and in the photosensitive unit 11 with increased sensitivity, the received light amount A0, Since charges corresponding to A1, A2, and A3 are mainly generated, charges corresponding to the received light amounts A0, A1, A2, and A3 can be accumulated in the charge accumulating unit 13 corresponding to the photosensitive unit 11.

  By the way, the sensitivity control unit 12 changes the amount of charge generated in each period by changing the area (substantial light receiving area) of the region that generates the charge that can be used in the photosensitive unit 11. The charges accumulated in 13 include not only the charges generated during the period in which the received light amounts A0, A1, A2, and A3 are obtained, but also the charges generated during other periods. Now, in the sensitivity control unit 12, the sensitivity in the period for generating the charges corresponding to the received light amounts A0, A1, A2, A3 is α, the sensitivity in the other periods is β, and the photosensitive unit 11 is a charge proportional to the received light amount. Is generated. Under this condition, the charge accumulating unit 13 that accumulates charges corresponding to the received light amount A0 is proportional to αA0 + β (A1 + A2 + A3) + βAx (Ax is the received light amount other than the period during which the received light amounts A0, A1, A2, and A3 are obtained). In the charge accumulating unit 13 that accumulates charges and accumulates charges corresponding to the received light quantity A2, charges proportional to αA2 + β (A0 + A1 + A3) + βAx are accumulated. As described above, when obtaining the phase difference ψ, (A2−A0) is obtained, and A2−A0 = (α−β) (A2−A0), and similarly, A1−A3 = (α−). β) Since (A1-A3), (A2-A0) / (A1-A3) theoretically has the same value regardless of the presence or absence of charge mixing. The phase difference ψ has the same value.

  The photodetecting element 1 including the photosensitive unit 11, the sensitivity control unit 12, and the charge accumulating unit 13 is configured as one semiconductor device, and the photodetecting element 1 transmits charges accumulated in the charge accumulating unit 13 to the outside of the semiconductor device. A charge extraction unit 14 is provided for extraction. The charge extraction unit 14 has the same configuration as the vertical transfer unit and horizontal transfer unit in the CCD image sensor.

  As described above, since each photosensitive unit 11 generates an amount of electric charge corresponding to the amount of received light, each of the received light amounts A0, A1, A2, A3 reflects the brightness of the object Ob. That is, the added value or average value of the received light amounts A0, A1, A2, A3 corresponds to the density value in the grayscale image. In other words, in addition to obtaining the distances from the received light amounts A0, A1, A2, A3 to the object Ob in each photosensitive portion 11, it is also possible to obtain the density value of the object Ob. In addition, since the distance and the density value of the object Ob are obtained using the photosensitive portion 11 at the same position, it is possible to obtain both the density value and the distance information for the same position.

  The electric charges extracted from the electric charge extraction unit 14 are given to the image generation unit 4 as an image signal, and the distance to the object Ob in the target space in the image generation unit 4 uses the above-described equation 1 to determine the received light amount A0, A1, A2. , A3. That is, the image generation unit 4 calculates the distance to the object Ob in each direction corresponding to each photosensitive unit 11, and calculates the three-dimensional information of the target space. By using this three-dimensional information, it is possible to generate a distance image in which the pixel values of the pixels matching each direction of the target space are distance values. In addition, the image generation unit 4 generates a grayscale image of the target space based on the density value obtained by each photosensitive unit 11. That is, the image generation unit 4 generates a grayscale image and a distance image. If the average value of the received light amounts A0, A1, A2, and A3 is used for the grayscale value, the influence of light from the light source 2 can be removed.

  With this configuration, the gray value and distance value of the target space can be obtained from the received light amounts A0, A1, A2, and A3 at the photosensitive portion 11 provided in the light detecting element 1, and the gray image and the distance image at substantially the same time can be obtained. And you can get Moreover, since each pixel of the grayscale image and the distance image has information in the same direction of the target space, the information obtained from the grayscale image and the distance image is used together, so that the target space is more than that using only the grayscale image. A lot of information can be obtained.

  Hereinafter, a specific structural example of the light detection element 1 will be described. The photodetecting element 1 shown in FIG. 6 has a plurality of (for example, 100 × 100) photosensitive portions 11 arranged in a matrix and is formed on a single semiconductor substrate. Each column in the vertical direction in the photosensitive portion 11 shares the semiconductor layer 21 that is integrally continuous, and the semiconductor layer 21 is used as a transfer path for charges in the vertical direction (electrons are used in this embodiment). It is possible to adopt a configuration in which a semiconductor substrate is provided with a horizontal transfer portion Th (see FIG. 7) which is a CCD that receives charges from one end of the semiconductor layer 21 and transfers the charges in the horizontal direction.

  That is, as shown in FIG. 7, the semiconductor layer 21 is used as the photosensitive portion 11 and the charge transfer path, and has a structure similar to a frame transfer (FT) type CCD image sensor. Similarly to the FT type CCD image sensor, a light-shielded accumulation region Db is provided adjacent to the imaging region Da in which the photosensitive portions 11 are arranged, and charges accumulated in the accumulation region Db are transferred to the horizontal transfer unit Th. To do. The transfer of charges from the imaging area Da to the storage area Db is performed at once in the vertical blanking period, and the horizontal transfer unit Th transfers charges for one horizontal line in one horizontal period. The charge extraction unit 14 illustrated in FIG. 1 represents a function including a horizontal transfer unit Th along with a function as a charge transfer path in the vertical direction in the semiconductor layer 21. However, the charge accumulation unit 13 does not mean the accumulation region Db, but represents a function of accumulating charges in the imaging region Da. In other words, the accumulation region Db is included in the charge extraction unit 14.

  The semiconductor layer 21 is doped with impurities, the main surface of the semiconductor layer 21 is covered with an insulating film 22 made of an oxide film, and a plurality of control electrodes 23 are arranged on the semiconductor layer 21 via the insulating film 22. . This light detection element 1 has a structure known as a MIS element, but a normal MIS element is that a plurality of (five in the illustrated example) control electrodes 23 are provided in a region functioning as one light detection element 1. Is different. A material is selected for the insulating film 22 and the control electrode 23 so that light having the same wavelength as the light emitted from the light source 2 to the target space can be transmitted. When light enters the semiconductor layer 21 through the insulating film 22, the semiconductor layer 21. A charge is generated inside the. The conductivity type of the semiconductor layer 21 in the illustrated example is n-type, and electrons e are used as charges generated by light irradiation. FIG. 6 shows only the region corresponding to one photosensitive portion 11, and a plurality of regions having the structure shown in FIG. 6 are arranged on the semiconductor substrate (not shown) and the charge extraction is performed. A structure to be part 14 is provided. The vertical transfer unit provided as the charge extraction unit 14 is assumed to transfer charges in the left-right direction in FIG. 6, but it is also possible to adopt a configuration in which charges are transferred in a direction orthogonal to the plane in FIG. 6. is there. In addition, when transferring charges in the left-right direction of FIG. 6, it is desirable to set the width dimension of the control electrode 23 in the left-right direction to about 1 μm.

  In the light detection element 1 having this structure, when a positive control voltage + V is applied to the control electrode 23, a potential well (depletion layer) 24 that accumulates electrons e in a portion corresponding to the control electrode 23 is formed in the semiconductor layer 21. The That is, when light is applied to the semiconductor layer 21 with a control voltage applied to the control electrode 23 so as to form the potential well 24 in the semiconductor layer 21, a part of the electrons e generated in the vicinity of the potential well 24. Are captured in the potential well 24 and accumulated in the potential well 24, and the remaining electrons e disappear by recombination with holes in the deep portion of the semiconductor layer 21. Further, the electrons e generated at a location away from the potential well 24 are also extinguished by recombination with holes in the deep portion of the semiconductor layer 21.

  Since the potential well 24 is formed at a portion corresponding to the control electrode 23 to which the control voltage is applied, the potential well 24 along the main surface of the semiconductor layer 21 is changed by changing the number of the control electrodes 23 to which the control voltage is applied. (In other words, the area of a region that generates a charge that can be used on the light receiving surface) can be changed. That is, changing the number of control electrodes 23 to which the control voltage is applied means adjusting sensitivity in the sensitivity control unit 12. For example, when the control voltage + V is applied to three control electrodes 23 as shown in FIG. 6A, and when the control voltage + V is applied to one control electrode 23 as shown in FIG. 6B. Since the area occupied by the potential well 24 on the light receiving surface changes, the area of the potential well 24 is larger in the state of FIG. 6A, so that the same amount of light is obtained as compared with the state of FIG. 6B. As a result, the ratio of the charge that can be used increases and the sensitivity of the photosensitive portion 11 is substantially increased. Thus, it can be said that the photosensitive portion 11 and the sensitivity control portion 12 are constituted by the semiconductor layer 21, the insulating film 22, and the control electrode 23. The potential well 24 functions as the charge accumulation unit 13 because it holds charges generated by light irradiation.

  In order to extract charges from the potential well 24, a technique similar to that of the FT type CCD may be employed. After the electrons e are accumulated in the potential well 24, a control voltage of an applied pattern different from that during charge accumulation is controlled. By applying the voltage to the electrode 23, the electrons e accumulated in the potential well 24 can be transferred in one direction (for example, the right direction in FIG. 6). That is, the semiconductor layer 21 can be used as a charge transfer path in the same manner as the vertical transfer portion of the CCD. Further, the electric charge is transferred through the horizontal transfer portion Th shown in FIG. 7 and is taken out of the photodetecting element 1 from an electrode (not shown) provided on the semiconductor substrate. In short, by controlling the application pattern of the control voltage to the control electrode 23, the sensitivity of each photosensitive portion 11 is controlled, charges generated by light irradiation are integrated, and the integrated charges are transferred. Can do.

  The sensitivity control unit 12 in the present embodiment switches the sensitivity of the photosensitive unit 11 in two steps, high and low, by switching the area of a region for generating available charges into two steps, large and small, and the received light amount A0, A1, A2, The sensitivity corresponding to any one of A3 is set to high sensitivity only during a period when the photosensitive portion 11 is to be generated (the area of the charge generation region is increased), and the sensitivity is decreased during other periods. The period of high sensitivity and the period of low sensitivity are set in synchronization with the modulation signal that drives the light emitting source 2. In addition, after the charges are accumulated in the potential well 24 over a plurality of periods of the modulation signal, the charges are extracted to the outside of the light detection element 1 through the charge extraction unit 14. Charges are accumulated over a plurality of periods of the modulation signal because the period for which the photosensitive unit 11 can generate usable charges within one period of the modulation signal is short (for example, if the frequency of the modulation signal is 20 MHz). This is because less than a quarter of 50 ns is generated. That is, by integrating charges for a plurality of periods of the modulation signal, signal charges (charges corresponding to light emitted from the light emission source 2) and noise charges (external light components and shots generated inside the light detection element 1). A large signal-to-noise ratio can be obtained, and a large SN ratio can be obtained.

  By the way, if the charges corresponding to the four kinds of received light amounts A0, A1, A2, and A3 necessary for obtaining the phase difference ψ are generated by one photosensitive portion 11, the resolution in the line-of-sight direction is increased. There arises a problem that the time difference for obtaining the charges corresponding to the received light amounts A0, A1, A2, A3 becomes large. On the other hand, if the charges corresponding to the received light amounts A0, A1, A2, A3 are generated by the four photosensitive portions 11, respectively, the time difference for obtaining the charges corresponding to the received light amounts A0, A1, A2, A3 becomes small. However, a shift occurs in the line-of-sight direction for obtaining the four types of charges, and the resolution in the line-of-sight direction decreases. Therefore, in the present embodiment, a configuration is adopted in which two types of charges corresponding to the received light amounts A0, A1, A2, and A3 are generated in one cycle of the modulation signal by using the two photosensitive portions 11. . That is, two photosensitive portions 11 are used as a set, and light from the same line-of-sight direction is incident on the two photosensitive portions 11 in the set.

  By adopting the above-described configuration, the resolution in the line-of-sight direction can be made relatively high, and the time difference for generating charges corresponding to the received light amounts A0, A1, A2, and A3 can be reduced. That is, by reducing the time difference for generating the charges corresponding to the received light amounts A0, A1, A2, and A3, the distance detection accuracy of the object Ob moving in the target space can be kept relatively high. Can do. In the configuration of this embodiment, the resolution in the line-of-sight direction is lower than that in the case where charges corresponding to four types of received light amounts A0, A1, A2, and A3 are generated by one photosensitive unit 11, but the line-of-sight direction The resolution can be improved by downsizing the photosensitive unit 11 or designing the light receiving optical system 8.

  The operation will be specifically described below. In the example shown in FIG. 6, an example in which five control electrodes 23 are provided for one photosensitive portion 11 is shown. However, two control electrodes 23 on both sides are charged (electrons e) by the photosensitive portion 11. Is formed to form a potential barrier for preventing the charge from flowing out to the adjacent photosensitive portion 11 while the two photosensitive portions 11 are used as a pair. Since any one of the photosensitive portions 11 forms a barrier between the potential wells 24 of the photosensitive portions 11, it is only necessary to provide three control electrodes 23 for each photosensitive portion 11. With this configuration, the occupation area per one photosensitive portion 11 is reduced, and it is possible to suppress a decrease in resolution in the line-of-sight direction while using the two photosensitive portions 11 as a set.

  Here, as shown in FIG. 8, the numbers (1) to (6) are assigned to the control electrodes 23 in order to distinguish the three control electrodes 23 provided in the two photosensitive sections 11 respectively. Attached. The two photosensitive portions 11 having the control electrodes 23 assigned with the numbers (1) to (6) correspond to one line-of-sight direction and constitute pixels in the image sensor. In addition, it is desirable to provide an overflow drain in association with the photosensitive portion 11 for each pixel.

  8A and 8B show a state in which a control voltage + V is applied to the control electrode 23 in a different application pattern (a state in which a control voltage + V is applied between a substrate electrode (not shown) provided on the semiconductor substrate and the control electrode 23). As can be seen from the shape of the potential well 24, in FIG. 8A, a positive control voltage + V is applied to the control electrodes (1) to (3) of the two photosensitive portions 11 serving as one pixel. In addition, a positive control voltage + V is applied to the central control electrode (5) among the remaining control electrodes (4) to (6). In FIG. 8B, a positive control voltage + V is applied to the central control electrode (2) among the control electrodes (1) to (3), and the remaining control electrodes (4) to (6) are applied. A positive control voltage + V is applied. In other words, the application pattern of the control voltage + V applied to the two photosensitive portions 11 constituting one pixel is alternately replaced. The timing of switching the application pattern of the control voltage + V applied to the two photosensitive portions 11 is the timing of the opposite phase (the phase is 180 degrees different) in the modulation signal. In addition, except for the period in which the control voltage + V is simultaneously applied to the three control electrodes 23 provided in each photosensitive portion 11, one central control electrode 23 (that is, the control electrode) provided in each photosensitive portion 11 is provided. (2) The control voltage + V is applied only to (5)), and the other control electrodes 23 are kept at 0V.

  For example, when the charges corresponding to the received light amounts A0 and A2 are alternately generated in the two photosensitive portions 11 constituting one pixel, the charges corresponding to the received light amount A0 in one photosensitive portion 11 as shown in FIG. While the control voltage + V is applied to the three control electrodes (1) to (3) in order to generate the signal, the other photosensitive portion 11 has one charge to hold the charge corresponding to the received light quantity A2. A control voltage + V is applied only to the control electrode (5). Similarly, while the control voltage + V is being applied to the three control electrodes (4) to (6) in order to generate a charge corresponding to the received light amount A2 in one photosensitive unit 11, the other photosensitive unit 11 is exposed. The unit 11 applies the control voltage + V only to one control electrode (2) in order to hold the charge corresponding to the received light quantity A0. In addition, the control voltage + V is applied only to the control electrodes (2) and (5) except for the period in which charges corresponding to the received light amounts A0 and A2 are generated. FIGS. 5B and 5C show the timing of applying the control voltage + V to the control electrodes (1) to (6) when accumulating charges corresponding to the received light amounts A0 and A2. 5B and 5C, the shaded portion indicates a state where the control voltage + V is applied, and the blank portion indicates a state where no voltage is applied to the control electrodes (1) to (6).

  The same applies to the case where the charges corresponding to the received light amounts A1 and A3 are generated in the two photosensitive portions 11 constituting one pixel, and the case where the charges corresponding to the received light amounts A0 and A2 are generated is different from the control electrode 23. The only difference is that the timing at which the control voltage + V is applied differs by 90 degrees in the phase of the modulation signal. In addition, the charge is transferred from the imaging region Da to the accumulation region Db between a period for generating charges corresponding to the received light amounts A0 and A1 and a period for generating charges corresponding to the received light amounts A1 and A3. That is, charges corresponding to the received light amount A0 are accumulated in the potential well 24 corresponding to the control electrodes (1) to (3), and charges corresponding to the received light amount A2 correspond to the control electrodes (4) to (6). When accumulated in the potential well 24, the charges corresponding to the received light amounts A0 and A2 are taken out. Next, charges corresponding to the received light amount A1 are accumulated in the potential well 24 corresponding to the control electrodes (1) to (3), and charges corresponding to the received light amount A3 are applied to the control electrodes (4) to (6). When accumulated in the corresponding potential well 24, charges corresponding to these received light amounts A1 and A3 are taken out to the outside. By repeating such an operation, the charges corresponding to the received light amounts A0, A1, A2, and A3 of the four sections can be taken out of the light output element 1 by two reading operations, and the extracted charges are used. The phase difference ψ can be obtained. For example, in order to obtain an image of 30 frames per second, the period for generating charges corresponding to the received light amounts A0 and A2 and the period for generating charges corresponding to the received light amounts A1 and A3 are shorter than 1/60 second. A short period.

  In the above example, the control voltage applied simultaneously to the three control electrodes 23 ((1) to (3) or (4) to (6)) and one control electrode 23 ((2) or (5)) Since the control voltage applied only to is equal, the area of the potential well 24 changes, but the depth of the potential well 24 is equal. In this case, the charges generated at the control electrode 23 ((1) (3) or (4) (6)) to which no control voltage is applied flow into the potential well 24 with a similar probability. That is, by applying the control voltage + V to only one of the three control electrodes 23 constituting the photosensitive portion 11, the region functioning as the charge accumulation portion 13 and all the three control electrodes 23 are applied. A similar amount of charge flows into both the region to which the control voltage + V is applied. That is, since the noise component flowing into the potential well 24 holding the charge is relatively large, the dynamic range is lowered.

  Therefore, as shown in FIG. 9, it is desirable to set the depth of the small-area potential well 24 to be smaller than the depth of the large-area potential well 24. The control voltage applied simultaneously to the control electrodes (1) to (3) or (4) to (6) is higher than the control voltage applied only to one control electrode (2) or (5). It is desirable to set. As described above, the potential well 24 that mainly generates charges (electrons e) is made deeper than the potential well 24 that mainly holds charges, so that the control electrode (1) to which no control voltage is applied is applied. Charges generated at the sites corresponding to (3) or (4) and (6) are likely to flow into the deeper potential well 24. That is, as compared with the case where a constant control voltage + V is applied to the control electrode 23, the noise component flowing into the potential well 24 holding the charge can be reduced.

  In the configuration example of the distance image sensor DS described above, four periods corresponding to the received light amounts A0, A1, A2, and A3 are set so that the phase interval is 90 degrees within one period of the modulation signal. However, if the phase for the modulation signal is known, the four periods can be set at appropriate intervals other than 90 degrees. However, the formula for obtaining the phase difference ψ differs if the interval is different. Also, the period of taking out the charge corresponding to the amount of received light in the four periods is within one period of the modulation signal as long as the reflectance and the external light component of the object Ob do not change and the phase difference ψ does not change. It is not essential to take out four signal charges. Furthermore, when there is an influence of disturbance light such as sunlight or illumination light, it is desirable to dispose an optical filter that transmits only the wavelength of light emitted from the light source 2 in front of the photosensitive portion 11. In the configuration example described with reference to FIGS. 8 and 9, three control electrodes 23 are associated with each photosensitive portion 11. However, four or more control electrodes 23 may be provided. In the above example, the same configuration as the FT type CCD image sensor is adopted, but the same configuration as the interline transfer (IT) method and the frame interline transfer (FIT) method is adopted. Is also possible.

  Next, the specific part extraction unit 5 that extracts a specific part of a person existing in the detection target area based on the distance image generated by the distance image sensor DS described above will be described. Here, in order to reduce the incidence of external light components on the light detection element 1 when generating a distance image, the target space is irradiated with infrared light from the light source 2 and an infrared transmission filter is provided in front of the light detection element 1. Assume that it is arranged. Therefore, the grayscale image becomes a grayscale image with respect to infrared rays.

  In the gesture switch of this embodiment, as shown in FIG. 1, the distance image generated by the image generation unit 4 of the distance image sensor DS is given to the specific part extraction unit 5, and the specific part extraction unit 5 converts the distance image into the distance image. Based on the data extracted from the specific part extraction unit 5, the specific part of the person existing in the detection target area is extracted based on the data output from the specific part extraction unit 5, and the specific space is the same as the detection target area. A predetermined gesture in a space, that is, a whole range of the distance image, or a part of the space in the detection target area, that is, a part of the distance image) Is recognized. In short, the gesture switch according to the present embodiment is based on a distance image generated by one distance image sensor DS that continuously generates a distance image having a pixel value as a distance value to the object Ob existing in the detection target area E. The specific part of the person existing in the detection target area E is continuously extracted by the specific part extraction unit 5, and the person in the specific space is based on the data output from the specific part extraction unit 5 in the gesture recognition unit 6. It is easy to use because there is no need to go to the installation location and touch the control target device or the control switch for the control target device, and there is no need for a person to wear anything. In addition, it is possible to extract a specific part of a person without being affected by the background of the detection target area E.

  As described above, the distance image generated by the image generation unit 4 of the distance image sensor DS has a pixel value as the distance value from the distance image sensor DS to the object Ob. When extracting a specific part of a person, the distance image is binarized using a value obtained by adding a predetermined value to the minimum distance value for each distance image as a threshold, and an area that is equal to or less than the threshold is extracted as the specific part. . Therefore, by appropriately setting the predetermined value, for example, when using the distance image sensor DS by holding a hand and performing a gesture at a portion ahead of the wrist, a portion ahead of the wrist of a human hand is used. It can be easily extracted as a specific part. In this case, if a distance image (distance image data) as shown in FIG. 3A is given from the distance image sensor DS to the specific part extraction unit 5, the specific part extraction unit 5 as shown in FIG. A binary image is obtained (the pixel of the binarized image is white if the pixel has a distance value less than or equal to the threshold in the distance image, and the pixel of the binarized image is black if the pixel has a distance value greater than the threshold in the distance image. The white region W is extracted from such a binarized image as a specific part (a part ahead of the wrist of the hand). The distance image in FIG. 3A shows an example in which the color of the pixel is changed in four stages according to the distance value. The image generation unit 4 uses L1, L2, L3, L4, L5 (reference distance values) as reference distance values. L1 <L2 <L3 <L4 <L5) is set, and an area composed of pixels whose distance value is equal to or smaller than the reference distance value L1 is F1, and an area composed of pixels whose distance value is larger than the reference distance value L1 and equal to or smaller than the reference distance value L2. F2, an area composed of pixels with a distance value greater than the reference distance value L2 and less than or equal to the reference distance value L3, F3, an area composed of pixels with a distance value greater than the reference distance value L3 and less than or equal to the reference distance value L4, distance Assuming that an area composed of pixels whose values are larger than the reference distance value L4 is F5, the area F1 is white, the area F5 is black, the area F2 is gray near white, the area F4 is near gray, and the area F3 is area F2. And gray in between area F4 And generating a distance image such that (Note, P point in FIG. 3 (a) shows the position of the pixel distance value is minimized). On the other hand, the binarized image in FIG. 3B shows the binarized image when the predetermined value is set so that the portion ahead of the wrist of the human hand is the white region W. .

By the way, the method of extracting the specific part of the person from the distance image in the specific part extraction unit 5 is not limited to the above example. For example, the distance image is binarized with a predetermined threshold value, and an area that is equal to or less than the threshold value is used as the contour of the human body. If extraction is performed, by appropriately setting the threshold value, for example, the whole body or upper body of a person can be extracted as the specific part, and a gesture using the whole body or upper body of the person can be recognized. In addition, the specific part extraction unit 5 adjusts the threshold value so that the number of pixels that are equal to or less than the threshold value with respect to the surrounding area of the pixel that has the minimum distance value in the distance image, and sets the distance image to the threshold value. If binarization is performed and an area that is equal to or less than the threshold value is extracted as a specific part, for example, when a hand is held over the distance image sensor DS and a gesture is performed at a part beyond the wrist, It is possible to stably extract the portion ahead of the wrist of the hand as the specific part. In the specific part extraction unit 5 in this case, the threshold value is adjusted by the following flows (a) to (e).
(A) A process for searching for a pixel having a minimum distance value from the distance image is performed as a reference pixel.
(B) Processing for setting the initial value of the threshold to the minimum value is performed.
(C) In the distance image, a process for examining the number of pixels whose distance value is equal to or smaller than a threshold value in the peripheral area of the reference pixel is performed.
(D) When the number of pixels is equal to or smaller than a predetermined constant value, the value 1 is added to the threshold value, and the process returns to the process (c). When it exceeds the predetermined value, the process proceeds to the next process.
(E) The threshold immediately before the number of pixels> the above constant value is determined as the final threshold.

  Further, if the specific part extraction unit 5 binarizes only a specific input determination region in the distance image and extracts the specific part based on the comparison result with the threshold value, all pixels in the distance image As compared with the case where binarization is performed, the processing speed of the specific part extraction unit 5 can be increased.

  In the gesture recognition unit 6, for example, a gesture is recognized based on time-series data of a shape (information) regarding the specific part extracted by the specific part extraction unit 5. In this case, when the gesture recognition unit 6 recognizes the gesture, for example, the 3D centroids O1 to O5 of the five fingers and the 3D centroid O6 of the palm in the shape of the specific part as shown in FIG. If the feature time series data is learned by HMM (Hidden Markov Model) and the gesture is recognized, it is used by holding the hand to the distance image sensor DS and performing the gesture at the part ahead of the wrist. In doing so, the gesture recognition performance (gesture recognition accuracy) in the gesture recognition unit 6 can be improved. Note that the above-described feature amount is not limited to the 3D center of gravity of the finger, and the tilt of the finger, the number of fingers, and the like may be defined as the feature amount. In addition, a well-known method other than the above-described method may be adopted as the gesture recognition method. For example, when recognizing a gesture based on time-series data of a shape related to a specific part extracted by the specific part extraction unit 5, for example, It may be recognized that a predetermined gesture has been made when the number of fingers changes from 1 to 2 to 3 over time (that is, when the shape of a specific part changes in time series). In addition, a predetermined gesture is performed when a predetermined shape (for example, the shape of “guo”, “chokki”, “paa”, etc. in general when playing with a single hand) is continued for a specified time or longer). If the predetermined shape continues for less than the specified time, it may not be recognized as a gesture. Here, if the specified time is appropriately set, it is possible to prevent a movement of a hand not intended to control the control target device from being recognized as a gesture and giving a control output to the control target device.

  The control unit 7 includes a memory (not shown) that stores a table in which gestures and control outputs are associated with each other in advance. The control output corresponding to the gesture recognized by the gesture recognition unit 6 is sent to the control target device. To give. Here, as shown in FIG. 10 (a), two input spaces (here, the above-mentioned specific spaces) G1, as spaces for receiving an input by a predetermined gesture within the detection target area E of the distance image sensor DS. When G2 is set and the person M performs a predetermined gesture in the vicinity of the input space G2, the control unit 7 causes the input region H2 (in FIG. 10B) corresponding to the input space G2 ( In the illustrated example, the number of pixels of a specific portion (the area below the threshold and the white area W) is the number of pixels of the input area H1 corresponding to the input space G1 is a black area). If the control output is given to the device to be controlled when the ratio exceeds the specified value, a motion that is not intended to control the device to be controlled in a space other than the specific space is a gesture. Be recognized can be prevented from control output to the control target device is given. The same applies when the person M performs a gesture in the vicinity of the input space G1, and the input spaces G1 and G2 can be assigned to different control target devices. Also, as shown in FIG. 11, one input space (here, a specific space) G that is a space for receiving an input by a predetermined gesture is set in the detection target area E of the distance image sensor DS, Even when the person M performs a gesture in the vicinity of the input space G, the control unit 7 determines that the ratio of the number of pixels in the specific part to the number of pixels in the input area corresponding to the input space G in the distance image exceeds the specified value. If the control output is given to the control target device from time to time, the control output is given to the control target device even if a movement that is not intended to control the human control target device in a space other than the specific space is recognized as a gesture. Can be prevented.

  As shown in FIG. 12A, a manipulator 50 in a virtual three-dimensional space is displayed on the screen in a room R where the detection target area E of the distance image sensor DS is set as shown in FIG. The display device U is provided, the hand H of the person M is extracted as a specific part by the specific part extraction unit 5, and the virtual hand H ′ for operating the manipulator 50 is displayed on the screen of the display apparatus U to display the hand of the person M The movement of the virtual hand H ′ may be synchronized with the movement of H (the arrows in FIGS. 12A and 12B indicate the directions of movement of the hands H and H ′, respectively). In short, the movement of the hand H extracted by the specific part extraction unit 5 may be used as the operation movement of the control target device in the virtual space.

(Embodiment 2)
The basic configuration of the gesture switch of the present embodiment is substantially the same as that of the first embodiment. In the specific configuration of the sensitivity control unit 12, as a technique for adjusting the ratio of charges given to the charge accumulation unit 13, the photosensitive unit 11 starts. The difference is that both the technique for adjusting the passing rate to the charge accumulating unit 13 and the technique for adjusting the discard rate for discarding charges from the photosensitive unit 11 are used. Since other configurations are the same as those of the first embodiment, illustration and description thereof are omitted.

  In the technique of adjusting the pass rate and the discard rate in the sensitivity control unit 12, as shown in FIG. 13, a gate electrode 12a is provided between the photosensitive unit 11 and the charge accumulation unit 13, and the pass voltage applied to the gate electrode 12a. Is changed to control the movement of charges from the photosensitive portion 11 to the charge accumulating portion 13 (that is, the passing rate). Further, by providing the charge discarding part 12c and changing the discarding voltage applied to the disposal electrode 12b attached to the charge discarding part 12c, the movement of the charge from the photosensitive part 11 to the charge discarding part 12c (that is, the discard rate) is changed. Control. The charge accumulating units 13 are provided so as to correspond one-to-one for each photosensitive unit 11, and the charge discarding units 12c are provided so as to correspond to the plurality of photosensitive units 11 so as to correspond one-to-many. In the illustrated example, all of the photosensitive portions 11 of the photodetecting element 1 share a set of discarding electrode 12b and charge discarding portion 12c.

  In order to control the sensitivity, it is conceivable to control only the pass rate from the photosensitive unit 11 to the charge accumulating unit 13 without discarding the charge from the photosensitive unit 11. Since charges remain in the unit 11 for a while, unnecessary residual charges out of the charges generated in the photosensitive unit 11 are mixed as noise components in the used charges (hereinafter referred to as signal charges). Therefore, in order to prevent the residual charge from being mixed into the signal charge, not only the passing voltage applied to the gate electrode 12a but also the discard voltage applied to the discard electrode 12b is controlled. The charge accumulating unit 13 receives received light amounts A0, A1, A2 in a specific section synchronized with the modulation signal in a plurality of cycles (tens of thousands to hundreds of thousands) of the modulation signal of the light emitted from the light source 2 to the space. , A3 are accumulated, the signal charges accumulated for each charge accumulation in each section are taken out, and the charges in the next section are accumulated. Here, the signal charge obtained in the charge accumulating unit 13 is extracted by the charge extracting unit 14 (see FIG. 1) as in the first embodiment.

  The photodetecting element 1 having the sensitivity control unit 12 shown in FIG. 13 can be realized by a CCD image sensor having an overflow drain, and an interline transfer (IT) method is used as a charge transfer method in the CCD image sensor. Adopted.

  FIG. 14 shows a configuration of an interline transfer type CCD image sensor having a vertical overflow drain. The illustrated example is a two-dimensional image sensor in which a plurality of photodiodes 41 serving as the photosensitive portions 11 are arranged in a horizontal direction and a vertical direction (3 × 4 in the figure), and the photodiodes 41 arranged in the vertical direction are arranged. A vertical transfer register 42 made of a CCD is provided on the right side of each column, and a horizontal transfer register 43 made of a CCD is provided below the area where the photodiodes 41 and the vertical transfer registers 42 are arranged. The vertical transfer register 42 includes two transfer electrodes 42 a and 42 b for each photodiode 41, and the horizontal transfer register 43 includes two transfer electrodes 43 a and 43 b for each vertical transfer register 42.

  The photodiode 41, the vertical transfer register 42, and the horizontal transfer register 43 are formed on one semiconductor substrate 40, and the photodiode 41, the vertical transfer register 42, and the horizontal transfer register 43 are formed on the main surface of the semiconductor substrate 40. An overflow electrode 44 which is an aluminum electrode is provided so as to directly contact the semiconductor substrate 40 without going through an insulating film over the entire circumference of the semiconductor substrate 40 so as to surround the whole. If an appropriate disposal voltage that is positive with respect to the semiconductor substrate 40 is applied to the overflow electrode 44, electrons (charges) generated by the photodiode 41 are discarded through the overflow electrode 44. The overflow electrode 44 functions as the discard electrode 12b because a discard voltage is applied when discarding unnecessary charges among the charges generated in the photodiode 41 which is the photosensitive portion 11, and the power supply for applying the discard voltage to the overflow electrode 44 Functions as a charge discarding unit 12c that discards electrons (charges) generated in the photosensitive unit 11. The surface of the semiconductor substrate 40 is covered with a light-shielding film 46 (see FIG. 15) except for the portion corresponding to the photodiode 41.

For the CCD image sensor shown in FIG. 14, a portion related to one photodiode 41 is cut out and shown in FIG. 15. An n-type semiconductor is used for the semiconductor substrate 40, and a well region 31 made of a p-type semiconductor is formed in a region straddling the photodiode 41 and the vertical transfer register 42 on the main surface of the semiconductor substrate 40. The well region 31 is formed so that the thickness dimension of the region corresponding to the vertical transfer register 42 is larger than the region corresponding to the photodiode 41. An n ^{+ -type} semiconductor layer 32 is provided in a region corresponding to the photodiode 41 in the well region 31, and the photodiode 41 is formed by a pn junction between the well region 31 and the n ^{+ -type} semiconductor layer 32. A surface layer 33 made of a p ^{+ type} semiconductor is laminated on the surface of the photodiode 41. The surface layer 33 is provided for the purpose of controlling the vicinity of the surface of the n ^{+ -type} semiconductor layer 32 so as not to be a charge passage path when the charge generated by the photodiode 41 is moved to the vertical transfer register 42. Such a structure is known as a buried photodiode.

An accumulation transfer layer 34 made of an n-type semiconductor is overlaid in a region corresponding to the vertical transfer register 42 in the well region 31. The surface of the accumulation / transfer layer 34 and the surface of the surface layer 33 are substantially flush with each other, and the thickness dimension of the accumulation / transfer layer 34 is larger than the thickness dimension of the surface layer 33. The accumulation transfer layer 34 is in contact with the surface layer 33, but a separation layer 35 made of a p ^{+ type} semiconductor having the same impurity concentration as that of the surface layer 33 is interposed between the storage layer 34 and the n ^{+ type} semiconductor layer 32. Transfer electrodes 42 a and 42 b are disposed on the surface of the accumulation transfer layer 34 via an insulating film 45. Two transfer electrodes 42a and 42b are provided for each photodiode 41, and one of the two transfer electrodes 42a and 42b is formed wider than the other in the vertical direction. Specifically, as shown in FIG. 16, of the two transfer electrodes 42a and 42b corresponding to one photodiode 41, the narrow transfer electrode 42b is formed in a flat plate shape, and the wide transfer electrode 42a. Are arranged on the same plane as the narrow transfer electrode 42b and disposed between the pair of transfer electrodes 42b, and from both ends in the vertical direction (left and right direction in FIG. 16) of the flat plate portion. And a curved portion that extends and overlaps the transfer electrode 42b. Here, the insulating film 45 is formed of SiO ₂ , the transfer electrodes 42 a and 42 b are formed of polysilicon, and the transfer electrodes 42 a and 42 b are insulated from each other through the insulating film 45. Further, the surface of the light detection element 1 is covered with a light shielding film 46 except for a portion where light is incident on the photodiode 41. In the well region 31, the region corresponding to the vertical transfer register 42 and the storage transfer layer 34 are formed over the entire length of the vertical transfer register 42. Therefore, the storage transfer layer 34 has a wide transfer electrode 42a and a narrow transfer electrode 42b. And are alternately arranged.

  In the photodetector 1 described above, the photodiode 41 corresponds to the photosensitive portion 11, the transfer electrode 42 a corresponds to the gate electrode 12 a, the overflow electrode 44 corresponds to the discard electrode 12 b, and the vertical transfer register 42 corresponds to the charge accumulation portion 13. And functions as a part of the charge extraction unit 14. Further, the horizontal transfer register 43 also becomes a part of the charge extraction unit 14. That is, if light enters the photodiode 41, a charge is generated, and the ratio of the charge generated as a signal charge to the vertical transfer register 42 among the charges generated by the photodiode 41 is equal to the passing voltage applied to the transfer electrode 42a and the overflow. It can be determined according to the relationship with the waste voltage applied to the electrode 44. When a pass voltage is applied to the transfer electrode 42a, a potential well is formed in the storage transfer layer 34, and the depth of the potential well can be controlled by controlling the pass voltage. Therefore, by controlling the depth of the potential well and the time during which the passing voltage is applied, the ratio of charges delivered from the photodiode 41 to the vertical transfer register 42 can be adjusted. Further, if the discard voltage applied to the overflow electrode 44 is controlled, the potential gradient between the photodiode 41 and the semiconductor substrate 40 can be controlled. Therefore, if the potential gradient and the time for applying the discard voltage are controlled. The rate of charge delivered to the vertical transfer register 42 can be adjusted.

  The signal charges delivered from the photodiode 41 to the vertical transfer register 42 are signals corresponding to the received light amounts A0, A1, A2, and A3 of each one of the four received light amounts A0, A1, A2, and A3 described above. It is read each time charge is accumulated. For example, when a signal charge corresponding to the received light amount A0 is accumulated in a potential well formed corresponding to each photodiode 41, the signal charge is read, and then a signal charge corresponding to the received light amount A1 is accumulated in the potential well. Then, the operation of reading the signal charge again is repeated.

It is a block diagram of the gesture switch of Embodiment 1. It is explanatory drawing of the installation place of the distance image sensor used for the same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing of the principal part of the photon detection element used for the same as the above. It is a top view of the photon detection element used for the same as the above. It is operation | movement explanatory drawing of the principal part of the photon detection element used for the same as the above. It is operation | movement explanatory drawing of the principal part of the photon detection element used for the same as the above. It is operation | movement explanatory drawing of the other structural example same as the above. It is operation | movement explanatory drawing of the other structural example same as the above. It is operation | movement explanatory drawing of the other structural example same as the above. 10 is a block diagram illustrating a configuration example of a sensitivity control unit in the gesture switch of Embodiment 2. FIG. It is a top view which shows the structural example of the photon detection element used for the same as the above. It is a principal part disassembled perspective view of the photon detection element shown in FIG. It is the sectional view on the AA line of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Light detection element 2 Light emission source 3 Control circuit part 4 Image generation part 5 Specific site | part extraction part 6 Gesture recognition part 7 Control part 11 Photosensitive part Ob Object DS Distance image sensor

Claims (7)

  A distance image sensor that continuously generates a distance image having a pixel value as a distance value to an object existing in the detection target area, and identification of a person existing in the detection target area based on the distance image generated by the distance image sensor A specific part extraction unit for extracting a part, a gesture recognition unit for recognizing a predetermined gesture in a specific space based on time-series data of the shape of the specific part extracted by the specific part extraction unit, and a gesture recognition unit A gesture switch comprising: a control unit that gives a control output associated with the gesture recognized in advance to the device to be controlled.   The control unit provides the control output to the control target device when a ratio of the number of pixels of the specific part to the number of pixels of the specific space in the distance image exceeds a specified value. The gesture switch according to 1.   The gesture switch according to claim 1, wherein the specific part extraction unit binarizes the distance image with a predetermined threshold and extracts a region that is equal to or less than the threshold as the specific part.   The specific part extraction unit binarizes the distance image using a value obtained by adding a predetermined value to a minimum distance value for each distance image as a threshold, and extracts a region that is equal to or less than the threshold as the specific part. The gesture switch according to claim 1 or 2.   The specific part extraction unit is set with the predetermined value on the assumption that a part ahead of a wrist of a hand directed to the distance image sensor is the specific part, and the gesture recognition unit includes the specific part extraction unit. The gesture switch according to claim 4, wherein the gesture is recognized based on time-series data of the feature amount using the 3D centroid of each of the finger and the palm as the specific portion extracted by the feature.   The specific part extraction unit adjusts the threshold value so that the number of pixels that are equal to or less than the threshold value with respect to the surrounding area of the pixel that has the minimum distance value in the distance image is a constant value, and sets the distance image to the threshold value of 2 The gesture switch according to claim 1, wherein the gesture switch is extracted and an area that is equal to or less than the threshold is extracted as the specific part.   The gesture switch according to any one of claims 3 to 6, wherein the specific part extraction unit binarizes only the input determination region in the distance image.