JP2006081771A - Method and apparatus for estimating information about living organisms using electromagnetic waves - Google Patents

Abstract

[PROBLEMS] To estimate information such as a person's position, three-dimensional posture, behavior, etc. without being affected by "lighting", "temperature", "clothes and shoes worn by a person", etc. Estimate high.
In estimation of information related to a living organism, a reflected electromagnetic wave reflected from the living organism is detected by irradiating a space where the organism is located with an electromagnetic wave including a frequency in the terahertz region from millimeter waves. By obtaining data including the direction and distance of at least a part of the organism 1, the spatial position of at least a part of the organism 1 is obtained. Electromagnetic waves including frequencies from the millimeter wave to the terahertz range are transmitted through an object, but are not reflected by a living body due to its moisture and are partially reflected.
[Selection] Figure 1

Description

The present invention relates to a method and apparatus for estimating information about a living organism using electromagnetic waves, and in particular, estimates information on the position, posture, behavior, etc. of a living organism such as a person using electromagnetic waves including frequencies in the millimeter to terahertz region. It relates to a method and an apparatus.

Conventionally, as a method for estimating the posture of a person without attaching special equipment to the person, a method using a color image and a distance image obtained from a video rate stereo machine (see Patent Document 1) or a thermal camera (infrared camera) is used. There was a method to be used (see Patent Document 2).
JP-A-9-237348 Japanese Patent Laid-Open No. 10-258044

However, in the system using the video rate stereo machine, it is necessary to first extract the skin color region from the color image and determine the three-dimensional positions of the head and both hands. Therefore, there is a problem that the posture cannot be estimated when the skin color of both hands cannot be extracted due to the influence of lighting or gloves. On the other hand, in the method using a thermal camera, it is difficult to distinguish the person from the surroundings at high temperatures, and when one part of the person, for example, the arm overlaps with another part, for example, the torso, that part There was a problem that it was difficult to detect. Another problem common to these systems is that when a person wears clothes and shoes, the color image shows clothes and shoes instead of the person itself, and with a thermal camera, clothes and shoes block heat. Therefore, there is a problem that it is difficult to accurately estimate the posture of the whole person.

In view of the above problems, the method for estimating information relating to a living body of the present invention irradiates a space where a living body such as a person is irradiated with an electromagnetic wave including a frequency from the millimeter wave to the terahertz region, and generates a reflected electromagnetic wave reflected by the living body. It is characterized by having at least a first step of obtaining data including the direction and distance of at least a part of an organism by detection and determining a spatial position of at least a part of the organism. In the case where the posture of the organism is included in the information to be estimated, the information processing method further includes a second step of estimating the posture of the organism from the relationship between the spatial positions of the parts of the organism obtained in the first step. . If the information to be estimated includes the behavior of the organism, a third step of estimating the behavior of the organism from the temporal position change of the posture of the organism obtained in the second step is further included. Have.

In addition, in view of the above problems, the information estimation apparatus for living organisms according to the present invention includes electromagnetic wave irradiation means for irradiating electromagnetic waves including frequencies in the terahertz region from millimeter waves to a space in which living organisms are reflected, and reflected electromagnetic waves reflected by the living organisms. It is characterized by having position detecting means for obtaining data including the direction and distance of at least a part of the organism by detecting the position and obtaining the spatial position of the at least part of the organism. Here again, when the posture of the organism is included in the information to be estimated, there is further provided posture estimation means for estimating the posture of the organism from the relationship of the spatial position of each part of the organism obtained by the position detection means. Have. Further, when the behavior to be estimated includes the behavior of the organism, the behavior estimation means for estimating the behavior of the organism from the temporal position change of the posture of the organism obtained by the posture estimation means is further provided.

According to the present invention, the position of a person or the like using electromagnetic waves including a frequency from the millimeter wave to the terahertz region has a characteristic that an object is transmitted but is not transmitted by a living body due to moisture or the like but partially reflected. To estimate information such as 3D posture and behavior, 3D such as a person with relatively high accuracy without being affected by “lighting”, “temperature”, “clothes and shoes worn by a person”, etc. Posture, action, etc. can be estimated.

In the following, an embodiment of a human posture / behavior estimation method or apparatus using electromagnetic waves including frequencies from the millimeter wave to the terahertz region will be described.

In the present invention, an electromagnetic wave including a frequency from the millimeter wave to the terahertz region is irradiated to a space where a living organism such as a person is present, and a reflected electromagnetic wave reflected by the living organism is detected to detect at least a part of the direction of the living organism. Obtain data including distance, obtain the spatial position of at least a part of the organism, estimate the posture of the organism from the relationship of the spatial position of each part of the obtained organism, and obtain the posture of the organism In a typical embodiment, the head vertex, the hand point, and the toe point are obtained from image data obtained from reflected electromagnetic waves reflected on the human skin surface. The position of the characteristic part of the person is obtained, the posture of the person is estimated from the positional relationship of the characteristic part of the person, and the behavior of the person is estimated from the temporal position change of the posture.

In determining the position of the characteristic part of the person, a spatial distance image of the skin surface of at least a part of the person is obtained, and the person including the head vertex, the hand point, and the toe point is obtained by signal processing the distance image. The three-dimensional position information of the representative point of the characteristic part can be obtained.

Further, in estimating the posture of the person, a vector between representative points of the characteristic part of the person is calculated from the three-dimensional position information of the representative point of the characteristic part of the person including the head vertex, the hand point, and the toe point. The posture of the person can be estimated from the vector between the representative points.

Furthermore, in estimating the person's action, the person's action can be estimated from the temporal change of the vector between the representative points of the person's characteristic part.

Further, an embodiment of the estimation device irradiates a space where a person is present with an electromagnetic wave including a frequency from a millimeter wave to a terahertz region, and detects a reflected electromagnetic wave reflected by the person, thereby obtaining information on the skin surface shape of the person. A device for detecting the position of a characteristic part of a person including a head apex, a hand point, and a foot point from image data obtained from reflected electromagnetic waves reflected on the skin surface of the person, the characteristics of the person It may take a form including means for estimating the posture of the person from the positional relationship of the parts and means for estimating the action of the person from the temporal position change of the posture.

Here, the image data is, for example, image data including three-dimensional position information from the delay time of the reflected electromagnetic wave to the human skin surface obtained. The electromagnetic wave may be a continuously generated electromagnetic wave pulse. Continuous images can be obtained by using continuously generated electromagnetic wave pulses.

The electromagnetic wave can take a form of propagating in space in the form of a beam narrowed down to about its wavelength, and detecting part or the entire information in the in-plane direction by scanning the beam position in two dimensions. An image can be formed by two-dimensionally scanning the beam position. In addition, the electromagnetic wave may be propagated through the space in the form of an expanded beam so that it can be irradiated with a spot size on a person region to be acquired, and a form in which a part or whole information in the in-plane direction is detected can be taken. An image can also be formed using an expanded beam.

Hereinafter, more specific examples will be described.
Example 1
A method and apparatus for estimating the posture and behavior of a person according to this embodiment will be described with reference to FIG.

Here, it is assumed that the space including the person 1 who is exercising is irradiated with the electromagnetic wave pulse 8 including the frequency in the terahertz region from the millimeter wave to estimate the posture and action of the person 1. The three-dimensional information sensing device 15 includes an electromagnetic wave generator 2, an antenna 10 that emits an electromagnetic wave pulse 8 into space, an antenna 11 that receives a reflected electromagnetic wave pulse 9 from a person 1, an electromagnetic wave detector 3, and a pulse from the electromagnetic wave generator 2. A delayer 12 that delays the generation timing, a mixer 14 that mixes the electromagnetic wave pulse delayed by the delayer 12 and the received pulse, and three-dimensional position information (for example, direction and distance) from this delay time to each part of the person 1 Is the distance from the object to the detection unit that detects the reflected electromagnetic wave, and the direction is the direction calculated from the inclination of the detection unit that detects the reflected electromagnetic wave. And a data storage unit 4 for storing the three-dimensional position information calculated by the calculator 13 as data. These elements can be integrated or integrated in one housing, but not all of them are necessarily housed in one. Note that the waveform of the generated pulse from the electromagnetic wave generator 2 is controlled by the calculator 13 and is initially set in consideration of the resolution described later.

The output of the mixer 14 is such that the maximum output can be obtained when the timing of the signal obtained by delaying the transmission pulse and the timing of the reception pulse coincides with each other, and the arithmetic unit 13 sweeps and controls the delay amount by the delay unit 12. By monitoring the output, the propagation delay time of the electromagnetic wave can be identified, and the distance to each part of the person 1 can be detected.

Terahertz waves have the property of passing through fibers, plastics, rubber, etc., so even if a person 1 who is trying to estimate posture or action wears thick clothes, sunglasses, shoes, etc., terahertz waves It is transmitted and reflected by the human skin surface. Assuming that the delay time obtained from the delay device 12 is τ and the speed of propagation of electromagnetic waves is the speed of light c, the linear distance d to the person 1 is
d = c ・ τ / 2
It is obtained. If a pulse with a half-value width of about 0.4 psec is used as the electromagnetic wave and the pulse is separated to half of the pulse, the distance d corresponding to the delay time τ of 0.2 psec is 30 μm, and the distance to the person 1 Can be measured with a resolution of 30 μm. The waveform of the electromagnetic wave pulse may be appropriately determined according to the required specifications.

The distance obtained as described above is linear distance information to the skin surface of a certain point of the person. In order to obtain linear distance information to the skin surface of the whole person, a terahertz wave is propagated in space in the form of a beam narrowed down to about the wavelength, and the beam position is scanned two-dimensionally. For example, a method of mechanically vibrating a parabolic mirror or the like may be used. If a plurality of sets of beam propagation directions and linear distance information at that time are obtained, a spatial image of at least a part of the skin surface of a person can be estimated. Since the spatial resolution in the two-dimensional direction is about a wavelength, an image with a resolution of 100 μm can be obtained in the case of an electromagnetic wave of about 3 THz (wavelength of about 100 μm).

Information on the straight line distance and direction to the skin surface of the entire person obtained in this way, that is, three-dimensional position information, is stored in the data storage unit 4, and from the arithmetic processing unit 16 when estimating the posture and action of the person 1. Called at any time. The arithmetic processing device 16 includes a person characteristic part detection means 5, a person posture estimation means 6, and a person behavior estimation means 7.

The characteristic part detection means 5 of the person 1 is a means for executing a step of obtaining a distance image from the three-dimensional position information stored in the data storage unit 4 to at least a part of the skin surface of the person 1, and the signal processing of the distance image Means for executing the step of obtaining the three-dimensional position information of the representative point of the characteristic part of the person including the head vertex, the hand tip point, the toe point, and the like. These execution means are configured as software in a computer, for example, as a program. In the step of obtaining the three-dimensional position information of the representative points of the characteristic part of the person including the head vertex, the hand point, the toe point, etc. by performing the signal processing on the distance image, for example, the feature point extraction and the feature point by the edge detection filter are performed. The region of such a characteristic part is specified by pattern matching or the like, the center of gravity of the specified region is selected as a representative point, and three-dimensional position information corresponding to the representative point is calculated.

The posture estimation means 6 of the person 1 calculates a vector between the representative points of the person's characteristic part from the three-dimensional position information of the representative points of the person's characteristic part including the head vertex, the hand point, the toe point, and the like. Means for executing, and means for executing a step of estimating the posture of the person from the vectors between the representative points. For example, as shown in FIG. 2, as vectors between representative points, vectors 22 and 23 from the head vertex 17 to the representative points 18 and 19 of both hands, and from the head vertex 17 to the representative points 20 and 21 of both feet, respectively. By calculating the vectors 24 and 25, the posture of the person can be estimated. In addition, as vectors between representative points, vectors between representative points 18 and 19 of both hands, vectors between representative points 20 and 21 of both feet, vectors between representative points of right hand 18 and right foot 20, and left hand 19 and A vector between the representative points of the left foot 21 may be calculated for estimating the posture of the person 1, and the direction of the vector may be freely set so as to be convenient for estimating the posture of the person.

The action estimation means 7 of the person 1 includes a means for executing a step of estimating the action of the person from the temporal change of the vector between the representative points of the characteristic part of the person. For example, as shown in FIG. 3, assuming that at a certain moment, the three-dimensional position of the representative point of the right hand is 18 and after 26 seconds, the position of the representative point of the right hand for t seconds. (Ie, change from vector 22 to vector 27) is represented by vector 28. Since this vector 28 is a three-dimensional vector, it is possible to know the three-dimensional movement of the person 1, for example, how much the right hand has moved in the upper right front direction.

By using electromagnetic waves including frequencies from the millimeter wave to the terahertz region, it is possible to estimate the position, posture, behavior, etc. of at least a part of a person and the whole body even when wearing thick clothes, sunglasses, shoes, etc. be able to. Further, the movement of the person in the three-dimensional direction can also be known from the temporal change of the vector between the representative points of the characteristic part of the person.

(Example 2)
A second embodiment according to the present invention will be described with reference to FIG. In this embodiment, instead of detecting a part or the whole information in the in-plane direction by two-dimensionally scanning the beam position as in the first embodiment, the electromagnetic wave is irradiated to the person area to be acquired with a spot size. It propagates through the space in the form of an expanded beam as possible, and detects part or all of the information in the in-plane direction.

The three-dimensional information sensing device 15 includes an electromagnetic wave generator 2, a beam expander 29 that converts the millimeter wave generated from the electromagnetic wave generator 2 into an electromagnetic wave beam 32 by expanding the beam diameter of the electromagnetic wave including the frequency in the terahertz region, and the person 1. Are delayed by a lens 30 for collecting the reflected electromagnetic wave beam 33, an electromagnetic wave detection array 31 for detecting the collected electromagnetic wave, a delay device 12 for delaying the timing of pulse generation from the electromagnetic wave generator 2, and the delay device 12. The mixer 14 that mixes the electromagnetic waves and the respective pixels of the electromagnetic wave detection array 31 in parallel, the calculator 13 that calculates the three-dimensional position information from the delay time, and the three-dimensional position information calculated by the calculator 13 are stored as data. The data storage unit 4 is provided. Since each pixel of the electromagnetic wave detection array 31 corresponds to each direction of the space, the calculator 13 obtains a plurality of sets of directions and straight line distance information to each skin surface of the person, and at least a part of the skin surface of the person Can be estimated.

Again, the waveform of the generated pulse from the electromagnetic wave generator 2 is controlled by the calculator 13. In addition, these can be integrated or integrated in one housing, but not all of them are necessarily housed in one. The three-dimensional position information stored in the data storage unit 4 is called up by the arithmetic processing device 16 at any time and is used for estimating the posture, behavior, etc. of the person 1. Functions such as 7 are the same as those in the first embodiment.

The present invention can be used in various fields. For example, they are used in fields such as virtual reality systems, games, model creation in computer graphics, and gesture input interfaces that operate devices using body movements as input information.

The figure explaining the apparatus which estimates the attitude | position, action, etc. of a person by 1st Example of this invention. The figure which expressed the posture of a person with a vector. The figure which expressed the action of the person with the vector. The figure explaining the apparatus which estimates the attitude | position, action, etc. of a person by 2nd Example of this invention.

Explanation of symbols

1 ... person (organism)
2, 10, 29 ... electromagnetic wave irradiation means 3-5, 11-14, 30, 31 ... position detection means 6 ... posture estimation means 7 ... behavior estimation means 8, 32 ... electromagnetic wave 9 33 ... Reflected electromagnetic wave

Claims (9)

By irradiating the space where the organism is located with electromagnetic waves including frequencies from the millimeter wave to the terahertz region, the reflected electromagnetic waves reflected by the organism are detected to obtain data including the direction and distance of at least a part of the organism, A method for estimating information relating to an organism, comprising at least a first step of obtaining a spatial position of at least a part of the organism. The method for estimating information about an organism according to claim 1, further comprising a second step of estimating the posture of the organism from the relationship between the spatial positions of the parts of the organism obtained in the first step. The method for estimating information related to an organism according to claim 2, further comprising a third step of estimating the behavior of the organism from the temporal position change of the posture of the organism obtained in the second step. The first step includes a step of obtaining a spatial position of a characteristic part of a person including a head vertex, a hand point, and a foot point from image data obtained from a reflected electromagnetic wave reflected from the skin surface of the person. The estimation method of the information regarding the organism in any one of. Electromagnetic wave irradiation means for irradiating electromagnetic waves including frequencies in the terahertz region from millimeter waves to the space in which the organism is located, and data including the direction and distance of at least a part of the organism by detecting the reflected electromagnetic wave reflected by the organism An apparatus for estimating information relating to an organism, characterized by comprising position detection means for obtaining a spatial position of at least a part of the organism. 6. The apparatus for estimating information related to a living organism according to claim 5, further comprising posture estimating means for estimating the posture of the living organism based on a spatial position relationship of each part of the living organism obtained by the position detecting means. The apparatus for estimating information about an organism according to claim 6, further comprising behavior estimation means for estimating the behavior of the organism from the temporal position change of the posture of the organism obtained by the posture estimation means. The biological information according to any one of claims 5 to 7, wherein the irradiating means is configured to propagate electromagnetic waves in space in the form of a beam narrowed down to about its wavelength and to perform two-dimensional scanning of the beam position. Estimating device. The information on the living organism according to any one of claims 5 to 7, wherein the irradiation means is configured to propagate the electromagnetic wave to the space in an expanded beam shape so that the organism region to be acquired can be irradiated with a spot size. Estimating device.

Images