WO2023286677A1 - Information processing device, information processing method, and program - Google Patents

Abstract

This information processing device comprises: an inference unit that infers a plurality of items related to a physical state of a person in time series on the basis of information obtained by detecting the person by radar; a detection unit that detects a change in the physical state of the person on the basis of inference results obtained in time series for the plurality of items; and an output unit that outputs information pertaining to the detected change in the physical state.

Description

Information processing device, information processing method, and program

The present disclosure relates to an information processing device, an information processing method, and a program.

There is a technology that estimates a person's walking speed based on the person's image taken by a surveillance camera and evaluates the person's cognitive function based on changes in the estimated walking speed. Technology that monitors (watches over) a person's behavior in a dwelling based on the combination and order of electrical devices operated by the person within the dwelling, or based on the location information of the wireless tag attached to the person's clothing. There is

WO2021/014938 Japanese Patent Application Laid-Open No. 2020-53070 WO2020/262526 JP 2017-200572 A

There is room for further examination of the technology for detecting changes in a person's physical condition.

A non-limiting embodiment of the present disclosure contributes to providing an information processing device, an information processing method, and a program capable of contactlessly detecting changes in a plurality of items related to a person's physical condition.

An information processing apparatus according to an embodiment of the present disclosure includes an estimating unit for estimating a plurality of items related to a person's physical condition in time series based on information obtained by detecting a person by radar; and an output unit for outputting information on the detected change in the physical state of the person based on the estimation result obtained in (1).

In addition, these generic or specific aspects may be realized by systems, devices, methods, integrated circuits, computer programs, or recording media. may be realized by any combination of

A non-limiting embodiment of the present disclosure can, for example, detect changes in a plurality of items related to a person's physical condition without contact.

Further advantages and effects of one embodiment of the present disclosure will be made clear from the specification and drawings. Such advantages and/or advantages may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all provided to obtain one or more of the same features. no.

1 is a diagram showing a configuration example of a body condition detection system according to an embodiment; FIG. 1 is a block diagram showing a configuration example of a body condition detection system according to an embodiment; FIG. 3 is a flow chart showing an example of the overall operation of the physical condition detection system according to one embodiment; Flowchart showing an example of the sleep time zone detection process illustrated in FIG. A diagram showing a setting example of a sleep place definition area according to one embodiment A diagram showing an example of detection of sleep hours according to an embodiment. A diagram showing an example of detection of a non-REM sleep time zone according to an embodiment. A diagram showing an example of detection of an apnea sleep time period according to an embodiment. FIG. 11 is a diagram showing an example of detection of midway awakening according to an embodiment; A diagram showing an example of detection of a person sitting upright according to an embodiment. FIG. 4 is a diagram showing an example of detection of walking speed according to one embodiment; FIG. 4 is a diagram showing an example of filtering processing of detected positions according to an embodiment; FIG. 4 is a diagram showing an example of state change detection processing according to an embodiment; A diagram showing an example of a screen display (graphical user interface; GUI) according to an embodiment. Diagram showing an example of computer hardware configuration

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and drawings, constituent elements having substantially the same functions are denoted by the same reference numerals, thereby omitting redundant description.

<One embodiment>
Early detection of changes in a person's physical condition and taking countermeasures are also important for maintaining good health. For example, detection of changes in sleep or walking speed leads to an understanding of physical conditions such as concentration, motivation, appetite, and exercise capacity, and can be applied to prevent depression or injuries, and even detect dementia. be.

Examples of changes in a person's physical condition include changes in sleep and changes in walking speed (for example, decrease). Examples of changes in sleep state include changes in the amount of time spent in non-REM sleep (deep sleep), changes in the amount of time spent in apnea sleep, and changes in the frequency of midway awakenings.

In order to detect changes in physical condition as described above, for example, the methods shown in (1) to (3) below are considered.

(1) Detection of REM or non-REM sleep states, or apnea sleep states, using pulse or respiration detection by wearable devices (2) REM, using body motion, respiration, or pulse detection by sensor mats Detection of the state of sleep or non-REM sleep, or awakening in the middle (3) Detection of walking speed using images (hereinafter sometimes referred to as "camera images") of the person to be detected taken by a camera

In the following description, the terms "detection" and "detection" may be interchanged with terms such as "estimation" and "measurement", for example.

Here, in method (1), since the device is attached to the person to be detected, for example, the person to be detected may feel uncomfortable or uncomfortable. In method (2), items other than the sleep state (for example, walking speed) are not measured.

Since method (3) uses the camera image of the person to be detected, privacy violations, for example, can become a problem. In method (3), when estimating the position change of the detection target based on the change in the size of the detection target in the camera image, the size in the camera image changes depending on the movement route of the detection target. may not. Therefore, it may be difficult to estimate walking speed.

In one embodiment of the present disclosure, for example, temporal changes (in other words, temporal changes) of a plurality of items related to the physical condition of a person to be detected are detected by a single radio wave sensor (for example, a millimeter wave radar). Detect and present to touch. Non-limiting examples of multiple items to be detected are "change in sleep state" and "decrease in walking speed".

Millimeter-wave radar, for example, can detect the position of a person three-dimensionally without contact, so the distance traveled by the person can be calculated without depending on the movement path of the person, and the walking speed of the person can be estimated. .

In addition, millimeter-wave radar can detect, for example, the body movement of the detected person (for example, including changes in the body surface associated with heartbeat or breathing), so based on the body movement during sleep , REM sleep) can be estimated. In addition, an apnea sleep state can be estimated based on the presence or absence of breathing during sleep.

<System configuration example>
FIG. 1 is a diagram showing a configuration example of a physical condition detection system 1 according to one embodiment. As shown in FIG. 1, the physical condition detection system 1 may include, for example, a millimeter wave radar 10, a personal computer (PC) 20, servers 30A and 30B, and databases (DB) 40A and 40B.

These components 10, 20, 30A, 30B, 40A and 40B may be communicatively connected to each other via a network 50 such as the Internet or a local area network, for example. The connection form of the components 10, 20, 30A, 30B, 40A and 40B may be wired or wireless, or may be a mixture of wired and wireless. Each of the PC 20, the server 30A and the server 30B is an example of an information processing device.

The millimeter wave radar 10 is an example of a radio wave sensor, and is installed, for example, in a living room in a building 60 to detect a person P located in the living room as a detection target by radio waves (radar waves) without contact. Non-limiting examples of buildings 60 include homes, shops, dormitories, apartment complexes, offices, inns, restaurants, school buildings, laboratories, hospitals, clinics, and the like.

"Detection" may be read as, for example, "detection" or "sensing". As a non-limiting example, the frequency of the millimeter wave radar 10 may range from several tens of gigahertz (GHz) to several hundreds of GHz, such as a frequency in the 60 GHz band.

For example, the PC 20 acquires (or receives) detection results from the millimeter wave radar 10 (hereinafter may be referred to as “radar detection results”, “radar detection information”, or “radar data”) from the millimeter wave radar 10. .

Radar detection results may be transmitted from the PC 20 via the network 50 to one or both of the servers 30A and 30B, for example. If the millimeter wave radar 10 has a connection (or access) function (in other words, a communication function) to the network 50, the radar detection result is transmitted to one or both of the servers 30A and 30B without going through the PC 20. may

Also, the PC 20 may display information received from one or both of the servers 30A and 30B via the network 50 on the display of the PC 20, for example. For example, information processed or generated based on radar detection results in one or both of the servers 30A and 30B (for example, information regarding changes in the physical condition of the person P) may be displayed on the PC 20 . Information received via the network 50 may be printed on a printer connected to the PC 20 .

It should be noted that a mobile terminal such as a smart phone may perform functions equivalent to at least part of the functions of the PC 20 described above as a substitute for or in addition to the PC 20 .

One of the servers 30A and 30B (for example, the server 30A) detects (or estimates or measures) information such as the position of the person P in the living room, body movement, breathing rate (or heart rate) based on the radar detection result. )do. The detected information may be sequentially (in other words, chronologically) stored (or accumulated) in the DB 40A, for example.

Further, the server 30A, for example, reads information for a predetermined period (for example, one day's worth) from the DB 40A, and determines the sleep state of the person P (for example, non-REM sleep, apnea sleep, and at least one time period of awakening). , and information such as walking speed is detected (or estimated or measured). The detected information may be stored (or accumulated) in the DB 40B in chronological order (for example, daily).

The server 30B, for example, determines (or detects) whether or not there is a change in the physical condition of the person P based on time-series information (hereinafter sometimes referred to as "time-series data") accumulated in the DB 40B. do. When determining that there is a change in the physical condition of the person P, the server 30B sends information indicating that the physical condition of the person P has changed (hereinafter sometimes referred to as "physical condition change information") to the network. 50 to provide (or notify) the PC 20 . The PC 20 displays the received physical condition change information on a display of the PC 20 or prints it out by a printer. The physical condition change information may be displayed or printed not only by the PC 20 but also by the server 40A or 40B.

The detection (or estimation or measurement) of various information such as the position of the person P, body movement, breathing rate (or heart rate), sleeping state, and walking speed is performed by one or both of the servers 30A and 30B using radar detection. It may be implemented centrally or distributedly based on the results. In other words, it is not necessary that specific information among various types of information (which may be referred to as “measurement items”) is detected in a specific server.

For example, the servers 30A and 30B may be integrated into one server, for example. Also, three or more servers may dispersively perform the processing related to detection of the measurement items described above.

The DBs 40A and 40B may be provided inside the servers 30A and 30B, respectively. In other words, the storage units or storage devices provided in the servers 30A and 30B may correspond to the DBs 40A and 40B. The DBs 40A and 40B may be integrated into one DB, for example. In three or more DBs, the measurement items described above may be accumulated and managed in a distributed manner.

FIG. 2 is a block diagram showing a configuration example of the physical condition detection system 1 according to one embodiment. In FIG. 2, the millimeter wave radar 10 may include, for example, a detection section 101, a processing section 102 and an output section 103.

The detection unit 101, for example, transmits a radar wave and receives and detects a reflected wave of the radar wave reflected by a detection target (eg, person P).

The processing unit 102 detects information such as the position, speed, and angle with respect to the millimeter wave radar 10 of the detection target, based on point cloud data corresponding to the detection target, for example, obtained by reflected waves.

The output unit 103 transmits information (in other words, radar data) obtained by the processing unit 102 to the radar control device 70, for example.

The radar control device 70 may include a radar communication section 701 and a radar data storage section 702, for example. Note that the radar control device 70 may correspond to, for example, the PC 20 illustrated in FIG. 1, or may correspond to one or both of the servers 30A and 30B illustrated in FIG.

The radar communication unit 701 , for example, communicates with the millimeter wave radar 10 by wire or wirelessly, receives radar data, and stores (memorizes) it in the radar data storage unit 702 . The radar communication unit 701 also transmits radar data to the state change detection device 80 by wire or wirelessly, for example.

The state change detection device 80 detects a change in the physical state of the detection target (for example, person P) based on, for example, radar data. The state change detection device 80 is an example of an information processing device, and may correspond to one or both of the servers 30A and 30B illustrated in FIG. 1, for example.

The state change detection device 80 includes, for example, a radar data reading unit 801, a sleep state estimation unit 802, a walking speed estimation unit 803, an estimation result storage unit 804, an estimation result reading unit 805, a state change detection processing unit 806, a communication unit 807, A radar data storage unit 808 and an estimation result information storage unit 809 may be provided.

For example, the radar data reading unit 801 reads radar data stored in the radar data storage unit 808 and outputs it to the sleep state estimation unit 802 . Note that the radar data storage unit 808 may correspond to, for example, the DB 40A illustrated in FIG.

The sleep state estimation unit 802 estimates the sleep state of the person P who is the detection target (for example, non-REM sleep, apnea sleep, and a period of midway awakening), for example, based on radar data.

The walking speed estimation unit 803 estimates the walking speed of the person P, for example, based on radar data. For example, since the walking speed estimation need not be performed during the sleep hours estimated by the sleep state estimation unit 802, it may be performed for a time slot other than the estimated sleep hours of the person P.

The sleep state estimation unit 802 and walking speed estimation unit 803 may be integrated into one estimation unit, for example.

The estimation result storage unit 804 stores (memorizes) each estimation result by the sleep state estimation unit 802 and the walking speed estimation unit 803 in the estimation result information storage unit 809, for example. Note that the estimation result information storage unit 809 may be integrated with the radar data storage unit 808, for example. Also, the estimation result information storage unit 809 may correspond to the DB 40B illustrated in FIG.

For example, the estimation result reading unit 805 reads the estimation result stored in the estimation result information storage unit 809 and outputs it to the state change detection processing unit 806 .

The estimation result storage unit 804 and the estimation result reading unit 805 may be integrated into one writing and reading control unit, for example.

The state change detection processing unit 806 detects changes in the physical state of the person P based on the estimation result input from the estimation result reading unit 805, for example. A detection result may be output to the communication unit 807, for example.

The communication unit 807 is an example of an output unit, and for example, communicates with the display terminal device 90 (for example, the communication unit 901) wirelessly or by wire, and transmits detection results regarding changes in the physical state of the person P to the display terminal device 90. do.

The display terminal device 90 may comprise a communication section 901 and a display section 902, for example. The display terminal device 90 may correspond to the PC 20 illustrated in FIG. 1, for example.

The communication unit 901 , for example, communicates with the state change detection device 80 (for example, the communication unit 807 ) wirelessly or by wire, receives detection results regarding changes in the physical state of the person P, and outputs the received information to the display unit 902 . do.

The display unit 902 displays information input from the communication unit 901, for example.

The configuration example of the physical condition detection system 1 according to one embodiment has been described above.

<Operation example>
Next, an operation example of the physical condition detection system 1 having the configuration described above will be described with reference to the flow charts of FIGS. 3 and 4. FIG.

FIG. 3 is a flowchart showing an example of overall operation of the physical condition detection system 1 according to one embodiment. FIG. 4 is a flowchart showing an example of the sleep period detection process (S803) illustrated in FIG.

As shown in FIG. 3, radar data is detected in the millimeter wave radar 10 (S101), and the detected radar data is stored (stored) in the radar data storage unit 808 of the state change detection device 80, for example (S102 ).

The state change detection device 80 periodically reads radar data from the radar data storage unit 808 by the radar data reading unit 801, for example. For example, the radar data reading unit 801 reads one day's worth of radar data from the radar data storage unit 808 at 1:00 every day (S801, S802).

The radar data read from the radar data storage unit 808 is output to, for example, the sleep state estimation unit 802, and the sleep state estimation unit 802 estimates the sleep state of the person P to be detected based on the radar data. (S803).

For example, the sleep state estimation unit 802 determines whether or not there is a sleep time period based on radar data for one day (S804). An example of the sleep time period detection process will be described later with reference to FIG. As a result of the determination, if there is no sleep time period (S804; NO), for example, estimation by the sleep state estimation unit 802 may be terminated and estimation by the walking speed estimation unit 803 may be executed (S808).

If there is a sleep time zone (S804; YES), the sleep state estimation unit 802 estimates, for example, non-REM sleep, apnea sleep, and midway awakening time zones (S805-S807).

Next, for example, the walking speed estimation unit 803 estimates the walking speed of the person P based on the radar data (S808).

The estimation result obtained by each process of S805 to S808 is stored (stored) in the estimation result information storage unit 809 by the estimation result storage unit 804, for example (S809).

The processing of S805 to S808 may be performed serially as described above, or may be performed in parallel. When implemented serially, the processing order of S805 to S807 may be changed as appropriate.

The estimation result stored in the estimation result information storage unit 809 is read by, for example, the estimation result reading unit 805 and output to the state change detection processing unit 806 (S810).

The state change detection processing unit 806 detects a change in the physical state of the person P, for example, based on the estimation result input from the estimation result reading unit 805 (S811). For example, the state change detection processing unit 806 determines whether or not there is a change in the physical state of the person P based on the sleep state and walking speed estimation results (S812).

As a result of the determination, if there is no change in the physical state of the person P (S812; NO), the state change detection processing unit 806 may end the processing. On the other hand, if there is a change in the physical state of the person P (S812; YES), the state change detection processing unit 806 sends information about the change in the physical state of the person P to the display terminal device 90 via the communication unit 807, for example. Send (S813).

When the display terminal device 90 receives information about a change in the physical state of the person P from the state change detection device 80 (S901), the received information is displayed on the screen of the display unit 902, for example (S902).

<Estimation of sleep state>
Next, the estimation (or detection) of the sleep state of the person P (for example, sleep time period, non-REM sleep, apnea sleep, and each time period of awake) in the sleep state estimation unit 802 will be described item by item. .

<Detection of sleeping hours (S803 in FIG. 3)>
As exemplified in FIG. 4, the sleep state estimation unit 802 reads, for example, the setting of the sleeping place definition area 501 (see FIG. 5) of the person P (S831). The setting of the sleeping place definition area 501 may be performed by, for example, the sleep state estimation unit 802, or may be performed by a setting unit (not shown) in the state change detection device 80. FIG.

The setting data of the sleeping place definition area 501 may be stored in either one of the storage units 808 and 809, or may be stored in another storage unit (not shown) in the state change detection device 80, for example.

The sleeping place definition area 501 may be set as a three-dimensional (3D) space with respect to the place where the person P sleeps (for example, bedding such as a bed or a mattress) in the living room, as shown in the upper part of FIG.

In the example in the upper part of FIG. 5, a sleeping place definition area 501 defined by XYZ coordinates is set in the space above the person P's sleeping place. The size (area) of the XY plane of the sleeping place definition area 501 may be set according to the size of bedding such as a bed, mattress, and futon, for example.

For example, the size of the XY plane of the sleeping place definition area 501 may be set to a size that matches the size of the bedding, or may be set to a size that is slightly smaller or larger than the size of the bedding. Alternatively, the sleeping place definition area 501 may be set to a space size corresponding to the three-dimensional space occupied by at least one of the supine position, prone position, and side lying position in the person P's sleeping position.

For example, the length (in other words, height) of the sleeping place definition area 501 in the Z-axis direction corresponds to the height (in other words, body height) in the sleeping posture (or body position) of the person P. may be set to For example, it may be set according to the highest body height of the person P in each of the supine position, the prone position, and the side-lying position.

According to such a setting, for example, when the person P raises his/her upper body while staying in the sleep place definition area 501 , at least part of the person P's upper body is out of the sleep place definition area 501 . Therefore, the millimeter wave radar 10 can detect the person P sitting upright.

The height of the sleeping place definition area 501 may be set based on the top surface of bedding such as a bed, a mattress, and a mattress as illustrated in FIG. 5, or may be set based on the floor surface on which the bedding is placed. may In other words, the sleeping place definition area 501 may be set, for example, as a space covered with bedding or as a space not covered with bedding.

In the lower part of FIG. 5, for example, as illustrated in the upper part of FIG. 5, the positions of the person P (XYZ An example is shown in which the coordinate position) is detected as radar data.

Depending on the setting of the sleep place definition area 501, for example, whether or not the person P entered the sleep place definition area 501 to go to bed, or whether the person P woke up (including halfway awakening) and entered the sleep place definition area 501 It can be detected based on the radar data (for example, the positional information of the person P) whether or not the person P has left the outside.

Therefore, for example, by setting the sleep start determination condition and the sleep end determination condition based on the detection of entering and exiting the sleeping place definition area 501 of the person P as follows, the sleeping time zone of the person P is detected ( (or presumed).

(1) Sleep start determination condition: Detection of entry into sleep place definition area 501 (2) Sleep end determination condition: Detection of exit from sleep place definition area 501 and entry into sleep place definition area 501 within N minutes thereafter undetected

For example, N may be set to a time during which it can be determined that the person P will not return to his/her sleeping place after waking up, for example, several tens of minutes, such as 30 minutes. In other words, if entry into the sleeping place definition area 501 is (re)detected within N minutes or less, the sleeping end determination condition is not satisfied, and the sleep time of the person P continues without ending. It may be determined that It should be noted that the period of time of N minutes or less may be determined to be the period of time during which the person P has awakened in the middle.

Returning to FIG. 4, the sleep time period detection processing based on the above conditions (1) and (2) will be described below.

After reading the setting of the sleep place definition area 501 as described above, the sleep state estimation unit 802 acquires, for example, the data of the start time of the radar data (S832), and the acquired data satisfies the sleep start determination condition (1). (S833).

As a result of the determination, when it is determined that the data at the beginning time does not satisfy the sleep start determination condition (1) (S833; NO), the sleep state estimation unit 802, for example, as long as the data at the next time exists (S834; YES ), the data for the next time is acquired (S835), and it is determined whether or not the sleep start determination condition (1) is satisfied.

If there is data that satisfies the sleep start determination condition (1) (S833; YES), the sleep state estimation unit 802 determines whether the next time data satisfies the sleep end determination condition (2) (S836). .

As a result of the determination, when it is determined that the data for the next time does not satisfy the sleep end determination condition (2) (S836; NO), the sleep state estimation unit 802, for example, as long as the data for the next time exists (S837; YES ), the data for the next time is acquired (S838), and it is determined whether or not the sleep end determination condition (2) is satisfied.

As a result of the determination, if it is determined that the data for the next time satisfies the sleep end determination condition (2) (S836; YES), the sleep state estimation unit 802 determines, for example, the time of the data that satisfies the sleep start determination condition (1). and the time of the data that satisfies the sleep end determination condition (2) is recorded as the sleep time period of the person P (S839).

After that, the sleep state estimating unit 802 checks whether there is data for the next time (S840), and if there is data for the next time (S840; YES), for example, acquires data for the next time (S841), It is determined whether or not the next time data satisfies the sleep start determination condition (1) (S833).

In this way, the sleep state estimating unit 802 obtains data that satisfies the sleep start determination condition (1), then data that satisfies the sleep end determination condition (2), and data that satisfies the sleep end determination condition (2). , and detect and record the period between the times of both data as a sleeping time zone.

In such a data search, for example, if there is no data for the next time (NO in S834, S837, and S840), the sleep state estimation unit 802 may end the sleep time slot detection process. For example, the processes after S804 illustrated in FIG. 3 are executed in response to the end of the sleep time slot detection process.

Fig. 6 shows an example of detection of sleeping hours. FIG. 6 exemplarily shows a time zone (7 hours) from 00:00:00 to 07:00:00 and 22:00:00 to 24:00 in a day (24 hours from 00:00 to 24:00). Person P's sleeping time period is detected in the time period of :00:00 (4 hours), and the person P wakes up in the middle of the day in the time period of 01:10:20 to 01:20:20 (20 minutes<N). An example presumed to be is shown.

The estimation of the time zone of non-REM sleep, apnea sleep, and mid-wake may be performed for the sleep time zone detected as described above. In the following, estimation of non-REM sleep, apnea sleep, and nocturnal awakening will be described.

<Estimation of Non-REM Sleep Time Zone (S805 in FIG. 3)>
Respiratory rate and pulse rate are examples of indicators that can determine (or estimate) whether a person is in non-REM sleep. For example, during non-REM sleep, respiration rate and pulse rate tend to be stable and regular (in other words, periodic). In contrast, during non-REM sleep, the heart rate and respiratory rate tend to increase and become irregular compared to non-REM sleep.

Therefore, for example, based on the radar data, the sleep state estimation unit 802 measures the respiratory state (for example, breathing rate and pulse rate) during the sleep time zone in which the person P is detected, and the periodic waveform continues. The observed time period may be presumed to be non-REM sleep. For example, as shown in FIG. 7, the time slot from 01:10:25 to 01:50:14 may be estimated as the non-REM sleep time slot. The vertical axis in FIG. 7 represents the phase change of the radio signal due to the movement of the chest, and the unit is [rad], for example. The vertical axis in FIG. 8 is also the same as in FIG.

<Estimation of apnea sleep time period (S806 in FIG. 3)>
For example, based on radar data, the sleep state estimation unit 802 measures the respiratory state (for example, breathing rate and pulse rate) during the sleep time period during which the person P is detected, and ignores the time period during which breathing detection is interrupted. It may be presumed to be a respiratory sleep state. For example, as shown in FIG. 8, the time period from 03:20:27 to 03:20:36 may be estimated as the apnea sleep time period.

<Estimation of Mid-Awakening Time Zone (S807 in FIG. 3)>
For example, based on radar data, the sleep state estimation unit 802 detects an act of the person P temporarily leaving the sleeping place (sleep place definition area 501) during sleep, or an act of the person P raising the upper body. In this case, the duration may be estimated as the time when the awakening occurs.

For example, as shown in the upper part of FIG. 9, after the person P leaves the sleeping place (sleep place definition area 501) at a certain time (for example, 01:10:20) in the sleeping time zone, 01:10 minutes later. Assume that you return to your bed at 20:20.

In this case, the movement (in other words, change in position) of the person P is represented in the radar data as shown in the lower part of FIG. , the entry of the person P into the sleeping place is detected at time 01:20:20. Therefore, the sleep state estimation unit 802 may estimate, for example, the 10-minute period from 01:10:20 to 01:20:20 as the period of awakening.

As shown in the upper part of FIG. 10, when the person P stands up while staying in the sleeping place, at least part of the upper body is out of the sleeping place definition area 501 as described above. As shown in the lower part of FIG. 2, it is possible to detect the sitting up of the person P based on the radar data. Therefore, the sleep state estimating unit 802 may include, for example, the time period in which the person P's sitting up is detected in the time period of the midway awakening.

<Estimation of Walking Speed (S808 in FIG. 3)>
Next, estimation of the moving speed (for example, walking speed) of the person P by the walking speed estimator 803 (see FIG. 2) will be described. For example, the walking speed estimation unit 803 calculates the distance (D) that the person P has moved in the living room based on the radar data, and divides the distance by the time (T) required for the movement. Estimate the walking speed (V) of

For example, as shown in the upper part of FIG. 11, when the person P moves from time t1 to time t4, as shown in the lower part of FIG. A moving distance D of the person P is calculated by D=d1+d2+d3. Therefore, the walking speed V of the person P is estimated by V=(d1+d2+d3)/(t4-t1) where T=t4-t1 required for movement.

Note that the estimation of the walking speed V is performed in a time zone excluding the sleep time zone detected in the above-described sleep time zone detection process (S803 in FIG. 3) time zone).

In this way, by narrowing down (or limiting) the time zone for estimating the walking speed V to the non-sleep time zone, the amount of radar data used for estimating the walking speed can be reduced. can be reduced.

<Reducing the impact of radar detection position error on movement distance>
If the frequency of radar data acquisition by the millimeter wave radar 10 is, for example, several tens of milliseconds to several hundred milliseconds, or if there is an error in the detected position of several centimeters (cm) to several tens of cm, the actual A trajectory fluctuation or shift that does not match the movement of the person P (eg, the upper part of FIG. 12) may occur in the radar radar (eg, the middle part of FIG. 12).

Therefore, if the movement distance D is calculated by simply accumulating the detected positions based on the radar data, an error outside the allowable range may occur with respect to the actual movement distance of the person P. Therefore, the walking speed estimator 803 may perform correction (for example, moving average filtering) on the detected position based on the radar data, as shown in the lower part of FIG. 12, for example. As a result, the change in the detected position of the person P over time (in other words, the trajectory) can be made to match the actual movement trajectory of the person P, or can be brought close to the minimum error, thereby improving the accuracy of estimating the walking speed V. can.

<State change detection (S811 in FIG. 3)>
Next, an example of state change detection processing (S811 in FIG. 3) by the state change detection processing unit 806 (see FIG. 2) will be described.

The state change detection processing unit 806, for example, the sleep state estimated by the sleep state estimation unit 802 (for example, each time period of non-REM sleep, apnea sleep, and midway awakening), and estimated by the walking speed estimation unit 803 Based on the time-series data of the walking speed thus obtained, a change point of a tendency change (in other words, a trend) of the physical condition of the person P is detected.

For example, the state change detection processing unit 806 determines whether there is a change point in the trend fluctuation of the time-series data for each sleeping state and walking speed. When a change point is detected, the state change detection processing unit 806 creates, for example, a prediction model for time-series data after the change point based on time-series data before the change point. For example, an autoregressive (AR) model may be used to create the prediction model.

For example, as shown in (a) of FIG. 13, in the time-series data on non-REM sleep, if a change in trend fluctuation is detected on January 5, as shown in (e) of FIG. Based on the previous time-series data (for example, time-series data for several days such as 3 days), a forecast model for January 5th and later is created.

In addition, the prediction model, for example, for each time period of apnea sleep, each time period of arousal, and each time series data of walking speed illustrated in (b) to (d) of FIG. may be created separately if is detected.

Then, for example, the state change detection processing unit 806 determines whether or not the degree of abnormality is equal to or higher than the threshold using the difference between the measured value and the predicted value by the prediction model as an indicator of the degree of abnormality. The timing determined to be present is detected as a change point of trend fluctuation. For example, (e) of FIG. 13 shows an example in which a change in trend fluctuation (in other words, anomaly) was detected on January 5 with respect to the non-REM sleep period.

The state change detection processing unit 806, for example, separates measurement items such as non-REM sleep time, apnea sleep time, midway awakening time, and walking speed from each measurement item into which a change point of trend variation is detected, and outputs information to the person. This information is transmitted to the display terminal device 90 as information on changes in P's physical condition.

The display terminal device 90 displays the information received from the state change detection device 80 on the screen of the display unit 902 of the display terminal device 90, for example. As a result, for example, the person P himself or another person involved in the health management of the person P (for example, the person P's family, caregiver, administrator, doctor in charge, etc.) can be informed about the change in the physical condition of the person P. Information is presented.

FIG. 14 shows an example of the screen display (GUI) of the display unit 902. FIG. 14 shows an example in which the display screen of display unit 902 is divided into status display area 921 and notification area 922 . As illustrated in FIG. 14, the state display area 921 displays the time-series data illustrated in (a) to (d) of FIG. For example, it may be indicated that a decrease in NREM sleep time has been detected and an increase in apnea sleep time has been detected.

The screen display example of the display unit 902 is not limited to the example shown in FIG. For example, one of the status display area 921 and the notification area 922 may be selectively displayed. In the state display area 921, any one or more of the time-series data illustrated in (a) to (d) of FIG. 13 may be selectively displayed. For example, in the state display area 921, time-series data that contributed to the detection of changes in physical state may be selectively displayed. Alternatively or additionally, the time-series data that contributed to the detection of the physical condition change may be highlighted in the condition display area 921 . Any mode of highlighting is acceptable. Also, the display mode in the notification area 922 is not limited to the example in FIG. 14 . Along with the display in the notification area 922, a sound such as an alarm sound may be used to alert the user that a change in physical condition has been detected.

As described above, according to the above-described embodiments, changes in the physical state of the person P, such as non-REM sleep time, apnea sleep time, mid-wake time, and walking speed, are detected by the millimeter wave radar 10 without contact. can be presented as

Therefore, for example, physical conditions such as concentration, motivation, appetite, and athletic ability of the person P can be quickly grasped in accordance with changes in the sleep state and walking speed of the person P, so measures for maintaining the health of the person P can be made. can be taken early. For example, it can be applied to prevent depression or injury of person P, and to detect dementia.

Further, according to the above-described embodiment, based on time-series data with a uniform time axis such as a day (in other words, timing-synchronized), non-REM sleep time, apnea sleep time, mid-awakening time, In addition, since a plurality of measurement items related to the physical condition of the person P, such as walking speed, are detected or estimated, the timing at which the physical condition of the person P changes can be easily grasped for each measurement item.

Therefore, for example, based on the point of change (for example, the degree of agreement or disagreement) of the trend fluctuation for each measurement item regarding the physical condition, it is possible to easily investigate the factors that are presumed to have caused the change in the physical condition of the person P. . Further, for example, factors of changes in the physical condition of the person P can be analyzed and estimated in a complex manner based on the interrelationships of points of change among a plurality of measurement items.

<Overall Supplement>
Sleeping place definition area 501 (see FIG. 5) may not be set as a three-dimensional space, but may be set, for example, in two dimensions (eg, XY axes excluding the Z-axis direction).

The items to be measured by the millimeter wave radar 10 may not be all of the non-REM sleep time, the apnea sleep time, the awakening time, and the walking speed, and may be, for example, a combination of some measurement items. . In addition, the items to be measured by the millimeter wave radar 10 are not limited to the four items of non-REM sleep time, apnea sleep time, halfway awakening time, and walking speed.

Some of the four items may be replaced with other items related to the physical condition of the person P detectable by the millimeter wave radar 10. Moreover, five or more items related to the physical condition of the person P that can be detected by the millimeter wave radar 10 may be measured by the millimeter wave radar 10 .

Also, the number of persons to be detected by the millimeter wave radar 10 is not limited to one, and may be two or more. According to the distance resolution of the millimeter wave radar 10, for example, body motion and movement can be detected for each person, so the above-described change in physical condition can be detected for each of a plurality of persons.

Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is obvious that a person skilled in the art can conceive of various modifications or modifications within the scope described in the claims, and these also belong to the technical scope of the present disclosure. Understood. Also, the components in the above embodiments may be combined arbitrarily without departing from the gist of the disclosure.

In addition, the specific examples of the present disclosure are merely illustrations and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples illustrated above.

In addition, the notation of "... section" in the above-described embodiments may be "... circuitry," "... device," "... unit," or "... module." may be replaced with other notations such as

In addition, the present disclosure can be realized by software, hardware, or software in cooperation with hardware. For example, the functionality of the system described above can be implemented by a computer program.

FIG. 15 is a diagram showing the hardware configuration of a computer (or information processing device) that implements the functions of each device that constitutes the physical condition detection system 1 described above by means of a program. In FIG. 15, a computer 1100 includes an input device 1101 such as a keyboard, mouse, or touch pad, an output device 1102 such as a display or speaker, a CPU (Central Processing Unit) 1103, a GPU (Graphics Processing Unit) 1104, a ROM (Read Only Memory). ) 1105, RAM (Random Access Memory) 1106, storage device 1107 such as a hard disk device or SSD (Solid State Drive), DVD-ROM (Digital Versatile Disk Read Only Memory) or from a recording medium such as USB (Universal Serial Bus) memory A reading device 1108 for reading information and a transmitting/receiving device 1109 for communicating via a network are provided.

For example, the reading device 1108 reads a program for realizing the functions of the terminal PC 20 from a recording medium, and stores the program in the storage device 1107 . Alternatively, the transmitting/receiving device 1109 communicates with a server device (which may be the server 30 or may be a server device different from the server 30) connected to the network, and realizes the functions of the above devices downloaded from the server device. program is stored in the storage device 1107 .

The CPU 1103 copies the program stored in the storage device 1107 to the RAM 1106 and sequentially reads and executes the instructions included in the program from the RAM 1106, thereby realizing the functions of the terminal PC 20 according to the above-described embodiment.

Each functional block used in the description of the above embodiments is partially or wholly implemented as an LSI, which is an integrated circuit, and each process described in the above embodiments is partially or wholly , may be controlled by one LSI or a combination of LSIs. An LSI may be composed of individual chips, or may be composed of one chip so as to include some or all of the functional blocks. The LSI may have data inputs and outputs. LSIs are also called ICs, system LSIs, super LSIs, and ultra LSIs depending on the degree of integration.

The method of circuit integration is not limited to LSI, and may be realized with a dedicated circuit, a general-purpose processor, or a dedicated processor. Further, an FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of the circuit cells inside the LSI may be used. The present disclosure may be implemented as digital or analog processing.

Furthermore, if a technology for integrating circuits that replaces LSIs emerges due to advances in semiconductor technology or another technology derived from it, that technology may naturally be used to integrate the functional blocks. Application of biotechnology, etc. is possible.

The present disclosure can be implemented in all kinds of apparatuses, devices, and systems (collectively referred to as communication apparatuses) having communication functions. A communication device may include a radio transceiver and processing/control circuitry. A wireless transceiver may include a receiver section and a transmitter section, or functions thereof. A wireless transceiver (transmitter, receiver) may include an RF (Radio Frequency) module and one or more antennas. RF modules may include amplifiers, RF modulators/demodulators, or the like. Non-limiting examples of communication devices include telephones (mobile phones, smart phones, etc.), tablets, PCs (laptops, desktops, notebooks, etc.), cameras (digital still/video cameras, etc.), digital players (digital audio/video players, etc.), wearable devices (wearable cameras, smartwatches, tracking devices, etc.), game consoles, digital book readers, telehealth telemedicine Devices, vehicles or mobile vehicles with communication capabilities (automobiles, planes, ships, etc.), and combinations of the various devices described above.

Communication equipment is not limited to portable or movable equipment, but any type of equipment, device or system that is non-portable or fixed, e.g. smart home devices (household appliances, lighting equipment, smart meters or measuring instruments, control panels, etc.), vending machines, and any other "Things" that can exist on the IoT (Internet of Things) network.

Also, in recent years, CPS (Cyber Physical Systems), a new concept that creates new added value by linking information between physical space and cyber space, is attracting attention in IoT (Internet of Things) technology. This CPS concept can also be employed in the above embodiments.

That is, as a basic configuration of CPS, for example, an edge server located in physical space and a cloud server located in cyber space are connected via a network, and processing is performed by processors installed in both servers. Distributed processing is possible. Here, each processing data generated in the edge server or cloud server is preferably generated on a standardized platform. By using such a standardized platform, various sensor groups and IoT application software can be used. Efficiency can be achieved when constructing a system that includes.

Communication includes data communication by cellular system, wireless LAN system, communication satellite system, etc., as well as data communication by a combination of these.

Communication apparatus also includes devices such as controllers and sensors that are connected or coupled to communication devices that perform the communication functions described in this disclosure. Examples include controllers and sensors that generate control and data signals used by communication devices to perform the communication functions of the communication apparatus.

Communication equipment also includes infrastructure equipment, such as base stations, access points, and any other equipment, device, or system that communicates with or controls the various equipment, not limited to those listed above. .

The disclosure contents of the specification, drawings and abstract contained in the Japanese application of Japanese Patent Application No. 2021-116993 filed on July 15, 2021 are incorporated herein by reference.

An embodiment of the present disclosure is suitable for detecting changes in a person's physical condition, for example.

1 body condition detection system 10 millimeter wave radar 20 personal computer (PC)
30A, 30B Server 40A, 40B Database (DB)
70 radar control device 80 state change detection device 90 display terminal device 101 detection unit 102 processing unit 701 radar communication unit 702 radar data storage unit 801 radar data reading unit 802 sleep state estimation unit 803 walking speed estimation unit 804 estimation result storage unit 805 estimation Result reading unit 806 State change detection processing unit 807, 901 Communication unit 808 Radar data storage unit 809 Estimation result information storage unit 902 Display unit 921 State display area 922 Notification area 1100 Computer 1101 Input device 1102 Output device 1103 CPU
1104 GPU (Graphics Processing Unit)
1105 ROM (Read Only Memory)
1106 RAM (Random Access Memory)
1107 storage device 1108 reading device 1109 transmitting/receiving device 1110 bus P person

Claims (11)

an estimating unit for estimating a plurality of items related to the physical condition of the person in chronological order based on information obtained by detecting the person by radar;
a detection unit that detects changes in the physical state of the person based on estimation results obtained in time series for the plurality of items;
an output unit that outputs information about the detected change in the physical state of the person;
An information processing device. The plurality of items include the sleeping state and walking speed of the person,
The information processing device according to claim 1 . Estimating the sleep state includes estimating at least one of each time period of non-REM sleep, apnea sleep, and midway awakening,
The information processing apparatus according to claim 2. The estimation unit sets an area corresponding to the sleeping place of the person, and estimates the time zone of the sleeping state based on the position of the person indicated by the information with respect to the area.
The information processing apparatus according to claim 2. The estimation unit estimates the walking speed based on the information in the time zone other than the estimated time zone.
The information processing apparatus according to claim 4. The area has a spatial size corresponding to a three-dimensional space occupied by at least one of the person's supine position, prone position, and side-lying position in the sleeping place.
The information processing apparatus according to claim 4. The estimation unit estimates the walking speed based on the change in the position of the person and the time required for the change in position, which are indicated in the information.
The information processing apparatus according to claim 2. The detection unit detects a timing at which a difference between a prediction model obtained by applying an autoregression model to the estimation results obtained in the time series and the estimation results is equal to or greater than a threshold, and the estimation obtained in the time series. Detect as a change point in the trend of the result,
The information processing device according to claim 1 . The radar is a millimeter wave radar,
The information processing device according to claim 1 . By one or more information processing devices,
estimating a plurality of items related to the physical condition of the person in chronological order based on information obtained by detecting the person by radar;
Detecting a change in the physical state of the person based on the estimation results obtained in chronological order for the plurality of items,
outputting information about the detected change in the physical state of the person;
Information processing methods. By one or more information processing devices,
a process of estimating a plurality of items related to the physical condition of the person in time series based on information obtained by detecting the person by radar;
A process of detecting a change in the physical state of the person based on the estimation results obtained in time series for the plurality of items;
a process of outputting information about the detected change in the physical state of the person;
to run
program.

Images