JP2018108300A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To calculate a measured value closer to an actual situation based on the acquired coordinate information. .. The array of instantaneous stride lengths is approximated to a sine function, and the stride length of the subject is calculated based on the approximated result. [Selection diagram] Fig. 1

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  In recent years, effects in nursing care and rehabilitation have been measured using motion capture technology that digitally records the movement of a person or object.

  As one of the methods of motion capture technology, there is a method of detecting a marker by attaching a marker to a subject and photographing with a camera from a plurality of directions, and converting and recording the movement of the subject into digital data. In addition, there is a method in which a distance from a sensor to a subject is measured using an infrared sensor without using a marker, and the movement of the subject is recorded based on the measured value.

  Patent Document 1 discloses a mechanism using a motion capture technique in a rehabilitation site.

Japanese Patent Laying-Open No. 2015-61577

  Patent Document 1 describes a mechanism for acquiring coordinate information of each joint of a subject acquired by motion capture technology and verifying the motion of the subject.

  In such a mechanism that acquires coordinate information of each joint and verifies the movement of the subject, it is important to correctly recognize the coordinate information of the joint.

  However, there are cases where it is difficult to obtain correct coordinate information such as a state where joints overlap and a state where limbs intersect.

  Therefore, an object of the present invention is to provide a mechanism for calculating a measurement value closer to the actual condition based on the acquired coordinate information.

  The information processing apparatus of the present invention includes a coordinate acquisition unit that acquires coordinate information of a joint of a subject, an instantaneous step calculation unit that calculates an instantaneous step length of the subject based on the coordinate information acquired by the coordinate acquisition unit, and the step calculation An approximation means for executing a process of approximating an array of instantaneous stride calculated by the means to a sine function, and a stride calculation means for calculating the stride of the subject based on a result of the processing by the approximating means. And

  The information processing method of the present invention is an information processing method in the information processing device, wherein the coordinate acquisition unit of the information book device acquires coordinate information of the joint of the subject, and the information processing device An instantaneous stride calculation step in which the instantaneous stride calculation means calculates a subject's instantaneous stride based on the coordinate information acquired in the coordinate acquisition step, and an instantaneous step in which the approximation means of the information processing device is calculated in the stride calculation step. An approximation step for executing a process of approximating the step length array to a sine function, and a step length calculation step in which the step length calculation means of the information processing apparatus calculates the step length of the subject based on the result of the processing in the approximation step. It is characterized by providing.

The program of the present invention is a program that can be executed in the information processing apparatus,
The information processing apparatus includes a coordinate acquisition unit that acquires coordinate information of a joint of a subject, an instantaneous stride calculation unit that calculates a subject's instantaneous stride based on the coordinate information acquired by the coordinate acquisition unit, and the stride calculation unit A program for functioning as a stride calculating means for calculating the stride of the subject based on the result of the processing by the approximating means, and an approximating means for executing processing for approximating the array of instantaneous stride calculated by the above to a sine function.

  According to the present invention, it is possible to calculate a measurement value closer to the actual condition based on the acquired coordinate information.

The figure which shows an example of a function structure of the information processing apparatus 101 The figure which shows an example of the hardware constitutions of the information processing apparatus 101 The flowchart which shows an example of the process which the information processing apparatus in this embodiment performs The flowchart which shows the detail of the process of step S304 The figure which shows the mode of the test subject acquired by the motion information acquisition part 151. An example of a screen on which a subject (601) performing walking measurement and the skeleton information of the subject are displayed Formula for explaining the processing content of step S304 Formula to calculate positive direction amplitude Ap Formula to calculate amplitude A Formula for finding T with minimum residual square sum (RSS) of v (t) and v (t + T) Formula for finding α that minimizes the residual sum of squares of v (t) and f (t) A graph of actual measured stride values The figure which graphed the data which performed the process of step S304 The graph of the data after approximation to the sine function again in step S308

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating an example of a functional configuration of the information processing apparatus 101.

  The motion information acquisition unit 151 detects the motion of a person (in the present invention, a subject (a person who receives care or rehabilitation)) in a space where care or rehabilitation is performed, and acquires motion information of the subject.

  Specifically, the motion information acquisition unit 151 is a sensor group known as Kinect (registered trademark), and includes an RGB camera, a depth sensor, a voice sensor, and the like, and a subject (in the present invention, a subject (care or rehabilitation)). Can recognize the position, movement, voice, face, etc.

  The depth sensor uses a method of measuring the distance to the subject using infrared rays. According to this method, it is possible to measure the distance from the sensor to the subject without attaching a marker or the like to the subject.

  Each data acquired by the motion information acquisition unit 151 is stored in the storage unit as moving image data.

  The motion information acquired by each sensor of the motion information acquisition unit 151 is transmitted to the motion information analysis unit 152, and data analysis is performed.

  The motion information analysis unit 152 analyzes each data acquired by the motion information acquisition unit 151 and converts the motion of the subject into coordinate information.

  FIG. 5 shows the state of the subject acquired by the motion information acquisition unit 151. The motion information acquisition unit 151 detects the movement of the subject and matches the pre-stored human body pattern to obtain the coordinates of the human body surface of the subject. Then, the coordinates of each joint forming the subject's skeleton are calculated, and the subject's skeleton is formed based on the calculated coordinate information.

  Thus, by calculating the coordinate information of each joint and detecting the change in the coordinates of each joint, it becomes possible to recognize the movement of the subject raising his leg or bending his knee.

  The measurement unit 153 performs measurement in the event based on the motion information analyzed by the motion information analysis unit 152 and the setting corresponding to each item.

  FIG. 2 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus 101 according to the embodiment of the present invention.

  As shown in FIG. 2, in the information processing apparatus 101, a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a storage device 204, and an input controller 205 are connected via a system bus 200. , An audio input controller 206, a video controller 207, a memory controller 208, and a communication I / F controller 209 are connected.

  The CPU 201 comprehensively controls each device and controller connected to the system bus 200.

  The ROM 202 or the storage device 204 is a control program executed by the CPU 201 such as a basic input / output system (BIOS) or an operating system (OS), a computer-readable program for realizing this information processing method, and various necessary programs. Holds data (including data table).

  The RAM 203 functions as a main memory, work area, and the like for the CPU 201. The CPU 201 implements various operations by loading a program or the like necessary for executing the processing from the ROM 202 or the storage device 204 to the RAM 203 and executing the loaded program.

  The input controller 205 controls input from the input device 210. Examples of the input device 210 include a keyboard, a touch panel, and a pointing device such as a mouse.

  In the case where the input device 210 is a touch panel, various instructions can be given when the user presses (touches with a finger or the like) according to an icon, a cursor, or a button displayed on the touch panel.

  The touch panel may be a touch panel capable of detecting a position touched with a plurality of fingers, such as a multi-touch screen.

  When the microphone / speaker 211 operates as a microphone function, the voice input controller 206 can control voice input from the microphone / speaker 211 and recognize voice input from the microphone / speaker 211. In addition, when the microphone / speaker 211 operates as a speaker function, the voice input controller 206 can control voice output to the microphone / speaker 211 and output voice to the microphone / speaker 211.

  The video controller 207 controls display on an external output device such as the display 212. The display includes a display of a notebook computer integrated with the main body. The external output device is not limited to a display, and may be a projector, for example. An input device 210 is provided for a device that can accept the touch operation described above.

  Note that the video controller 207 can control a video memory (VRAM) for display control, and a part of the RAM 203 can be used as a video memory area, or a dedicated video memory can be provided separately. Is possible.

  In the present invention, the first video memory area used for display when the user normally uses the information processing apparatus and the display content of the first video memory area are overlapped when a predetermined screen is displayed. It has a second video memory area used for display. The number of video memory areas is not limited to two, and a plurality of video memory areas can be provided as long as the resources of the information processing apparatus permit.

  The memory controller 208 controls access to the external memory 213. The external memory is connected via an adapter to an external storage device (hard disk), flexible disk (FD), or PCMCIA card slot that stores boot programs, various applications, font data, user files, editing files, and various data. A compact flash (registered trademark) memory or the like can be used.

  The communication I / F controller 209 connects and communicates with an external device via the network 214, and executes communication control processing in the network. For example, communication using TCP / IP, telephone lines such as ISDN, and communication using 3G lines of mobile phones are possible.

  The storage device 204 is a medium for permanently storing information, and the form is not limited to a storage device such as a hard disk. For example, a medium such as SSD (Solid State Drive) may be used.

  It can also be used as a temporary memory area during various processes performed by the communication terminal in the present embodiment.

  Note that the CPU 201 enables display on the display 212 by executing, for example, outline font development (rasterization) processing on a display information area in the RAM 203. Further, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the display 212.

  Next, processing executed by the information processing apparatus 101 in this embodiment will be described with reference to FIGS. 3 and 4. 3 and 4 is a process in which the CPU 201 of the information processing apparatus 101 reads and executes a predetermined control program.

    Next, a process of performing measurement while photographing a subject will be described using the flowchart shown in FIG. Note that the process shown in the flowchart of FIG. 3 is a process in which the CPU 201 of the information processing apparatus 101 reads and executes a predetermined program.

  In step S301, the selection of the subject and the selection of the event are received from the user.

The subject information is assumed to have various information registered in advance as a subject list. However, the subject information may be newly registered as well as selected from already registered information.
. In addition, as an event, "open eye one leg standing" that measures the time that you can stand on one foot with your eyes open, "chair standing and sitting" that measures how many times you can stand and sit within the time limit, maximum step length “Maximum one step” to be measured, “Normal walking” to measure the time required for walking at a distance of 5 m, “TUG (Timed Up & Go)” to measure the time to rise up from the chair and sit back on the chair 3 m away is there. Since the present invention aims to measure a subject's stride, it can be applied when “normal walking” or the like is selected.

  By accepting the selection of an event, a setting file or program necessary for the measurement of the event is read.

  In step S302, a shooting instruction is received from the user. When receiving the imaging instruction, the motion information acquisition unit 151 of the information processing apparatus 101 starts imaging the subject. That is, an image including the subject is acquired. Also, motion information that is information obtained by each sensor (such as coordinate information of each joint of the subject) is acquired. In this embodiment, in order to calculate the subject's stride, joint coordinates of the ankle (or a part suitable for measuring the stride such as a toe or a heel) are acquired.

  In step S303, the instantaneous stride of the subject is calculated based on the joint coordinate information acquired in step S302. Specifically, the instantaneous stride is an interval between joint coordinates of the left and right ankles of the subject calculated from the acquired image.

  In step S304, a sine function (calculation formula shown in FIG. 7) that approximates the actual measured stride value (instantaneous stride time-series change) calculated in step S303 is obtained. Details of the processing in step S304 will be described using the flowchart shown in FIG.

  FIG. 4 is a flowchart showing details of the process in step S304.

  In step S401, an array of instantaneous strides is created from the measured stride values obtained by analyzing the acquired image.

  In step S402, the average value of the positive values of the instantaneous stride is calculated, and the positive direction amplitude Ap is calculated. The calculation formula is shown in FIG.

  In step S403, as in step S402, the average value of the negative values of the instantaneous stride is calculated, and the amplitude Am in the negative direction is calculated.

  In step S404, the amplitude A is calculated from the positive direction amplitude Ap and the negative direction amplitude Am calculated in steps S402 and S403. The calculation formula is shown in FIG.

  In step S405, the deviation β is calculated from the difference (Ap−A) between the amplitude Ap calculated in step S402 and the amplitude A calculated in step S404.

  In step S406, T that minimizes the residual sum of squares (RSS) of v (t) and v (t + T) is found by the calculation formula shown in FIG.

  In step S407, the residual sum of squares of v (t) and f (t) is minimized by the calculation formula of FIG. 11 using the amplitude A, the deviation β, and T obtained by the processing of steps S404 to S406. Find α that becomes.

  Through the above processing, the measured step length is corrected to approximate a sine function.

  In step S305, among the values obtained by the processing in step S301, the amplitude calculation is performed on the value until f (t) first becomes 0 and the value after f (t) finally becomes 0. Exclude from the target.

  By this processing, it is possible to exclude values that cause errors when calculating the amplitude.

  In step S306, the actual measurement value in which the distance between f (t) and v (t) is long is invalidated. The distance between f (t) and v (t) is set in advance so as to be invalidated when the distance is, for example, 20 cm or more.

  In step S307, it is determined whether or not the number of invalid values as a result of the processing in step S306 satisfies a predetermined ratio (for example, whether or not the number of values exceeding 50% of the total number of data has been invalidated). judge. The predetermined value is not limited to 50% and can be set as appropriate by an administrator or the like.

  When the number of invalid values satisfies a predetermined ratio (step S307: YES), the process proceeds to step S309.

  If the number of invalid values does not satisfy the predetermined ratio (step S307: NO), the process proceeds to step S308, and the sine function shown in FIG. An approximation process is executed (refine process).

In step S309, each measurement result obtained by the processing in steps S301 to S308 is stored. Specifically, the amplitude of the waveform obtained by the processing in steps S301 to S308 is calculated as the subject's stride, and the value is stored as a measurement result. Also,
FIG. 6 is an example of a screen on which the subject (601) who is measuring walking and the skeleton information of the subject are displayed. As shown at 602 in FIG. 6, the test subject's stride is displayed in a graph in real time. Although not shown, when the measurement is finished and the process shown in the flowchart of FIG. 3 is finished, the graph shown in FIG. 14 may be displayed in an area 602.

  FIG. 12 is a graph of the measured actual stride values. The X axis indicates time (or frame), and the Y axis indicates the instantaneous stride (distance between left and right ankles). That is, it is a graph showing the time series change of the instantaneous stride. It can be said that the value having the maximum instantaneous stride is the subject's stride (the stride at the moment of landing).

  If the left and right stride are equal, a clean sine curve will be drawn, but the actual measurement value may suddenly take a negative value, such as the value represented by 1201 in FIG. This is due to the fact that the ankle of the right foot and the ankle of the left foot were interchanged during measurement. In particular, when the state of walking is photographed from the side of the subject, a moment occurs in which the ankles of both feet are aligned (overlapped), causing a situation where the left and right feet are erroneously recognized.

  FIG. 13 is a graph of data obtained by executing the process of step S304. The rhombus plot shows actual measurement values, and the triangle plot shows the result of approximation to a sine function.

  The process of step S306 is a process of invalidating the value indicated by 1301 in FIG. 13 (actually measured value that is far from the value represented by the approximate sine function).

  FIG. 14 is a graph showing data after approximation to the sine function again in step S308. By approximating the data excluding the value invalidated in step S306, it is possible to obtain data closer to the actual stride.

  If it is determined in step S307 that there are many values to be invalidated, if approximated again, data that is far from the actual stride is obtained. Therefore, it is determined whether or not the approximation is performed again by the determination process in step S307.

  By the above processing, even when the timing at which the stride cannot be measured correctly occurs due to misrecognition of the left and right feet, it is possible to estimate the subject's stride with high accuracy.

  The embodiment as the information processing apparatus has been described above, but the present invention can take an embodiment as a system, apparatus, method, program, recording medium, or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

  Further, the program according to the present invention is a program that allows a computer to execute the processing method of the flowcharts shown in FIGS. 3 and 4, and the storage medium according to the present invention is a program that allows the computer to execute the processing method of FIGS. Is remembered. Note that the program in the present invention may be a program for each processing method of each apparatus shown in FIGS.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program is actually It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the processing and the processing is included.

  Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  The present invention may be applied to a system constituted by a plurality of devices or an apparatus constituted by a single device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention. In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

101 Information processing apparatus

Claims (5)

Coordinate acquisition means for acquiring coordinate information of the joint of the subject;
Based on the coordinate information acquired by the coordinate acquisition means, instantaneous step calculation means for calculating the subject's instantaneous stride,
Approximating means for executing processing for approximating an array of instantaneous stride calculated by the stride calculating means to a sine function;
Based on the result of the processing by the approximating means, a stride calculating means for calculating the stride of the subject,
An information processing apparatus comprising: Of the instantaneous stride calculated by the instantaneous stride calculating means, further comprising invalidating means for invalidating data whose distance from the result of processing by the approximating means exceeds a predetermined value,
The information processing apparatus according to claim 1, wherein the approximating unit executes a process of approximating again with data obtained by excluding data invalidated by the invalidating unit from the instantaneous stride array. Determination means for determining whether the number of values invalidated by the invalidation means satisfies a predetermined condition;
The information processing apparatus according to claim 2, wherein the approximation unit performs the approximation process again when the determination unit determines that a predetermined condition is satisfied. An information processing method in an information processing apparatus,
A coordinate acquisition step in which the coordinate acquisition means of the information writing device acquires coordinate information of the joint of the subject,
An instantaneous stride calculation step of calculating an instantaneous stride of the subject based on the coordinate information acquired by the coordinate acquisition step,
An approximating step in which the approximating means of the information processing apparatus performs a process of approximating an array of instantaneous stride calculated by the step calculating step to a sine function
The step calculation means of the information processing device calculates a step of the subject based on the result of the processing by the approximation step;
An information processing method comprising: A program executable in the information processing apparatus,
The information processing apparatus;
Coordinate acquisition means for acquiring coordinate information of the joint of the subject;
Based on the coordinate information acquired by the coordinate acquisition means, instantaneous step calculation means for calculating the subject's instantaneous stride,
Approximating means for executing processing for approximating an array of instantaneous stride calculated by the stride calculating means to a sine function;
A program for functioning as a stride calculation means for calculating a stride of the subject based on a result of processing by the approximating means.

Images