JP2016179048A - Joint load visualization system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a joint load visualization system capable of acquiring motion data and dynamics data of a trainee by a simple method without the trainee having to wear any device. A joint load visualization system for displaying a load applied to a joint of a trainee, wherein a distance camera for measuring a three-dimensional posture of the trainee and an instrument with which the hand of the trainee is in contact. Based on the force sensor 30 that detects the magnitude and direction of the force and moment applied to the trainee, information on the posture of the trainee 10 acquired by the distance camera 40, and information on the force and moment acquired by the force sensor 30. The calculation means 60 calculates the load of the joint of the multi-joint link model imitating the trainee 10, and the display means 50 visually displays the load information of the joint of the multi-joint link model calculated by the calculation means 60. . [Selection diagram] Figure 1

Description

  The present invention is a system for increasing the rehabilitation effect by displaying on the screen the load applied to the joint of the trainer who performs walking training, and enables the trainer to visualize the joint load and the like without wearing special equipment. To do.

In gait training for handicapped and elderly people, it is effective to improve the rehabilitation effect by visualizing the movement of the trainee and the load applied to each part of the body in real time and correcting the posture during training based on it. It is known that there is.
In the conventional visualization system, sensors or markers are attached to the trainee, or the trainee is allowed to walk on the floor reaction force meter (force plate) as described in Patent Document 1 below. Motion data and dynamics data are acquired, and based on this data, the joint load is visualized.

International Publication WO2009 / 147875

However, attaching a sensor or marker to the trainee's body is a burden on the trainee.
In addition, the floor reaction force meter is expensive, and if the floor reaction force meter is spread over a wide range of training facilities, the economic burden increases. Therefore, in practice, the place of training is limited to a narrow range where floor reaction force meters are installed.

  The present invention was devised in consideration of such circumstances, and it is not necessary to attach any equipment to the trainer, and it is possible to obtain exercise data and dynamics data of the trainer by a simple method. The purpose is to provide a system.

The present invention is a joint load visualization system that displays a load applied to a person's joint, and includes a distance camera that measures a three-dimensional posture of the person, and a magnitude of a force and a moment applied to an instrument that the person's hand contacts. The joint load of the multi-joint link model that imitates humans based on the force sensor that detects the direction, the posture information of the person acquired by the distance camera, and the information of the force and moment acquired by the force sensor Calculating means, and display means for visually displaying joint load information of the multi-joint link model calculated by the calculating means.
In this system, it is possible to estimate and display a load applied to a trainee's joint using measurement data of a distance camera and detection data of a force sensor installed in a training instrument such as a parallel bar.

In the joint load visualization system according to the present invention, the display unit displays the joint load information of the multi-joint link model superimposed on the moving image of the person photographed by the distance camera.
Therefore, the trainee can know the state of his / her body and how the load is applied to the joint in real time from the displayed moving image.

Further, in the joint load visualization system of the present invention, the calculation means further includes the information on the posture of the person acquired by the distance camera, the force of the person acquired by the force sensor, and the reaction force acting on the sole of the person. Based on the moment information, the display means visually displays the information on the foot reaction force together with the joint load information of the multi-joint link model.
In this system, the reaction force acting on the sole is displayed together with the load applied to the joint.

In the joint load visualization system of the present invention, the display means displays a moving image of the multi-joint link model, displays joint load information with a bar extending vertically from the joint of the multi-joint link model, and the sole reaction force Is displayed by a bar extending vertically from the foot position of the articulated link model.
Therefore, the trainee can easily know the load applied to his / her joint and the sole reaction force from the displayed moving image.

In the joint load visualization system of the present invention, the display means may change the length or color of the bar according to the joint load and the magnitude of the reaction force on the sole.
Therefore, the trainee can easily know the magnitude of the load applied to his / her joint and the reaction force of the sole from the displayed moving image.

Further, in the joint load visualization system of the present invention, the calculation means calculates a force acting on the hand of the person from the instrument based on the detection data of the force sensor, and the inertial force of the person's body GZMP (Generalized Zero Moment Point) that calculates the balance between the sole reaction force received from the floor of the person's sole and the sole reaction force acting on the left and right soles of the person based on the GZMP A sole reaction force calculation unit for calculating, and a joint load calculation unit for calculating a force acting on a joint of the multi-joint link model based on a force acting on a human hand and a sole.
By having these calculation units, the calculation means can calculate the sole reaction force and joint load of the trainee from the measurement data of the distance camera and the detection data of the force sensor.

  The joint load visualization system of the present invention can acquire exercise data and dynamics data of a trainee by a simple method and visualize the load of the trainee's joint.

The figure which shows typically the joint load visualization system which concerns on embodiment of this invention. The block diagram which shows the joint load visualization system of FIG. FIG. 2 is a flowchart showing the operation of the computer of FIG. Display screen example of the display shown in FIG. Other display screen examples of the display of FIG.

FIG. 1 schematically shows a joint load visualization system of the present invention.
This system is based on a force sensor 30 installed on a parallel bar 20 held by a trainer 10 by hand during walking training, a distance camera 40 that captures a state of walking training, and detection data of the distance camera 40 and the force sensor 30. A computer (calculation means) 60 for calculating the joint load of the trainer 10 and the repulsive force (foot reaction force) against the sole of the floor, and information on the joint load and the foot reaction force in the moving image of the trainer 10 And a display (display unit) 50 that displays the images in a superimposed manner.

The force sensor 30 can detect the force component of the applied force in the three-axis (x, y, z-axis) directions and the moment component acting around each axis.
The distance camera 40 has a function of measuring a reflection time of light irradiated to the photographing target and detecting a distance to the photographing target, and outputs image information in which three-dimensional distance information is added to the image of each pixel. .
The computer 60 uses the image information of the distance camera 40 and the detection information of the force sensor 30 to calculate the sole reaction force and the joint load of the multi-joint link model imitating a human.
The display 50 displays a graphic representing the multi-joint link model, the load applied to the joint, and the sole reaction force superimposed on the moving image of the trainee 10 taken by the distance camera 40.

Next, the operation of the computer 60 will be described.
As shown in FIG. 2, the computer 60 detects the center of gravity position / moment calculation unit 61 that calculates the center of gravity position of the trainee 10 and the moment at the center of gravity from the image information of the distance camera 40, and the detection of the force sensor 30. Based on the data, the hand reaction force calculation unit 62 that calculates the force acting on the hand of the trainer 10 from the parallel bar 20 (hand reaction force), and the inertial force of the body of the trainer 10 and the floor of the trainer 10 through the floor A GZMP calculating unit 63 that calculates a Generalized Zero Moment Point (GZMP) that balances the reaction force of the sole, and a foot reaction force calculation unit that calculates the reaction force acting on the left and right soles of the trainer 10 based on the GZMP. 64, and a joint load calculation unit 65 that calculates the force acting on the joint of the multi-joint link model based on the hand reaction force and the sole reaction force.
Note that these units of the computer 60 are realized by the computer 60 operating according to a program.

FIG. 3 is a flowchart showing the operation procedure of the computer 60.
The center-of-gravity position / moment calculation unit 61 obtains the center-of-gravity position (x, y, z) of the trainee 10 from the three-dimensional distance information of each joint and body segment of the trainer 10 included in the image information of the distance camera 40. The center-of-gravity acceleration is obtained from the temporal difference in the center-of-gravity position, and the angular momentum (Lx, Ly, Lz) of the trainee 10 is calculated (step 1).

The hand reaction force calculation unit 62 calculates the hand reaction force (f _Hjx , f _Hjy , f _Hjz ) acting on the left and right hands based on ( _Equation 1) based on the detection data of the force sensor 30 (step 2).
Here, j = 1 and 2, indicating the left and right hands. Further, (fs _kx , fs _ky , fs _kz ) is a force in each axial direction of the kth force sensor 30, and the hand reaction force is calculated from detection data of the Nth to Mth force sensors 30. The

The GZMP calculation unit 63 calculates the GZMP of the trainee 10 who receives the hand reaction force.
The point where the inertial force of the body of the trainer 10 and the sole reaction force (floor reaction force) balance when the trainer 10 does not receive the hand reaction force is called ZMP (Zero Moment Point). . ZMP (x _E , y _E ) is calculated by (Expression 2) using the center of gravity (x, y, z), angular momentum (Lx, Ly, Lz), mass M, and gravitational acceleration g of the trainee 10. Is done. It is assumed that the mass M of the trainer 10 has been measured in advance.

However, when the trainee 10 receives a hand reaction force, it is necessary to consider the moment of the hand reaction force. Therefore, the GZMP calculating unit 63 obtains the hand position (x _Hj , y _Hj , z _Hj ) of the trainee 10 from the image information of the distance camera 40 (step 31), and GZMP (x _Z , y _Z ) is calculated (step 3).

The foot reaction force calculation unit 64 obtains the foot position of the trainee 10 from the image information of the distance camera 40 in order to calculate the foot reaction force of each foot (step 41). The position of the right foot on the y-axis is y _FR , the position of the left foot on the y-axis is y _FL , the vertical force applied to the right foot to be obtained is f _FR , and the vertical force applied to the left foot is As f _FL , GZMP is a representative point of force applied to each sole,
f _FR (y _FR −y _Z ) = f _FL (y _Z −y _FL )
y _Z = (f _FR y _FR + f _FL y _FL ) / (f _FR + f _FL ) (Equation 4)
It can be expressed as.
In addition, since the inertial force acting on the center of gravity balances with the force acting on the hands and soles, the sum of the forces acting on the left and right soles is (Equation 5).
f _FR + f _FL = F (Equation 5)
Here, F is expressed as (Equation 6).
The sole reaction force calculation unit 64 calculates the sole reaction force of each sole by the following equation (Equation 7) from the relationship of (Equation 4) and (Equation 5).

The joint load calculation unit 65 obtains position information of each joint of the trainee 10 from the image information of the distance camera 40 (step 51), and a multi-joint link model obtained by modeling the body of the trainer 10 as a combination of hinge joints. The load applied to each joint is calculated (step 5).
This is because the position of each joint of the trainee 10 can be acquired from the image information of the distance camera 40, but it is difficult to obtain the posture (joint angle) of the joint.
Therefore, based on the multi-joint link model, the joint position information of each joint is obtained by inverse kinematics calculation, and then the joint angular velocity and joint angle are calculated by calculating the time difference of the calculated joint angles. Acceleration is calculated, and the speed and acceleration of each body segment are calculated.
Here, the space velocity vector of the i-th segment is assumed to be (Equation 8).
In ( _Equation 8), v _oi is a translation speed, and ω _i is an angular speed.
At this time, the equation of motion of the i-th body segment is as shown in (Equation 9).
Here, f _i is a force applied from the body segment of i-1, and τ _i is a moment applied from the body segment of i-1. F ^E _i is a force acting from the outside, and τ ^E _i is a moment acting from the outside. Further, I ^S is a spatial inertia matrix and is given by the following equation (Equation 10).
Here, m is the mass, I is the moment of inertia, and c is the position coordinate of the center of gravity. ^ Represents a distortion symmetric matrix.

The recurrence formula of (Equation 11) is obtained from the equation of motion of (Equation 9).
The values of (Equation 12) are zero at the tips of the hands and feet.
Therefore, the force and moment acting on each body segment can be calculated by calculating sequentially from the tip.
Here, hand reaction force (f _Hjx , f _Hjy , f _Hjz ) is used as the external force at the hand. In addition, since it is difficult to estimate the shearing force with the sole reaction force estimation algorithm, the external force at the foot is (0, 0, f _FR ) (0, 0, f _FL ).
When the force of each joint is estimated by the above method, the joint torque (joint load) in the i-th body segment can be calculated by the following equation (Equation 13).
Here, p _i is a joint position coordinate, and a _i is a joint axis vector.

  The computer 60 repeats the operations from Step 1 to Step 5 until the display on the display 50 is completed (Step 6). By doing so, on the display 50, the actual image of the trainer 10 who performs walking training, the image of the multi-joint link model indicating the posture of the trainer 10, and the information on the sole reaction force and joint load are superimposed. A moving image is displayed in real time.

4 and 5 show images displayed on the display 50 in the demonstration experiment of this system.
Here, the sole reaction force and joint load information are represented by bars extending vertically from the foot position and joint position of the multi-joint link model. The larger the sole reaction force and joint load, the longer the bar length. It is set to stretch.
The gait trainer can check this image and confirm the walking posture, the location where unnecessary force is included, and the like.

Moreover, the height of the horizontal bar which a trainee touches in the state of FIG. 4 and the state of FIG. 5 differ. Therefore, the magnitude | size of the load added to the joint of a trainee's hand differs in FIG. 4 and FIG.
As described above, it is also possible to distinguish from the image of the display 50 based on the magnitude of the joint load whether or not the height of the instrument used by the trainee is suitable for the trainee.
In addition, the display example of the sole reaction force and the joint load shown here is an example, and is not limited thereto. Various modifications are possible, such as color coding according to the magnitude of the sole reaction force and joint load, and display of the sole reaction force and joint load with an arrow having a line width according to the magnitude.

  The joint load visualization system of the present invention can be widely used in hospitals, rehabilitation facilities, elderly facilities, and the like because the physical burden on the walking trainer is low and the cost at the time of introduction is low.

DESCRIPTION OF SYMBOLS 10 Trainer 20 Parallel bar 30 Force sensor 40 Distance camera 50 Display 60 Computer 61 Center of gravity position / moment calculation part 62 Hand reaction force calculation part 63 GZMP calculation part 64 Foot reaction force calculation part 65 Joint load calculation part

Claims (6)

A joint load visualization system for displaying a load applied to a person's joint,
A distance camera for measuring the three-dimensional posture of the person;
A force sensor for detecting the magnitude and direction of the force and moment applied to the instrument that the person's hand contacts;
Calculating means for calculating a joint load of a multi-joint link model imitating a person based on information on the posture of the person acquired by the distance camera and information on the force and moment acquired by the force sensor; ,
Display means for visually displaying joint load information of the multi-joint link model calculated by the calculation means;
A joint load visualization system comprising: The joint load visualization system according to claim 1,
The joint load visualization system, wherein the display means displays joint load information of the multi-joint link model superimposed on a moving image of the person taken by the distance camera. The joint load visualization system according to claim 1,
Further, the computing means further calculates a foot reaction force acting on the sole of the person based on information on the posture of the person acquired by the distance camera and information on the force and moment acquired by the force sensor. Calculated,
The joint load visualization system, wherein the display means visually displays the information on the reaction force of the sole, together with the joint load information of the multi-joint link model. The joint load visualization system according to claim 3,
The display means displays a moving image of the multi-joint link model, displays load information of the joint by a bar extending vertically from a joint of the moving image, and information on the sole reaction force is displayed on the foot of the moving image. A joint load visualization system characterized by displaying a bar vertically extending from a position. The joint load visualization system according to claim 4,
The joint load visualization system, wherein the display means changes the length or color of the bar according to the load of the joint and the magnitude of the reaction force of the sole. The joint load visualization system according to claim 1,
The computing means is
A hand reaction force calculation unit for calculating a force acting on the hand of the person from the instrument based on detection data of the force sensor;
A GZMP calculating unit for calculating a GZMP (Generalized Zero Moment Point), which is a point where the inertial force of the person's body and the sole reaction force received from the floor on the sole of the person;
A sole reaction force calculation unit that calculates a sole reaction force acting on each sole of the person based on the GZMP;
A joint load calculation unit for calculating a force acting on a joint of the multi-joint link model based on a force acting on the hand and sole of the person;
A joint load visualization system comprising: