JP2014068714A - Joint angle measuring system - Google Patents

Abstract

The present invention provides a system in which a subject can measure a joint angle by opening and closing a specified joint with his own will.
An imaging unit with depth information for generating captured image data of a subject, depth information of each pixel of the captured image data, and skeleton (skeleton) information of the subject in a virtual three-dimensional space, and the skeleton ( A joint angle calculation unit that generates space vector information of two sides that are specified in the skeleton) and that is connected to a joint whose range of motion is measured, and calculates an angle of the joint based on the space vector information of the two sides.
[Selection] Figure 1

Description

  The present invention relates to a joint angle measurement system capable of automatically grasping a joint angle of a subject for the purpose of confirming the effect of rehabilitation training.

  Until now, in rehabilitation medical care at medical institutions, when checking the effects of rehabilitation training, an assistant has an angle meter and directly touches the subject's joint part to manually joint angle (ROM: joint range of motion) Was measuring. In the manual measurement of ROM, the assistant attaches a large angle meter near the joint to be measured and performs the measurement according to the work procedure specified in advance.

  On the other hand, as a conventional technique, there has been proposed a joint range-of-motion inspection training system that can perform accurate measurement in a short time with respect to ROM inspection / training of rehabilitation patients (Patent Document 1).

  The proposed joint range-of-motion inspection training system includes two or more video cameras, images a measurement target (subject), and analyzes a digitized signal using a computer. As a result, the spatial positions of a plurality of markers (body positions) with different colors attached to the measurement object can be obtained.

  In the ROM measurement, the movement of the peripheral part according to the movement of the joint can be tracked by pasting a marker having a different color around the joint to which the movable range is to be measured. Furthermore, a calculation formula for calculating a joint angle from a spatial position of a marker to be attached around the joint is defined in advance.

  Therefore, the joint angle calculation means calculates the actual joint angle by substituting the spatial position of the marker obtained by the body part position measurement means into the calculation formula.

JP 2002-584 A

  In the system described in Patent Document 1, a plurality of video cameras are used. Moreover, the process which converts the picked-up image of a test subject into a digital image and extracts a body part from a digital image is required. Furthermore, it is necessary to define a combination of body parts, a joint angle, and a calculation formula that are defined in advance for each joint.

  As described above, in the method of manually measuring by a conventional assistant, it is necessary to measure with a large angle meter attached to the subject and record the measurement result every time the measurement is performed. In the system described in Patent Document 1, the apparatus becomes a large scale.

  In view of such prior art, the object of the present invention is to make it easier for the subject and the assistant to measure the joint angle displayed in real time by opening and closing the specified joint with his own will. It is to provide a system that can.

  The first aspect of the joint angle measurement system according to the present invention that achieves the above-described object is that the subject's captured image data, depth information of each pixel of the captured image data, and the subject's skeleton (skeleton) in a virtual three-dimensional space An imaging unit with depth information for generating information, and skeleton (skeleton) information obtained from the imaging unit with depth information, and generating space vector information of two sides connected to a joint for measuring a range of motion, It has a joint angle calculation part which calculates the angle of the joint based on space vector information of two sides.

  In the joint angle measurement system according to the present invention that achieves the above-mentioned object, in the first aspect, the joint angle calculation unit displays the skeleton (skeleton) information, and in the displayed skeleton (skeleton) information, the movable A display unit capable of specifying a joint for measuring a region, wherein the joint angle calculation unit captures the correspondence between the joint for measuring the specified movable range and the two sides connected to the joint with the depth information Generating space vector information of the two sides based on depth information obtained from the unit, calculating an angle of the joint based on the space vector information of the two sides, and displaying the calculated angle of the joint on the display unit It is characterized by having a CPU to be operated.

  In the joint angle measurement system according to the present invention that achieves the above object, in the above, the joint angle calculation unit includes a memory that stores a maximum value and a minimum value of a joint angle calculated after a predetermined time, A change in the range of motion of the joint is displayed on the display based on a minimum value of a joint angle calculated after a predetermined time.

  In the joint angle measurement system according to the present invention that achieves the above-described object, the CPU further specifies the joint that measures the range of motion on the display surface in the skeleton information displayed on the display. The rehabilitation menu is displayed, and a warning is displayed for an incorrect joint movement corresponding to the examination item selected from the rehabilitation menu.

  Moreover, the joint angle measuring system according to the present invention that achieves the above-described object is characterized in that, in each of the above features, the joint angle calculation unit includes means for transferring the calculated angle information to a remote place. .

  The joint angle measurement system according to the present invention that achieves the above-described object is, as a specific aspect, in the one aspect, the imaging unit with depth includes captured image data of a subject, depth information of each pixel of the captured image data, and , Generating the subject's skeleton information in a virtual three-dimensional space in real time, and the joint angle calculation unit generates, in real time, space vector information of two sides connected to the joint for measuring the movable range, The angle of the joint is calculated in real time based on the space vector information of the two sides.

  In the specific aspect, the joint angle measurement system according to the present invention that achieves the above-described object is such that the imaging unit with depth includes captured image data of a subject, depth information of each pixel of the captured image data, and a virtual three-dimensional space. The skeleton (skeleton) information of the subject is generated in real time, and the joint angle calculation unit generates space vector information of two sides connected to the joint for measuring the movable range in real time, and the space of the two sides The joint angle is calculated in real time based on vector information, and the display displays the joint angle calculated in real time in real time.

  Furthermore, the joint angle measurement system according to the present invention that achieves the above-described object is that, in the specific aspect, the imaging unit with depth includes captured image data of a subject, depth information of each pixel of the captured image data, and virtual tertiary. The skeleton (skeleton) information of the subject in the original space is generated in real time, and the joint angle calculation unit generates space vector information of two sides connected to the joint for measuring the movable range in real time, and the two sides The angle of the joint is calculated in real time based on the space vector information and the means for transferring the calculated angle information to the remote location is configured to transfer the calculated angle information to the remote location in real time. To do.

  According to the first aspect of the present invention, it is possible to automatically record the measurement items, the maximum angle and the minimum angle of the ROM during the measurement. Therefore, the amount of work for the assistant can be reduced.

  Further, by combining the display, it is possible to easily confirm the effect of rehabilitation training from the recorded data.

  In addition, by transmitting measurement data to a remote location and combining it with a display system installed at a remote location, the effectiveness of rehabilitation training can be easily confirmed as part of the function of the remote diagnosis support system in a facility without a physical therapist be able to.

1 is a conceptual diagram of an embodiment of a real-time joint angle display system according to the present invention. It is a figure which shows the structural example of a joint angle measuring apparatus. It is a figure which shows an example of a skeleton (skeleton) model. It is a processing flow performed by the joint angle measuring apparatus having the configuration shown in FIG. It is an example of a display of the 1st field of 2 fields displayed on the display of a joint angle measuring device. It is an example of a display of the 2nd field of 2 fields displayed on a display. It is a figure explaining the menu display which performs rehabilitation. It is a part of the display of the skeleton displayed in the image display area of the first field of the display. It is an example of the 2nd field of a display in the time of measurement of a rehabilitation item.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a conceptual diagram of an embodiment of a joint angle measurement system according to the present invention.

  The basic configuration of the joint angle measurement system is configured by connecting an imaging unit 1 with depth information and a joint angle calculation unit 2. The connection unit 1 and the calculation unit 2 may be integrated on one board, or the imaging unit 1 with depth information and the joint angle calculation unit 2 are independent devices, and are connected by an interface line 3. It may be.

  The imaging unit with depth information 1 is the captured image data of the subject 100 including the background at a constant period (for each frame), the depth information (size in the Z direction) at each pixel of the captured image, and the image of the subject 100 in the captured image. , And corresponding skeleton model information is generated and output from the estimated image of the subject 100.

  The depth information represents the distance to the subject 100 in the normal direction of the imaging surface of the imaging unit 1 with depth information, and the skeleton model information is three-dimensional information obtained by estimation from the depth information. It is information in a virtual three-dimensional space in the sense that it is information.

  As an example of an existing device that generates captured image and depth information including the skeleton model information as output information, a game terminal Xbox 360 (registered trademark) of US Microsoft Corporation or an input terminal device used for a personal computer. Some Kinect (R) can be used.

  The joint angle calculation unit 2 is an information processing apparatus having a function equivalent to a general personal computer. FIG. 2 shows an example of the configuration.

  The joint angle calculation unit 2 includes a CPU 20 as a main processing unit. The CPU 20 executes an application program (also stored in the storage device 21) that implements the functions of the present invention based on an OS (operation system) stored in the storage device 21 such as an HDD.

  The captured image data of a fixed period (for each frame) sent from the imaging unit 1 with depth information to the joint angle calculation unit 2, depth information corresponding to the pixels of the image, and skeleton model information of the subject 100 are input. Input through the output interface 22.

  The joint angle calculation unit 2 converts the input captured image data and skeleton (skeleton) model information into a video signal, draws it in the video memory 23, and displays it on the display 24. The depth information and the skeleton model information are stored in the RAM 25 at the same time.

  Further, when measurement is performed in a facility without a physical therapist, data is transmitted through a communication device 27 such as a modem, a LAN adapter, an electric / photoelectric converter, etc., and a remote physical therapist is requested for diagnosis. It is also possible to build a support system.

  Returning to FIG. 1, the configuration of the imaging unit 1 with depth information will be further described.

  The imaging unit 1 with depth information shown in FIG. 1 outputs the captured image of the subject 100, the depth information (Z-axis coordinates) corresponding to the pixel of the image, and the skeleton model information of the subject 100 at a constant period (every frame). It is shown as a conceptual block diagram similar to a terminal device Kinect (registered trademark) of Microsoft Corporation.

  In FIG. 1, the imaging camera 10 of the imaging unit 1 with depth information is an imaging unit that captures an image of a predetermined range including the subject 100. Under the control of the control unit 13, infrared rays are emitted from the infrared radiation unit 11, and infrared rays reflected from the subject 100 are detected by the infrared detection unit 12. Captured image data from the imaging camera 10 and reflected infrared rays detected by the infrared detection unit 12 are input to the control unit 13.

  The control unit 13 generates a virtual three-dimensional space based on the captured image data from the imaging camera 10 and the reflected infrared rays detected by the infrared detection unit 12 by executing a control program stored in the memory 14. , Distance (depth) information (Z-axis direction [direction toward the subject 100 side]) is generated for each pixel of the virtual three-dimensional space image within the imaging angle of view for every fixed period (one frame).

  Furthermore, skeleton (skeleton) model information is generated by applying a subject image in a virtual three-dimensional space corresponding to the imaging angle of view to a predetermined skeleton (skeleton) model from the depth information.

  Here, the imaging unit with depth information 1 is superior to the WEB camera that captures a normal planar image in the following points.

  In the case of using a WEB camera that captures a planar image, for example, in order to measure the movable range of the elbow joint of the arm, the arm is identified by image processing, and the joint angle can be calculated based on the planar image data.

  However, since the distance information in the depth direction is missing, the precondition that the angle formed by the elbow to be measured can be accurately calculated from image processing is that the plane of the angle formed by the elbow is parallel to the camera lens surface. Is required. On the other hand, the imaging unit 1 with depth information shown in FIG. 1 measures depth distance information (Z-axis coordinates) corresponding to each pixel and notifies the joint angle measuring device 2 of the depth distance information. The spatial coordinates can be grasped.

  Thereby, more accurate angle calculation is possible compared with a plane camera.

  FIG. 3 is a diagram showing an example of the skeleton (skeleton) model obtained by the imaging unit 1 with depth information, and is represented by a plurality of skeleton constituent elements and connecting portions connecting the skeleton constituent elements.

That is, constituent elements conceptually representing the trunk, upper limbs, and lower limbs constituting the human body are represented by sides 30, 31 and the like, and black circle connecting portions 32 (32 ₁ , 32 ₂ ) connecting the sides 30, 31 etc., which are constituent elements Is shown with the concept of joints.

  For example, as components, the sides 30 and 31 are associated with the forearm portion and the upper arm portion, respectively, and the connection portion 32 is associated with the sides 30 and 31, that is, a joint (elbow joint) connecting the forearm portion and the upper arm portion. ing.

Then, in accordance with the size of the subject (subject), the lengths of the sides 30, 31 and the like between the connection portions 32 (32 ₁ , 32 ₂ ) are changed, and the skeleton (skeleton) of the subject 100 is specified.

Each connection part 32 has three-dimensional coordinates (X, Y, Z) in the virtual three-dimensional space, and when the subject 100 moves, the three-dimensional coordinates (X, Y, Z) of each connection part 32 are changed. The At the same time, each component (side 30 corresponding to the forearm, side 31 corresponding to the upper arm, etc.) has a specified length, and the tertiary of the connecting portions 32 ₁ and 32 ₂ (corresponding to joints) at both ends. Corresponding to the change of the original coordinates (X, Y, Z), the position and angle of the component in the virtual three-dimensional space change.

  FIG. 4 is a processing flow executed by the joint angle measurement calculation unit 2 having the configuration shown in FIG.

  First, depth information and skeleton information generated by the imaging unit 1 with depth information and temporarily held in the RAM 25 through the input / output interface 22 are acquired for each frame (step S1). Next, the obtained skeleton (skeleton) information is drawn on the video RAM 23, and a skeleton (skeleton) image is displayed on the display 24 (step S2).

  FIG. 5 is an example of display on the display 24 of the joint angle calculation unit 2. The display 24 has a two-field information display area on one screen, and the display shown in FIG. 5 is one of the two-field information display areas.

  This display area has an image display area 50 and an input button display area 51. An input signal corresponding to each button designation in the input button display area 51 is input to the joint angle calculation unit 2 through the input / output interface 26, sent to the CPU 20, and corresponding control is executed by the CPU 20.

  The image display area 50 can be displayed by switching the background display button 52 in the input button display area 51 to display a skeleton image on the background image, or to display only the skeleton image without displaying the background image. The display can be switched. In the example shown in FIG. 5, only the skeleton image is displayed without displaying the background image.

  FIG. 6 is a display example of the second field of the above two fields, and displays an operation instruction message for the subject 100 or an assistant such as “Please select a joint to be measured with reference to the screen”.

  The skeleton information displayed on the display 24 shown in FIG. 5 is displayed so that the connection part 32, that is, the joint to be measured and the measurement content (measurement menu) corresponding to the connection part 32 can be selected.

  That is, the subject 100 or the assistant observes the skeleton (skeleton) image of FIG. 5 displayed on the display 24 according to the message display of the second field shown in FIG. The joint portion corresponding to the movable range), for example, the shoulder joint portion 53 in FIG. 5 can be identified by moving the cursor on the screen (step S3).

  That is, information on a portion corresponding to the ROM to be measured specified by the subject 100 by moving the cursor on the screen is input to the joint angle measuring device 2 via the input / output interface 26.

  When information of a portion corresponding to the ROM to be measured is input, a menu (measurement item) indicating a preset type of rehabilitation performed corresponding to the identified shoulder joint portion 53 is displayed.

  FIG. 7 is a diagram for explaining a menu display for performing rehabilitation, and shows only the skeleton (skeleton) image shown in FIG. 5 in an enlarged manner.

  That is, when the shoulder joint portion 53 is specified in FIG. 5, for example, three measurement items of flexion / extension, abduction / addition, and abduction / internal rotation are displayed as a menu display 54 for performing rehabilitation in the vicinity thereof. .

  When the cursor is placed on the “flexion / extension” measurement item and specified, space vector information of two sides centering on the shoulder joint 53 is generated (step S4).

Thus, as shown in FIG. 7, to identify the scapula 53 ₁ and the upper arm portion 53 ₂ which are connected by a shoulder joint 53, the bending of the upper arm 53 _2, enables the angle of the measured corresponding to stretch and (Step S5).

  Here, in FIG. 7, a bar-shaped display 55 indicates a reference index determined corresponding to the rehabilitation menu. That is, it means a position (angle 0 ° C.) serving as a reference for how much the shoulder joint portion 53 can be bent with reference to the reference index 55.

  Next, when the joint is moved according to the rehabilitation items specified above, the imaging unit with depth 1 captures the captured image data of the subject, the depth information of each pixel of the captured image data, and the skeleton information of the subject in the virtual three-dimensional space. Are sent to the joint angle calculator 2. The joint angle calculation unit 2 generates space vector information of two sides connected to the joint whose range of motion is measured based on the sent information, and calculates the angle of the joint unit based on the generated space vector information of the two sides. Information such as numerical values and skeleton shape is displayed on the display 24 (step S5).

  Note that in one embodiment, the processes of steps S1 to S5 are all performed in real time.

  FIG. 8 shows a part of the display of the skeleton displayed in the image display area 50 of the first field of the display 24. The identified shoulder joint 53 is at an angle to move the upper arm 57 according to the rehabilitation item. The corresponding states are shown sequentially (FIGS. 8 (1) to (3)).

  In the angle display 56, the area changes in a wide and narrow manner with the reference index 55 as a base point corresponding to the moving angle, so that the subject 100 can easily recognize the angle.

  Here, when the arm is moved in a direction different from the intended movement of the arm specified by the specified rehabilitation item, as shown in FIG. 8 (4), a display warning the upper arm 57 (for example, displayed in red) is displayed. Done.

  The above display continues for each frame until the end of the ROM measurement (step S6, Yes) (step S7).

  FIG. 9 is an example of the second field of the display 24 during the measurement of the rehabilitation item.

  In the rightward bending extension test, the current angle at which the shoulder joint portion 53 is moved and the maximum and minimum values of the moved angle are displayed on the screen. Such display is switched by inputting and specifying the corresponding button in the input button display area 51 shown in FIG.

  As described above, according to the present invention, the indirect range of motion of the subject can be automatically acquired. Further, in the embodiment having the display 24, both the subject and the assistant can know the change of the joint angle by the numerical value displayed on the display 24 and the image display. In one embodiment, the measurement value can be displayed on the display 24 in real time in accordance with the movement of the subject's joint. Since the subject's intentional effort to move the joint is immediately reflected in the angle display, the application of the present invention is expected to enhance the effect of rehabilitation treatment aimed at expanding the joint movable range (ROM). The

1 Imaging device with depth information 2 Joint angle measuring device (information processing device)
3 Interface line 10 Imaging camera 11 Infrared radiation unit 12 Infrared detection unit 13 Control unit 14 Memory 20 CPU
21 Storage devices 22, 26 Input / output interface 23 Video RAM
24 display 25 RAM
27 Communication device

Claims (8)

An imaging unit with depth information that generates captured image data of the subject, depth information of each pixel of the captured image data, and skeleton information of the subject in a virtual three-dimensional space;
Two-sided space vector information that is specified in the skeleton (skeletal) information obtained from the imaging unit with depth information and is connected to the joint for measuring the range of motion is generated, and the angle of the joint based on the two-sided space vector information A joint angle calculation unit for calculating
A joint angle measuring system characterized by that. In claim 1,
The joint angle calculator
The skeleton (skeleton) information is displayed, and the displayed skeleton (skeleton) information has a display unit that can identify the joint that measures the range of motion.
The joint angle calculation unit, based on the depth information obtained from the imaging unit with depth information, shows the correspondence between the joint that measures the specified range of motion and the two sides connected to the joint. A CPU that generates information, calculates an angle of the joint based on the space vector information of the two sides, and displays the calculated angle of the joint on the display unit;
A joint angle measuring system characterized by that. In claim 2,
The joint angle calculation unit has a memory for storing a maximum value and a minimum value of a joint angle calculated after a predetermined time, and the maximum value of the joint angle calculated after the predetermined time is set as a minimum value. A joint angle measuring system characterized in that a change in the joint range of motion is displayed on the display. In claim 2,
The CPU causes the display to display a rehabilitation menu in connection with the specification of the joint that measures the range of motion on the display surface in the displayed skeleton information.
A joint angle measurement system that displays a warning for an incorrect joint movement corresponding to an inspection item selected from the rehabilitation menu. In any one of Claims 1-4,
The joint angle calculation unit has a means for transferring information of the calculated angle to a remote place. In Claim 1, the said imaging part with depth produces | generates the test subject's picked-up image data, the depth information of each pixel of the said picked-up image data, and the said test subject's skeleton (skeleton) information in virtual three-dimensional space in real time. ,
The joint angle calculation unit generates, in real time, space vector information of two sides connected to the joint that measures the range of motion, and calculates the angle of the joint in real time based on the space vector information of the two sides. A joint angle measurement system. In any one of Claims 2-4,
The imaging unit with depth generates in real time the captured image data of the subject, the depth information of each pixel of the captured image data, and the skeleton information of the subject in a virtual three-dimensional space,
The joint angle calculation unit generates space vector information of two sides connected to the joint for measuring the range of motion in real time, calculates the angle of the joint in real time based on the space vector information of the two sides,
The joint angle measurement system characterized in that the display displays the joint angle calculated in real time in real time. In claim 5,
The imaging unit with depth generates in real time the captured image data of the subject, the depth information of each pixel of the captured image data, and the skeleton information of the subject in a virtual three-dimensional space,
The joint angle calculation unit generates space vector information of two sides connected to the joint for measuring the range of motion in real time, calculates the angle of the joint in real time based on the space vector information of the two sides,
The joint angle measurement system characterized in that the means for transferring the calculated angle information to the remote location allows the calculated angle information to be transferred to the remote location in real time.

Images