JP2010172394A - Walking state display - Google Patents

Abstract

A walking state display device capable of analyzing a walking motion in detail with a relatively simple configuration.
SOLUTION: A plurality of marks 2 attached to a predetermined part of a pedestrian's body, an imaging unit 3 that repeatedly captures an optical image including each mark 2 attached to the pedestrian, and a mark for each captured image 2 using an extraction unit 41 that extracts 2, a coordinate setting unit 42 that represents the position of each mark 2 extracted for each captured image by coordinates in a predetermined common coordinate system, and the coordinates of each mark 2. In the case of dividing the walking cycle into a plurality of gait phases based on a predetermined condition for the parameter and a calculation unit 43 that calculates a parameter value for a parameter for representing a walking motion for each captured image. The correspondence between each photographed image and the walking phase is detected based on the parameter value calculated for the photographed image, and a classification unit 5 that classifies each photographed image for each walking phase using the detection result, and classifies each photographed image 5 segment results by using and a display unit 6 for displaying.
[Selection] Figure 1

Description

  The present invention relates to an apparatus for displaying a walking state of a pedestrian.

  In recent years, the pedestrian's walking condition has been measured and improved for the purpose of improving pedestrian walking to a proper way for rehabilitation, a walking style that can enhance exercise effects, or a beautiful walking style with good posture. Various apparatuses for inspecting have been proposed (for example, Patent Documents 1 and 2 below).

  The following Patent Document 1 states that “the left detection means 7 detects landing and takeoff of the left foot of the trainee 2, or landing or takeoff, and the right detection means 8 detects landing and takeoff of the right foot of the trainee 2. The human body side sensor unit 9 serving as a photographing / measuring means is attached to the body of the trainee 2 ... (omitted) ..., this human body side sensor unit 9 Includes an acceleration sensor, a gyro, etc. The left sensor unit 10 as a photographing / measuring means incorporates a force plate for detecting a force applied to the left walking surface 3, and the right sensor unit 11 is on the right side. A force plate or the like for detecting the force applied to the walking surface 4 is incorporated ([0015]) ”,“ the control means 13 is (omitted) ... the left foot of the trainee 2 (omitted) ... or ... (Omitted) ... Right foot wear And the walking cycle, the stance period of the left and right legs, the free leg period, etc., are measured based on the signal of either the takeoff or the landing or the takeoff. (Omitted) ... Measured data such as the measured posture and floor reaction force, the torque of the motor driving the left and right annular belts detected by the driving means 5, the moving speed of the left and right annular belts, etc. These data are input from the control means 13 to the storage means 15. Further, the control means 13 ... (omitted) ... processes the image and measurement data stored in the storage means 15. Output to the display means 16 ([0016]), “when the right foot is landed in one walking cycle, stance phase 1 (after 20% of the stance period has elapsed), stance phase 2 (after 40% of the stance period has elapsed) , Stance phase 3 (after 80% of stance period) and separation Are displayed superimposed five images. Are described as thereby a change in the walking state of the trainee 2 in 1 walking cycle can be observed from the side ([0024]). "

  In Patent Document 2 below, “When it is necessary to attach a marker to the subject 28 in order to calculate a movement trajectory in walking motion analysis, a marker (not shown) is attached to a predetermined part as necessary. When the operator inputs an instruction to capture a moving image to the input unit 26, the moving image captured by the imaging unit 12 is transferred to the image capturing unit 14, where it is digitally encoded and stored as a moving image in the storage unit 16. At the same time, information such as the file name and shooting date of the captured moving image is written in the database provided in the storage unit 16 in advance ([0021]), “a marker indicating the measurement point to the shooting target 28” Is attached, the part of the marker is extracted from the moving image data, the center of gravity of each marker is obtained, and the obtained surface The center of gravity and measuring points of the marker. The measurement points of the markers are described as calculated at each frame captured ([0026]). "

JP 2002-306628 A Japanese Patent Laid-Open No. 2001-420

  In a normal human walking motion, the walking posture periodically changes, but one walking cycle in the walking motion (the foot of one foot has landed on the walking surface, and then the foot has landed on the walking surface). Is a period with a unit period of 1 unit), when multiple types of walking parameters are set to represent (represent) walking motion, such as the change in the position of the external fruit in the vertical direction and the hip joint angle. Each of the walking parameters can be divided into a plurality of sections having characteristic parameter values.

  Here, if it is possible to judge the quality of the walking motion of the pedestrian for each of the sections, it is possible to grasp the difference from the preferable walking method and to provide detailed guidance for improving the walking method in detail. it is conceivable that.

  However, in the said patent document 2, the point which divides the walking motion of a pedestrian into the several area mentioned above is not proposed. In addition, the above-mentioned patent document 1 states that “when the right leg is landing in one walking cycle, stance phase 1 (after 20% of the stance period has elapsed), stance phase 2 (after 40% of the stance period has elapsed), stance phase 3 ( 5 images of 80% of the stance period) and five images of the takeoff are superimposed and displayed. However, in order to detect the walking motion of the pedestrian, various detection means and sensor units are described. (Left side detection means 7, right side detection means 8, human body side sensor unit 9, left side sensor unit 10 and right side sensor unit 11) are necessary, and accordingly, the apparatus becomes larger and requires a great deal of cost.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a walking state display device that can measure and analyze the walking motion of a pedestrian in detail with a relatively simple configuration. And

  The invention according to claim 1 is an optical image including one or a plurality of marks attached to a predetermined part of a pedestrian's body and a mark attached to the pedestrian in a certain optical axis direction and An imaging unit that repeatedly captures images within the imaging range and generates a plurality of captured images, an extraction unit that extracts each of the marks for each of the captured images, and a predetermined common position for each mark extracted by the extraction unit A coordinate setting unit represented by the coordinates of the coordinate system and a coordinate value for each mark set by the coordinate setting unit, and a predetermined parameter value for a walking parameter for representing a walking motion for each captured image Based on the parameter value, the calculation unit for calculating the gait and the captured image of each gait phase of the gait phase divided into a plurality of gait phases based on a predetermined condition. A walking state comprising: a sorting unit that detects each of the captured images using the detection result; and a display unit that displays each captured image using the sorting result of the sorting unit. It is a display device.

  According to the present invention, which walking phase (walking posture) each captured image is captured based on the parameter value is detected, and each captured image is walked using the detection result. Since each phase is divided into phases and each captured image is displayed on the display unit using the result of the division, for example, by displaying an image for each walking phase, the walking motion of the pedestrian is detailed. Can be analyzed.

  Further, according to the present invention, since the pedestrian's walking motion is detected based on the position of the mark obtained from the photographed image, various types of detection are performed in order to detect the pedestrian's walking posture as in the prior art. It is not necessary to provide a sensor, and it is possible to avoid or suppress an increase in cost and an increase in the size of the apparatus as compared with the prior art.

  In addition, as in the invention according to claim 2, the section is more specifically a period from when the heel of one foot has landed on the walking surface until the heel has landed on the walking surface next time. When the walking cycle of the pedestrian is set as a unit and the walking cycle is divided into a plurality of walking phases based on a predetermined condition for the walking parameter, which walking phase each captured image has It is good to detect whether each of the walking motions is captured based on a parameter value calculated for the captured image, and to perform a process of classifying the captured images for each walking phase using the detection result. .

  According to a third aspect of the present invention, in the walking state display device according to the first or second aspect, the predetermined part includes an outer fruit of the pedestrian, and the coordinate system includes the walking of the pedestrian. The coordinate phase is an axis extending in the direction, and the plurality of gait phases, when focusing on each mark attached to the outer fruit of two photographed images obtained sequentially in time, The difference between the coordinates of the mark in the walking direction includes initial grounding, which is a walking phase that first falls within a predetermined range within one walking cycle.

  According to the present invention, it is possible to detect a captured image representing the initial grounding.

  According to a fourth aspect of the present invention, in the walking state display device according to the first or second aspect, the predetermined part includes an outer fruit and a toe of the pedestrian, and the coordinate system is in a vertical direction. The plurality of gait phases has a difference between coordinates in the vertical direction with respect to a mark attached to the outer fruit and a mark attached to the toe of the same photographed image. Includes a load response period that first falls within a predetermined range within one walking cycle.

  According to the present invention, it is possible to detect a captured image representing a load response period.

  According to a fifth aspect of the present invention, in the walking state display device according to the first or second aspect, the predetermined part includes a greater trochanter and a knee of the pedestrian, and the coordinate system includes the walking The coordinate direction is an axis extending in the walking direction of the person, and the plurality of walking phases are the walking direction of the mark attached to the greater trochanter and the mark attached to the knee of the same photographed image. This includes a mid-stance phase in which the difference between the coordinates in the graph becomes zero within one walking cycle.

  According to the present invention, it is possible to detect a captured image representing the mid-stance.

  According to a sixth aspect of the present invention, in the walking state display device according to the first or second aspect, the predetermined part includes the greater trochanter and the knee of the pedestrian, and the coordinate system includes the walking The coordinate direction is an axis extending in the walking direction of the person, and the plurality of walking phases are the walking direction of the mark attached to the greater trochanter and the mark attached to the knee of the same photographed image. This includes the end of the stance in which the difference between each coordinate in the is the maximum within one walking cycle.

  According to the present invention, it is possible to detect a photographed image representing the end of stance.

  According to a seventh aspect of the present invention, in the walking state display device according to the first or second aspect, the predetermined part includes the pedestrian's knees, an external fruit and a toe, and the plurality of walking phases are In the same photographed image, a line connecting the mark attached to the outer fruit and the mark attached to the toe, and a line connecting the mark attached to the outer fruit and the mark attached to the knee When the minutes are set, the ankle joint angle, which is the intersection angle of those line segments, includes the previous swing leg period in which the maximum is within one walking cycle.

  According to the present invention, it is possible to detect a photographed image representing the front swing leg period.

  The invention according to claim 8 is the walking state display device according to claim 1 or 2, wherein the predetermined portion includes the greater trochanter, knee and outer fruit of the pedestrian, and the plurality of walking The phase includes a line connecting a mark attached to the knee and a mark attached to the outer fruit, a mark attached to the knee, and a mark attached to the greater trochanter in the same photographed image. When the line segment connecting is set, the knee joint angle that is the intersection angle of these line segments includes the initial stage of the free leg where the minimum is within one walking cycle.

  According to the present invention, it is possible to detect a captured image representing the initial stage of the free leg.

  A ninth aspect of the present invention is the walking state display device according to the first or second aspect, wherein the predetermined part includes the pedestrian ridge, greater trochanter and knee of the pedestrian, and the plurality of walking The phase is attached to the line segment connecting the mark attached to the greater trochanter and the mark attached to the knee, the mark attached to the greater trochanter, and the shoulder peak in the same photographed image. When the line segment that connects the mark is set, the hip joint angle that is the intersection angle of these line segments includes the middle period of the free leg in which the minimum is within one walking cycle.

  According to the present invention, it is possible to detect a photographed image representing the middle period of the free leg.

  The invention according to claim 10 is the walking state display device according to claim 1 or 2, wherein the predetermined portion includes the greater trochanter and outer fruit of the pedestrian, and the coordinate system is It has an axis extending in the walking direction of the pedestrian as a coordinate axis, and the plurality of gait phases are the mark about the mark attached to the greater trochanter and the mark attached to the outer fruit of the same photographed image. It includes the free leg end in which the difference between the coordinates in the walking direction is maximum within one walking cycle.

  According to the present invention, it is possible to detect a photographed image representing the end of the free leg.

  An eleventh aspect of the present invention is the walking state display device according to any one of the first to tenth aspects, wherein the input for selecting a captured image to be displayed on the display unit for each walking phase is performed. 1 input operation part is further provided, The said display part displays the picked-up image of the walk phase selected by the said 1st input operation part.

  According to this invention, it is provided with the 1st input operation part for performing the input which selects the picked-up image of the object displayed on the said display part for every said walking phase, The said display part is selected by the said 1st input operation part. Since the captured image of the gait phase is displayed, the instructor or the like can analyze the pedestrian's walking motion for each gait phase.

  The invention according to claim 12 is an ideal image storage for storing an ideal image having an ideal parameter value provided in advance for each walking phase in the walking state display device according to any one of claims 1 to 11. The display unit further has a function of displaying the captured image and an ideal image in which the captured image and the walking phase coincide with each other.

  According to the present invention, an ideal image storage unit that stores an ideal image having an ideal parameter value that is provided in advance for each walking phase is further included, and the display unit further includes the captured image and the captured image. And the ideal image that matches the gait phase are displayed together, so leaders can analyze the pedestrian's walking behavior for each gait phase and the pedestrian's walking behavior is ideal. It is possible to make a leader or the like easily recognize how much it differs from the walking motion.

  A thirteenth aspect of the present invention is the walking state display device according to any one of the first to twelfth aspects of the present invention, which stores a standard value for the walking parameter that is preset based on a combination of age and gait phase. A standard value storage unit, and the display unit further has a function of displaying the parameter value derived by the calculation unit and the standard value in a graph.

  According to this invention, since it has a function of displaying the parameter value of the walking parameter calculated by the calculation unit and the standard value in a graph, the instructor or the like compares the walking motion of the pedestrian with the standard value. Can be analyzed.

  According to a fourteenth aspect of the present invention, in the walking state display device according to any one of the first to thirteenth aspects, an input for selecting one captured image from the captured images displayed on the display unit is performed. A second input operation unit for the display, and the display unit further performs an imaging operation within a certain period including a shooting timing of the captured image when one captured image is selected by the second input operation unit. It has a function to display a plurality of photographed images obtained in step 1 on the same screen.

  According to the present invention, when one photographed image is selected by the second input operation unit, a plurality of photographed images obtained by the imaging operation within a certain period including the photographing timing of the photographed image are displayed on the same screen. Since the display function is provided, the instructor or the like can analyze the walking motion of the pedestrian in detail.

  ADVANTAGE OF THE INVENTION According to this invention, a pedestrian's walking motion can be analyzed in detail, avoiding or suppressing the cost increase and the enlargement of an apparatus compared with a prior art.

It is a block diagram which shows the electrical structure of the walking state display apparatus which concerns on this invention. It is a figure which shows the imaging | photography condition of the walk motion by a pedestrian. (A)-(g) is explanatory drawing of the position of each of the shoulder peak, the greater trochanter, the knee, the external fruit, the toe, and each walking parameter. (A) is a figure which shows the difference of the X coordinate of the external fruit in the picked-up image and the picked-up image image | photographed immediately before the said picked-up image calculated | required about each picked-up image for two walk periods, (b). These are the figures which showed the picked-up image of the initial grounding image | photographed at the time shown by arrow P1 of (a). (A) is a figure which shows the difference of each Y coordinate of the external fruit and the toe in the said picked-up image calculated | required about each picked-up image for two walk cycles, (b) is arrow P2, P3 of (a). It is the figure which showed the picked-up image of the load response period image | photographed at the time shown. (A) is a figure which shows the difference of the X coordinate of the greater trochanter and knee in the said picked-up image calculated | required about each picked-up image for two walk cycles, (b) is arrow P4, P5 of (a). It is the figure which showed the picked-up image of the stance middle period image | photographed at the time shown. (A) is a figure which shows the difference of the X coordinate of the greater trochanter and knee in the said picked-up image calculated | required about the picked-up image for two walk cycles, (b) shows by arrow P6, P7 of (a). It is the figure which showed the picked-up image of the stance end image | photographed at the time. (A) is a figure which shows the ankle joint angle (alpha) in the said picked-up image calculated | required about the picked-up image for two walk cycles, (b) is the front play image | photographed at the time shown by arrow P8, P9 of (a). It is the figure which showed the picked-up image of a leg stage. (A) is a figure which shows the knee joint angle (beta) in the said picked-up image calculated | required about the picked-up image for two walk cycles, (b) is the free leg image | photographed at the time shown by arrow P10, P11 of (a). It is the figure which showed the initial picked-up image. (A) is a diagram showing hip joint angles φ in the captured images obtained for the captured images for two walking cycles, and (b) is the middle leg of the free leg imaged at the time indicated by arrows P12 and P13 in (a). It is the figure which showed the picked-up image. (A) is a figure which shows the difference of the X coordinate of the greater trochanter and outer fruit in the said picked-up image calculated | required about the picked-up image for two walk periods, (b) is the arrow P14 of FIG. It is the figure which showed the picked-up image of the free leg end image | photographed at the time shown by P15. It is a figure which shows the display screen which displayed the captured image and the ideal image about the walk phase which this captured image shows side by side. It is a figure which shows an example of the display screen which displays the calculated step length of a pedestrian. When one specific captured image is designated, it is a figure which shows the screen which displayed the some captured image imaged within the fixed period including the imaging timing of the image side by side. It is explanatory drawing of the identification method of the mark with which the external fruit was mounted | worn, and the mark with which the toe was mounted | worn. (A) is a figure which shows a hip joint angle, (b) is a figure which shows a knee joint angle, (c) is a figure which shows an ankle joint angle. It is explanatory drawing of the process performed by an image process part and an extraction part.

  Hereinafter, embodiments of a walking state display device according to the present invention will be described. FIG. 1 is a block diagram showing a configuration of a walking state display device according to the present invention.

  As shown in FIG. 1, the walking state display device 1 includes a mark 2, an imaging unit 3, a coordinate extraction unit 4, a sorting unit 5, and a display unit 6. As shown in FIG. 2, the mark 2 is attached to the shoulder ridge, greater trochanter, knee, outer fruit, and toe of the body of the pedestrian M who is the subject. The pedestrian M walks in a predetermined direction (direction indicated by the arrow X) in the predetermined walking area S with the marks 2 attached to the shoulder ridge, greater trochanter, knee, outer fruit, and toe, respectively.

  The imaging unit 3 includes a CCD (Charge Coupled Device) in which, for example, R (red), G (green), and B (red) color filters are Bayer-arrayed at a ratio of 1: 2: 1. The imaging unit 3 is configured such that the direction of the optical axis and the imaging range are constant, and at least the mark 2 of the body of the pedestrian M is attached during the walking period of the pedestrian M walking in the walking area S in the constant direction. The optical image in a range including the shoulder ridge, greater trochanter, knee, external fruit, and toe portion is taken repeatedly (for example, at a constant cycle).

  In addition, while providing the light emission part which outputs light in the imaging part 3, and providing the reflection part which reflects the light output from the light emission part in the mark 2, the image of the mark 2 is comparatively luminance in the whole image. Therefore, the walking state display device 1 can clearly identify the image of the mark 2 and the other images in the entire image.

  As shown in FIG. 1, the imaging unit 3 includes an image processing unit 31. As shown in FIG. 17, the image processing unit 31 performs color conversion processing on an image (Bayer image) composed of pixel information output from each pixel. In the color conversion process, R (red), G (green), B (at the position of each pixel from a Bayer image in which only pixel information about the color of the color filter disposed on the pixel is obtained from each pixel. This is a process of converting each pixel information of three colors (blue) into an image. For example, the image processing unit 31 obtains pixel information of the color “G (green)” at the position of the pixel where the “R” color filter is disposed from the G (green) pixels located around the pixel. Calculation is performed by averaging or weighting using the obtained pixel information.

  The image processing unit 31 also removes noise in the image after the color conversion process. The image after the color conversion processing and noise removal processing is called a color image. The imaging unit 3 has an unillustrated storage unit that stores the color image, and displays the color image stored in the storage unit when the display unit 6 needs to display the color image. 6 is output.

  Next, the image processing unit 31 performs redness removal processing and gray conversion processing on the color image. The redness removal process is a process of removing redness (R) of a pedestrian's skin that may be erroneously recognized as the mark 2 among the color components of R (red), G (green), and B (blue). This is a process of calculating R> ((G + B) / 2 + α) where RGB image information in one pixel is expressed as R, G, B. Note that “α” is a parameter that is set according to the degree of redness of the image, and for removing redness to the extent that it is not erroneously recognized. The gray conversion process is a process for generating an image (hereinafter referred to as a first gray image) that is assumed to be obtained when no color filter is provided for each pixel from the image after the redness removal process. .

  On the other hand, the imaging unit 3 performs the shooting operation in a state where the pedestrian M does not exist within the imaging range, separately from the shooting operation of the walking motion (mark 2). An image obtained by this photographing operation is referred to as a background image. The background image at this time is a Bayer image in which only pixel information regarding the color of the color filter disposed on the pixel is obtained from each pixel. The image processing unit 31 extracts only the G (green) component from the background image, and is assumed to be obtained when no color filter is provided for each pixel (hereinafter referred to as a second gray image). The gray conversion process is generated.

  When the first gray image and the second gray image are generated, the image processing unit 31 outputs the first gray image and the second gray image to the subsequent extraction unit 41. In the following description, the image output from the imaging unit 3 to the display unit 6 and the image output to the coordinate extraction unit 4 are collectively referred to as a captured image.

  The coordinate extraction unit 4 extracts the mark 2 for each photographed image (the first gray image), and the position of each mark 2 extracted by the extraction unit 41 is a predetermined common. A predetermined walking parameter for representing (representing) a walking motion using the coordinate setting unit 42 expressed by coordinates of the two-dimensional coordinate system and the coordinates for each mark 2 set by the coordinate setting unit 42 And a calculation unit 43 that calculates a parameter value for each captured image.

  As illustrated in FIG. 17, when the extraction unit 41 acquires the first gray image and the second gray image from the image processing unit 31, a background difference obtained by calculating a difference between the first gray image and the second gray image. Generate an image. Further, the extraction unit 41 binarizes the generated background difference image for each pixel, and then performs labeling for extracting pixels whose difference is equal to or greater than a predetermined threshold.

Further, the extraction unit 41 marks a region where the area S is within a predetermined range (S _α <S <S _β ) among regions where pixels having a difference equal to or greater than a predetermined threshold are consecutive. And the center point (or centroid) of each extracted region is set as the position of the mark 2.

  The coordinate setting unit 42 sets, for example, a two-dimensional coordinate system in which the position of the pedestrian immediately before the start of walking is the origin, the traveling direction of the pedestrian is the X axis, and the vertical direction is the Y axis, and the extraction unit 41 The coordinates of the center point obtained by the above are set as coordinates representing the position of the mark 2.

  The calculation unit 43 performs identification of which part (shoulder peak, greater trochanter, knee, outer fruit, and toe) of the pedestrian is attached to each mark 2 as follows.

  First, the calculation unit 43 tentatively assigns the wearing positions in order from the mark 2 with the largest Y coordinate to the shoulder peak, greater trochanter, knee, outer fruit, and toe. Next, the calculation unit 43 includes the mark 2 attached to the shoulder ridge, the mark 2 attached to the greater trochanter, the mark 2 attached to the knee, and the mark 2 attached to other parts (external fruit and toe). As for the discrimination between the height position of the acromus during walking, the height position of the greater trochanter, the height position of the knee and the height position of others (the external fruit and the toes), It is constant at any point in time and can be identified only by the Y coordinate.

  Therefore, using this point, the image of the mark 2 attached to the shoulder ridge, the image of the mark 2 attached to the greater trochanter, and the image of the mark 2 attached to the knee are determined. That is, the calculation unit 43 determines that the mark 2 having the largest Y coordinate is the mark 2 attached to the shoulder peak, and then sets the mark 2 having the next largest Y coordinate to the mark 2 attached to the greater trochanter. The mark 2 having the next largest Y coordinate is determined to be the mark 2 attached to the knee.

  Further, the calculation unit 43 identifies the mark 2 attached to the outer fruit and the mark 2 attached to the toe as follows. As shown in FIG. 15A, the calculation unit 43 sets the positions of the marks 2 respectively attached to the external fruit, the knee, and the toe as points A, B, and C, the point A as the start point, and the point B as the end point. A vector AB to be generated and a vector AC having the point A as a start point and the point C as an end point are generated.

Next, the calculation unit 43 calculates the outer product of the vector AB and the vector AC. That is, assuming that the vector AB = (X ₁ , Y ₁ ) and the vector AC = (X ₂ , Y ₂ ), the calculation unit 43 calculates the outer product AB × AC of the vector AB and the vector AC.
AB × AC = X ₁ × Y ₂ −X ₂ × Y ₁ (1)
Calculated by

On the other hand, as shown in FIG. 15A, when the angle to the vector AB with respect to the vector AC is represented by θ and the vectors AB and AC are simply represented by AB and AC, the outer product AB × AC is
AB × AC = | AB || AC | sinθ (2)
Can also be expressed. Here, | AB | is the length (scalar amount) of the vector AB, and | AC | is the length (scalar amount) of the vector AC.

  Here, when the value of the outer product calculated by the equation (2) becomes a negative value, as shown in FIG. 15B, the angle θ is a negative value, and the positions of the knee, the external fruit, and the toe Although the relationship shows that the relationship is the positional relationship shown in FIG. 15B, such a state does not occur due to the structure of the human body. That is, in this case, it is indicated that the previous provisional assignment relating to the positions of the external fruit, the knee, and the toe is not correct.

  The calculation unit 43 uses this to determine whether or not the calculated cross product value of the vector AB and the vector AC is a positive value, and when the cross product value is a positive value, FIG. As shown in a), it is determined that the mark 2 with the larger Y coordinate is the mark 2 attached to the outer fruit, and the mark with the smaller Y coordinate is the mark 2 attached to the toe. That is, the calculation unit 43 assumes the above-described assignment contents (contents that the mounting position of the mark 2 is the shoulder peak, the greater trochanter, the knee, the external fruit, and the toe in order from the mark 2 with the largest Y coordinate). Is determined to be correct.

  On the other hand, when the calculation unit 43 determines that the value of the outer product is a negative value, as illustrated in FIG. The temporary assignment of the tip set as the child, the knee, the external fruit, and the toe is incorrect, and as shown in FIG. 15C, the mark 2 with the larger Y coordinate is attached to the toe, It is determined that the mark with the smaller coordinate is the mark 2 attached to the outer fruit.

  Through the above processing, the calculation unit 43 identifies the mounting positions of the five marks 2. Then, the calculation unit 43 performs the process of specifying the mounting positions of the five marks 2 for each of the plurality of captured images obtained from the imaging unit 3, and outputs the processing result to the sorting unit 5.

  By the way, the walking motion of the pedestrian is a plurality of types for representing the walking motion such as a distance between two specific parts of the body part of the pedestrian, or a hip joint angle, ankle joint angle, a knee joint angle, which will be described later. When the walking parameter is set, the parameter value periodically changes. The period of the change is a period in which the period from when the heel of one foot has landed on the walking surface until the heel next landed on the walking surface is one unit. The one walking cycle can be divided into a plurality of sections (walking phases) having characteristic parameter values in any walking parameter.

  In the present embodiment, when one walking cycle is divided into a plurality of walking phases in this way, the walking phase indicated by each captured image indicates which walking phase each captured image captures. And each of the captured images is classified for each walking phase using the detection result. The parameter value for detecting which gait phase each captured image is captured can be detected by the coordinates of the mark 2 in each captured image. The plurality of gait phases are an initial ground contact, a load response period, a mid-stance phase, an end-of-stance phase, a pre-free leg phase, an early swing phase, an intermediate swing phase, and an end phase of the free leg, which will be described below.

  The classification unit 5 calculates the parameter value of each walking parameter indicated by each captured image, and sets one walking cycle to a plurality of walking phases based on a predetermined condition (feature point) for the parameter value of each walking parameter. In the case of division, each captured image is detected based on the parameter value calculated for each captured image, and each captured image is walked using the detection result. It is classified by phase.

  FIGS. 3A to 3G are explanatory diagrams of acromion, greater trochanter, knee, outer fruit, toe position, and each walking parameter. In the following description, the coordinates of the mark M attached to each part of the pedestrian's body (shoulder peak, greater trochanter, knee, external fruit, toe) are expressed as the coordinates of each part. .

  4 to 11, the imaging range of the imaging unit 3 has a length of four steps (a length corresponding to two walking cycles), and the imaging unit 3 images a walking motion for four steps at a predetermined time interval. The graph which shows the change of the parameter value of each walking parameter which each 60 picked-up images obtained by this shows, and the picked-up image which shows the walking posture of each walking phase are shown. In the two walking cycles, each gait phase except the initial grounding occurs twice (initial grounding only once).

  FIG. 4A shows the captured image obtained for each captured image when the identification numbers 1 to 59 assigned to the captured images corresponding to the two walking cycles in the order of the capturing timing are on the horizontal axis. It is a figure which shows the difference (vertical axis | shaft) of the X coordinates of the outer fruit in the picked-up image image | photographed immediately before the said picked-up image. This difference is one of the walking parameters.

  As shown in FIG. 4A, the initial grounding is the first point in time when the difference between the X coordinates of the outer fruits in the photographed image and the photographed image taken immediately before the photographed image is within a predetermined range. (For example, the time indicated by the arrow P1). The predetermined range is obtained, for example, by capturing a walking image in advance before performing measurement and taking the difference between the X coordinates at the time of initial contact in the captured image. In addition, the predetermined range is affected by the impact strength at the time of landing, but in the case of normal walking motion, it is not necessary to set it to a large range, and a length corresponding to the width of several pixels is required. It may be set as an allowable error. FIG. 4B shows a captured image (captured image indicating initial grounding) captured at the time indicated by the arrow P1 in FIG.

  FIG. 5 (a) shows the Y coordinates of the external fruit and the toe in the photographed image obtained for each photographed image when the identification numbers 1 to 59 of the photographed images for the two walking cycles are set on the horizontal axis. 4 is a diagram showing the difference (ΔY in FIG. 3B; vertical axis). This difference is also one of the walking parameters.

  In the load response period, as shown in FIG. 5A, the first time point when the difference ΔY between the Y-coordinates of the outer fruit and the toe in each captured image falls within a predetermined range (for example, as indicated by arrows P2 and P3). Time). The predetermined range is obtained, for example, by capturing a walking image in advance before performing measurement and taking the difference between the Y coordinates of the outer fruit and the toe in the captured image. In the load response period, the difference ΔY between the Y-coordinates of the outer fruit and the toe can usually be zero, so that it can be easily determined. However, in an exceptional case, the marker 2 vibrates due to the impact of landing. In this case, since the difference ΔY slightly deviates from zero, a length corresponding to the width of several pixels may be set as an allowable error in consideration of this. FIG. 5B shows a captured image (captured image indicating the load response period) captured at the time indicated by arrows P2 and P3 in FIG.

  FIG. 6A shows the X coordinate of the greater trochanter and the knee in the captured image obtained for each captured image when the identification numbers 1 to 59 of the captured images for the two walking cycles are set on the horizontal axis. 4 is a diagram showing the difference (ΔX in FIG. 3C; vertical axis). This difference is also one of the walking parameters.

  The middle stance phase is a gait phase in which the hip joint is progressing, and as shown by, for example, the arrow P4 and the arrow P5 in FIG. 6A, the difference ΔX between the X coordinate of the greater trochanter and the knee becomes substantially zero This is the point in time when the difference ΔX is closest to zero. FIG. 6B shows a captured image (captured image showing the mid-stance) taken at the time indicated by arrows P4 and P5 in FIG.

  FIG. 7A shows the X-coordinates of the greater trochanter and the knee in the photographed image obtained for each photographed image when the identification numbers 1 to 59 of the photographed images for the two walking cycles are set on the horizontal axis. 4 is a diagram showing the difference (ΔX in FIG. 3C; vertical axis).

  The end of stance is the time when the difference ΔX between the X coordinates of the greater trochanter and the knee becomes the maximum, as indicated by, for example, the arrow P6 and the arrow P7 in FIG. FIG. 7B illustrates a captured image (captured image indicating the end of stance) captured at the time indicated by arrows P6 and P7 in FIG.

  Note that the end of stance is not only the difference ΔX between the X coordinates of the greater trochanter and the knee, but also the gait phase in which the hip joint angle is maximized. Therefore, the captured image shows the end of stance based on this hip joint angle. It can be determined whether or not it represents. The method for calculating the hip joint angle will be described later.

  FIG. 8A shows an ankle joint angle α (see FIG. 3D) obtained for each captured image when the identification numbers 1 to 59 of the captured images for the two walking cycles are set on the horizontal axis. ) Reference; vertical axis). This ankle joint angle α is also one of the walking parameters.

  The front swing leg period is a time point at which the ankle joint angle α is maximized, as shown by arrows P8 and P9 in FIG. FIG. 8B shows a captured image (captured image indicating the previous swing leg period) captured at the time indicated by the arrows P8 and P9 in FIG.

  FIG. 9A shows the knee joint angle β in each captured image obtained for each captured image when the identification numbers 1 to 59 of the captured images for the two walking cycles are on the horizontal axis (FIG. 3 (e)). ) Reference; vertical axis). This knee joint angle β is also one of the walking parameters.

  The initial stage of the free leg is a time point when the knee joint angle β is minimum as shown by arrows P10 and P11 in FIG. FIG. 9B shows a captured image (captured image showing the initial free leg) captured at the time indicated by the arrows P10 and P11 in FIG. 9A.

  FIG. 10A shows a hip joint angle φ in each photographed image obtained for each photographed image when the identification numbers 1 to 59 of the photographed images for the two walking cycles are set on the horizontal axis (FIG. 3E). It is a figure which shows a reference; a vertical axis | shaft). This hip joint angle φ is also one of the walking parameters.

  The middle period of the free leg is a time point when the hip joint angle φ is minimum, as indicated by, for example, the arrow P12 and the arrow P13 in FIG. FIG.10 (b) has shown the picked-up image (photographed image which shows the middle period of a free leg) image | photographed at the time shown by arrow P12, P13 of Fig.10 (a).

  FIG. 11A shows the X coordinate of the greater trochanter and the outer fruit in the photographed image obtained for each photographed image when the identification numbers 1 to 59 of the photographed images for the two walking cycles are set on the horizontal axis. It is a figure which shows difference (DELTA) w (refer FIG.3 (g); vertical axis | shaft) between each other. This difference Δw is also one of the walking parameters.

  Here, a method of calculating the joint angles of the hip joint angle φ, the foot joint angle α, and the knee joint angle β will be described. The calculation of each joint angle is performed by the calculation unit 43.

  As shown in FIG. 16 (a), the hip joint angle φ is a line segment connecting the mark 2 attached to the greater trochanter and the mark 2 attached to the knee, and the mark attached to the greater trochanter. 2 and a line segment connecting the mark 2 attached to the shoulder ridge, the crossing angle of those line segments.

  The calculation unit 43 sets the positions of the marks 2 mounted on the greater trochanter, the shoulder ridge, and the knee as points O, P, Q, a vector OP having the point O as a start point, and a point P as an end point, and a point O A vector OQ having a start point and a point Q as an end point is generated.

Next, the calculation unit 43 calculates the inner product of the vector OP and the vector OQ. That is, assuming that the vector OP = (X ₃ , Y ₃ ) and the vector OQ = (X ₄ , Y ₄ ), the calculation unit 43 calculates the inner product OP × OQ between the vector OP and the vector OQ.
OP × OQ = X ₃ × X ₄ + Y ₃ × Y ₄ (3)
Calculated by

On the other hand, if the smaller angle among the angles formed by the vector OP and the vector OQ is represented by φ and the vector OP and the vector OQ are simply represented by OP and OQ, the inner product OP × OQ is
OP × OQ = | OP || OQ | cosφ (4)
Can also be expressed. Note that | OP | is the length (scalar amount) of the vector OP, and | OQ | is the length (scalar amount) of the vector OQ.

The calculation unit 43 is based on the equations (3) and (4).
X ₃ × X ₄ + Y ₃ × Y ₄ = | OP || OQ | cosφ
Is established,
φ = cos ⁻¹ {(X ₃ × X ₄ + Y ₃ × Y ₄ ) / | OP || OQ |}
To calculate the angle φ.

  As shown in FIG. 16 (b), the ankle joint angle α is defined by a line segment connecting the mark 2 attached to the outer fruit and the mark 2 attached to the toe, and the mark 2 attached to the outer fruit. And a line segment connecting the mark 2 mounted on the knee is an intersection angle between the line segments and can be calculated by a calculation method similar to the calculation method of the hip joint angle φ.

  In other words, the “mark 2 attached to the greater trochanter” in the calculation method of the hip joint angle φ is replaced with the “mark 2 attached to the external fruit”, and the “mark 2 attached to the knee” is attached to the toe. After replacing “mark 2 attached to the shoulder” with “mark 2 attached to the knee”, a vector (external capsule) is calculated in the same manner as the calculation method of the hip joint angle φ. 2 vectors starting from the position) are set, and the inner product of these vectors can be used for calculation.

  As shown in FIG. 16 (c), the knee joint angle β is defined by a line segment connecting the mark 2 attached to the knee and the mark 2 attached to the outer fruit, and the mark 2 attached to the knee. When the line segment connecting the mark 2 attached to the greater trochanter is set, it is the intersection angle of those line segments and is calculated by the same calculation method as the calculation method of the hip joint angle φ and the ankle joint angle α. can do.

  In other words, the “mark 2 attached to the greater trochanter” in the calculation method of the hip joint angle φ is replaced with the “mark 2 attached to the knee” and the “mark 2 attached to the knee” is attached to the outer fruit. After replacing “mark 2 attached to the shoulder” with “mark 2 attached to the greater trochanter”, the vector (knee) is calculated in the same manner as the calculation method of the hip joint angle φ. Can be calculated by setting two vectors starting from the position of (2) and using the inner product of these vectors.

  FIG. 11A shows the X coordinate of the greater trochanter and the outer fruit in the photographed image obtained for each photographed image when the identification numbers 1 to 59 of the photographed images for the two walking cycles are set on the horizontal axis. It is a figure which shows difference (DELTA) w (refer FIG.3 (g); vertical axis | shaft) between each other. This difference is also one of the walking parameters.

  As shown by arrows P14 and P15 in FIG. 11A, the end of the free leg is the time when the difference Δw between the X coordinate of the greater trochanter and the outer fruit in the photographed image becomes maximum. FIG.11 (b) has shown the picked-up image (photographed image which shows the free leg end stage) image | photographed at the time shown by arrow P14, P15 of Fig.11 (a).

  The classification unit 5 classifies the captured images for each walking phase based on the characteristic points (parameter values of the walking parameters) in each walking phase as described above. That is, for example, when the difference ΔY between the outer fruit and toe Y coordinates in a certain captured image falls within the predetermined range, the sorting unit 5 is a captured image indicating a load response period. Is performed for each captured image. And the classification | category part 5 classify | categorizes those picked-up images according to which walk phase of each walk image imaged the walk posture (type of walk phase).

  The display unit 6 displays a photographic image indicating a gait phase designated by an unillustrated input operation unit (corresponding to the first input operation unit) among the photographic images classified by the classification unit 5 according to the type of gait phase. indicate. For example, when an instruction to display a captured image indicating the load response period on the display unit 6 is input by the input operation unit, when there are five captured images indicating the load response period, the display unit 6 For example, five photographed images are displayed as videos (moving images) or displayed side by side as five still images.

  As described above, in the present embodiment, which gait phase each captured image is captured is determined based on the parameter values (differences ΔX, ΔY, α, β, etc.) calculated for the captured image. Since each of the detected images is detected and divided into each walking phase using the detection result, and when the walking phase is designated by the input operation unit, a captured image indicating the walking phase is displayed. The walking motion of the pedestrian can be analyzed in detail for each walking phase.

  In addition, since the walking phase indicated by the captured image is detected using the position (coordinates) of each mark 2 in the captured image, various sensors for detecting the pedestrian's posture as in the prior art are provided. It is not necessary to provide it, and it is possible to avoid or suppress an increase in cost and an increase in the size of the apparatus as compared with the prior art.

  In this case, the following modifications may be employed instead of or in addition to the embodiment.

  [1] An image showing an ideal walking posture in each walking phase is stored in advance in an unillustrated storage unit as an ideal image, and when a captured image is obtained by the imaging unit 3 as shown in FIG. When the display unit 6 displays the captured image and an ideal image of the walking phase indicated by the captured image (an ideal image in which the captured image and the walking phase coincide), the instruction to instruct an appropriate walking operation. A person, a pedestrian, or the like can clearly recognize an improvement point in the walking motion of the pedestrian. FIG. 12 shows a screen displayed in a state where the captured images and ideal images for the stance cycle, the previous swing leg period, and the swing leg initial stage are arranged side by side.

  [2] When the calculation unit 43 is equipped with a function for calculating the pedestrian's stride and the display unit 6 displays the stride calculated by the calculation unit 43, the instructor and the like perform the walking motion of the pedestrian. It can be analyzed from the viewpoint of stride. The stride can be calculated, for example, by taking the difference between the X coordinates of the outer fruits in the photographed image and the photographed image taken immediately before the photographed image.

  FIG. 13 is a diagram showing an example of a display screen that displays the detection result of the pedestrian's stride. In the display screen shown in FIG. 13, the age of the pedestrian is plotted on the horizontal axis and the stride is plotted on the vertical axis. A graph L1 showing the upper limit value of the standard stride range and a graph L2 showing the lower limit value of the standard stride range at each age, and the calculated pedestrian stride and its pedestrian The point W corresponding to the combination with the age of is shown.

  As a result, whether the subject or the pedestrian has a larger or smaller step even if the pedestrian's step is included in the standard range or not. Thus, it is possible to grasp that the walking motion of the pedestrian can be analyzed in detail from the viewpoint of the stride.

  Here, the stride is given as an example of the walking parameter for representing the walking motion of the pedestrian, and the detection result of the stride and the standard value are displayed on the display unit 6. The parameters are not limited to the stride, and for example, walking parameters such as the above-mentioned hip joint angle, ankle joint angle, knee joint angle, and walking speed can be employed. Note that the walking speed is calculated by dividing the stride calculated by taking the difference between the X coordinates of the fruits in two captured images taken successively by the time difference between the capturing timings of the two captured images. This can be calculated.

  [3] An input operation unit (not shown; corresponding to the second input operation unit) for performing input for designating one captured image among a series of captured images is provided. When one image is designated (the designated photographed image is referred to as a designated image), as shown in FIG. 14, the display unit 6 displays a plurality of photographed images within a certain period including the photographing timing of the designated image. A function for displaying an image on the same screen may be mounted on the display unit 6.

  Thereby, compared with the case where only one photographed image can be displayed on the display unit 6, a leader, a pedestrian, and the like can grasp the transition mode of the walking motion in detail, and the walking motion of the pedestrian It can be analyzed in detail. In FIG. 14, a display in which five photographed images of a designated image and four photographed images obtained by photographing two times each immediately before and after the designated image are arranged and displayed in order of photographing. The screen is shown.

DESCRIPTION OF SYMBOLS 1 Walking state display apparatus 2 Mark 3 Imaging part 31 Image processing part 4 Coordinate extraction part 41 Extraction part 42 Coordinate setting part 43 Calculation part 5 Classification part 6 Display part

Claims (14)

One or more marks attached to a predetermined part of the body of the pedestrian,
An imaging unit that repeatedly captures an optical image including a mark worn by a pedestrian in a certain optical axis direction and imaging range, and generates a plurality of captured images;
An extraction unit for extracting the mark for each captured image;
A coordinate setting unit that represents the position of each mark extracted by the extraction unit in coordinates of a predetermined common coordinate system;
Using the coordinates for each mark set by the coordinate setting unit, a calculation unit that calculates a predetermined parameter value for a walking parameter for representing a walking motion for each captured image;
Each of the captured images is detected based on the parameter value to determine which gait phase of the gait phase divided into a plurality of gait phases based on a predetermined condition, and the detection result A section for classifying each captured image for each walking phase using
A walking state display device comprising: a display unit configured to display each captured image using a classification result by the classification unit.   The segmenting unit sets the walking cycle of the pedestrian, which is a unit of a period from the time when a heel of one foot has landed on the walking surface until the next time the heel contacts the walking surface, and the one walking When the cycle is divided into a plurality of gait phases based on a predetermined condition for the gait parameter, which gait phase of the gait motion of each photographic image is calculated for the photographic image. The walking state display device according to claim 1, wherein each of the captured images is detected based on the detected parameter value, and each captured image is classified for each walking phase using the detection result. The predetermined part includes the pedestrian's external fruit,
The coordinate system has an axis extending in the walking direction of the pedestrian as a coordinate axis,
When the plurality of walking phases pay attention to each mark attached to the outer fruit of two photographed images obtained continuously in time, a difference between coordinates in the walking direction of each mark is The walking state display device according to claim 1 or 2, including an initial grounding that is a walking phase that first falls within a predetermined range within one walking cycle. The predetermined part includes the pedestrian's external fruit and toes,
The coordinate system has an axis extending in the vertical direction as a coordinate axis,
In the plurality of walking phases, the difference between the coordinates in the vertical direction between the mark mounted on the outer fruit and the mark mounted on the toe of the same photographed image is initially within a predetermined range within one walking cycle. The walking state display device according to claim 1, comprising an inside load response period. The predetermined site includes the greater trochanter and knee of the pedestrian,
The coordinate system has an axis extending in the walking direction of the pedestrian as a coordinate axis,
In the plurality of walking phases, the difference between the coordinates in the walking direction with respect to the mark attached to the greater trochanter and the mark attached to the knee of the same photographed image becomes zero within one walking cycle. The walking state display device according to claim 1 or 2, including a middle stance. The predetermined site includes the greater trochanter and knee of the pedestrian,
The coordinate system has an axis extending in the walking direction of the pedestrian as a coordinate axis,
In the plurality of walking phases, the difference between the coordinates in the walking direction for the mark attached to the greater trochanter and the mark attached to the knee of the same photographed image is maximum within one walking cycle. The gait state display device according to claim 1 or 2, comprising a stance end stage. The predetermined part includes the pedestrian's knees, external fruit and toes,
The plurality of gait phases are attached to a line segment connecting a mark attached to the outer fruit and a mark attached to the toe, a mark attached to the outer fruit and the knee in the same photographed image. The walking according to claim 1 or 2, including a front swing leg period in which an ankle joint angle, which is an intersection angle of the line segments, is maximized within one walking cycle when a line segment connecting the two marks is set. Status display device. The predetermined sites include the trochanter's greater trochanter, knees, and external fruit,
The plurality of gait phases are attached to the line segment connecting the mark attached to the knee and the mark attached to the outer fruit, the mark attached to the knee and the greater trochanter in the same photographed image. 3. The walking according to claim 1, wherein when a line segment connecting the marked mark is set, the walking includes an initial stage of a free leg in which a knee joint angle that is an intersection angle of the line segment becomes a minimum within one walking cycle. Status display device. The predetermined site includes the pedestrian's acromion, greater trochanter and knee,
In the same captured image, the plurality of gait phases include a line segment connecting a mark attached to the greater trochanter and a mark attached to the knee, a mark attached to the greater trochanter, and the shoulder ridge. When the line segment which connects with the mark with which it was equipped with is set, the hip joint angle which is the intersection angle of those line segments contains the middle part of the free leg which becomes the minimum within one walk cycle. Walking state display device. The predetermined site includes the trochanter's greater trochanter and outer fruit,
The coordinate system has an axis extending in the walking direction of the pedestrian as a coordinate axis,
In the plurality of walking phases, the difference between the coordinates in the walking direction between the mark attached to the greater trochanter and the mark attached to the outer fruit of the same photographed image is maximum within one walking cycle. The walking state display device according to claim 1, comprising a swing leg end stage. A first input operation unit for performing input to select a captured image to be displayed on the display unit for each walking phase;
The walking state display device according to any one of claims 1 to 10, wherein the display unit displays a captured image of a walking phase selected by the first input operation unit. An ideal image storage unit that stores an ideal image having an ideal parameter value provided in advance for each walking phase;
The walking state display device according to any one of claims 1 to 11, wherein the display unit further has a function of displaying the captured image and an ideal image in which the captured image and the walking phase coincide with each other. A standard value storage unit that stores a standard value for the walking parameter that is set in advance based on a combination of age and walking phase,
The walking state display device according to any one of claims 1 to 12, wherein the display unit further has a function of displaying the parameter value derived by the calculation unit and the standard value in a graph. A second input operation unit for performing input for selecting one photographed image from the photographed images displayed on the display unit;
The display unit further displays a plurality of photographed images obtained by an imaging operation within a certain period including a photographing timing of the photographed image on the same screen when one photographed image is selected by the second input operation unit. The walking state display device according to any one of claims 1 to 13, which has a function of displaying on the screen.