JP2020042734A - Visibility evaluation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a visibility evaluation system for quantitatively evaluating comprehensive visibility including a confirmation operation for a design element. A visibility evaluation system is stored in a biomechanics model 10 and a database 8 for simulating a confirmation operation with respect to the visual target point in a virtual 3D space in which an object and its visual target point 20 are defined. It includes a visual field determination model 50 for calculating the visibility 15 with respect to the visual field target point in the visual field image in the line of sight of the biomechanics model, and a general control unit 11 for coupling the biomechanics model and the visual field determination model. The integrated control unit gives a perturbation to the biomechanics model, calculates the visibility 15 with respect to the visual recognition target point in that posture, and calculates the burden 16 of the biomechanics model due to the perturbation, and the minimum burden and the maximum visibility. The operation of generating the posture is executed according to the behavior that becomes, and the easy visibility is calculated from the integrated value of the burden degree and the final value of the visibility degree. [Selection diagram] Fig. 1

Description

  The present invention relates to a 3D CAD visibility evaluation system using a biomechanical model.

  Conventionally, in 3D CAD product design, the evaluation of visual effects such as visibility and the evaluation of practical effects such as operability and ease of handling have been performed separately. For example, Patent Literature 1 discloses a design support apparatus that visualizes design data of a cockpit of a vehicle by 3DCG and evaluates usability and visibility. However, such an evaluation method cannot eliminate the influence of the subjectivity of the tester, and it is difficult to obtain a quantitative evaluation.

  On the other hand, Patent Literature 2 discloses a design support system that simulates an entry / exit operation with respect to a virtual vehicle using a human body model and evaluates the entry / exit performance in order to determine design specification values around a door portion of the vehicle. ing. Although this system is aimed at quantitative evaluation of getting on and off, it does not describe or suggest visibility or visual effect.

JP 2013-189055 A JP 2013-246663 A

  In product design, visual effects such as visibility may be related to practical effects such as operability and accessibility. For example, in the design of the cockpit of a vehicle, if all the operation units and display units cannot be arranged in the field of view due to space constraints, elements with a high operation frequency or viewing frequency or elements that require visual recognition while driving are in the field of view. The other elements are arranged outside the field of view or on a hidden surface hidden by other members. Such an element is visually recognized and operated with a confirmation operation performed by the occupant by changing the face direction or tilting the head or the upper body. Therefore, it is preferable that the visibility is quantitatively evaluated including the ease of such a checking operation.

  SUMMARY OF THE INVENTION The present invention has been made in view of the above situation, and an object of the present invention is to perform visibility evaluation for performing quantitative evaluation of visibility including a confirmation operation on a design element in product design by 3D CAD. It is to provide a system.

In order to solve the above problems, the present invention provides:
A visibility evaluation system for an object by 3D CAD,
A biomechanical model for simulating a confirmation operation on the visual target in the virtual 3D space in which the object and the visual target are defined;
A visual field determination model for calculating the visibility of the visual target in the visual field image in the line of sight of the biomechanical model,
A behavior in which a perturbation is given to the biomechanical model, the degree of visibility of the biomechanical model associated with the perturbation is calculated in the posture, and the degree of burden of the biomechanical model is calculated in accordance with the perturbation. An overall control unit that couples the biomechanical model and the visual field determination model by performing an operation of generating a posture at
And a visibility evaluation system configured to calculate easy visibility from the integrated value of the burden and the final value of the visibility.

  As described above, the visibility evaluation system according to the present invention simulates a confirmation operation with respect to a visual target point defined in the virtual 3D space using a biomechanical model, and uses a visual field image in the line of sight of the biomechanical model to perform visual recognition. Along with calculating the visibility for the points, the burden of the biomechanical model required for the confirmation operation is calculated, and the visibility is evaluated from the integrated value of the burden and the final value of the visibility. Quantitative evaluation of visibility including the above can be performed.

  Moreover, in the simulation of the confirmation operation using the biomechanical model, a perturbation is given to the biomechanical model, the visibility with respect to the visual target point is calculated in the posture, and the burden of the biomechanical model accompanying the perturbation is calculated. Is performed by an operation that generates a posture with a degree of visibility and maximum visibility, so that the confirmation operation is simulated autonomously by coupling the visual field judgment model and the biomechanical model, and reliable quantitative evaluation It can be performed.

  In a preferred aspect of the present invention, the biomechanical model includes a musculoskeletal model that models a human skeleton, joints, and skeletal muscles, and an eyeball that determines an eyeball posture in conjunction with a head movement in the musculoskeletal model. Since it is configured to include the motion model and to give the line of sight according to the eye posture, there is an advantage that the simulation of the confirmation operation including the eye movement in addition to the physical movement and the evaluation of the burden can be performed.

  In a preferred aspect of the present invention, the visual field determination model is a transparent solid body in which the viewing target point is radially enlarged in a virtual 3D space, and only a specific pixel information value is determined according to a distance from the viewing target point. Since the reduced or gradually reduced gradation solid is defined and the total visibility of the specific pixel information values in the visual field image is calculated as the visibility, an advantage that the visibility can be quantified and quantitatively evaluated. There is.

  In particular, even for a visual target point at a position that cannot be directly seen in the initial posture, the visibility including the state approaching the visible posture can be quantified and fed back to the motion command generation of the biomechanical model, and the visual field judgment An accurate coupling between the model and the biomechanical model becomes possible.

It is a block diagram showing the visibility evaluation system concerning the embodiment of the present invention. It is a typical side view showing the simulation by the visibility evaluation system concerning the embodiment of the present invention. It is a schematic plan view showing a simulation by a visibility evaluation system according to an embodiment of the present invention. (A) is a schematic diagram showing a musculoskeletal model, and (b) is a diagram showing a contact point and a contact plane in a sitting state of the musculoskeletal model. FIG. 3 is an ER diagram illustrating a data configuration used in the visibility evaluation system according to the embodiment of the present invention. It is a block diagram showing an eye movement model. It is a block diagram which shows a vestibular oculomotor reflection model. It is a block diagram which shows a tracking eyeball movement model. It is a block diagram which shows a saccade model. It is a figure showing an effective eyeball movement range in a saccade model. It is a block diagram which shows integration of an eyeball movement model and a musculoskeletal model. It is a flowchart which shows the process of the visibility evaluation system which concerns on embodiment of this invention. It is the flowchart which divided the process of the visibility evaluation system which concerns on embodiment of this invention into a biomechanics model and a visual field judgment model.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1. Overview of Evaluation System FIG. 1 is a block diagram showing a visibility evaluation system 1 according to an embodiment of the present invention. The evaluation system 1 of the present embodiment uses 3D CAD data of an instrument panel disposed in front of a cockpit of a vehicle as evaluation target data 2 and uses a biomechanical model 3 as shown in FIGS. 2 and 3. The simulation of the confirmation operation for visually recognizing the visual recognition target point 20 is performed, and the visibility including its easiness is evaluated.

  The evaluation system 1 includes a biomechanical model 10 and a visual field judgment model 50, and a program for each operation exists separately. Each program, data, and an operation program for operating the programs collectively are stored in an external storage device or the like of a computer (not shown), are read into a main memory (RAM), and arithmetic processing is executed by the CPU. Thus, it is configured to function as an evaluation system.

  The biomechanical model 10 is composed of a musculoskeletal model 3 and an eyeball movement model 4. By integrating these, a body movement and an eyeball movement for performing visual field judgment are generated, and a burden 16 Is calculated, and the attitude data of the eyeball 5 obtained as a result of each exercise is reflected on the camera 55 in the visual field judgment model 50 to calculate the visibility 15 with respect to the viewing target point 20. Based on the burden 16 and the visibility 15, the visibility (easy visibility) including the ease of the checking operation is evaluated.

  The visibility 15 calculated by the determination processing unit 51 of the visual field determination model 50 is fed back to the general control unit 11 of the biomechanical model 10 and a part of the exercise reference potential 14 that causes the biomechanical model 10 to perform a confirmation operation. Become. As described above, the biomechanical model 10 and the visual field judgment model 50 operate independently of each other and are coupled by sharing or exchanging data with each other, minimizing the burden 16 in the biomechanical model 10, A simulation for simultaneously maximizing the visibility 15 in the visual field judgment model 50 can be performed.

  That is, in the two models, the optimal operation for reducing the burden 16 in the biomechanical model 10 leads to the improvement of the visibility 15 in the visual field judgment model 50, and the improvement in the visibility 15 in the visual field judgment model 50 is There is a coupled relationship of providing an optimum operation for reducing the burden 16 on the device 10. The mode of sharing or exchanging data between the two models is performed by accessing the database 8 as shown in FIG. 5, but other than that, an arbitrary mode such as data communication or transfer of a data file by file operation is used. It can be implemented in.

  The database 8 is for acquiring simulation data 80 for storing conditions, results, and the like in each simulation (trial), behavior data 81 for storing data of each behavior process in each simulation (trial), and acquiring each behavior process. Includes a perturbation data 82 for storing data of a perturbation process virtually performed, and a one-to-many relation between a primary key (PK) of each data table and a foreign key (FK) of a corresponding data table. It is configured as a set relational database.

  The simulation data 80 stores input conditions for each simulation (trial). For example, in the illustrated example, the support structure information (first support point (hip point; HP) on the seat 6) and the second support point (left and right grip points; coordinate ( (x, y, z)), physique setting information (height, weight) of the person to be simulated, CAD data (name, identification code) to be evaluated, coordinates of target recognition point (coordinates (x, y, z)), etc. are input. Then, the visibility evaluation and the rendering information as a result of the simulation process are stored.

  Although not shown, basically, basic information such as support structure information (cockpit CAD data), physique setting information, CAD data to be evaluated, and rendering data are registered in separate data tables, respectively, and identification codes are stored. (Foreign key) to be referenced by the relation set. When the simulation is performed by transferring the data file, a data file of an appropriate format corresponding to the simulation data 80, the behavior data 81, and the perturbation data 82 is formed, and the data is additionally updated. After the behavior data 81 in each behavior process is acquired, the setting may be such that the perturbation data 82 required for the behavior data 81 is not retained.

As will be described in detail later, the behavior of the musculoskeletal model 3 and the eye movement model 4 in the biomechanical model 10 for the behavior data 81 and the perturbation data 82 are virtually equivalent to the number of state variables u in order to determine the optimal behavior. And the behavioral process of actually operating for a unit time under the conditions that result in the minimum burden (B _min ) and the maximum visibility (R _max ). The visual field determination model 50 calculates the visibility R for each perturbation process.

  Therefore, first, the behavior No. (ID) in the behavior data 81 is assigned by automatic numbering or the like, and then, with respect to the behavior No., a perturbation No for the state variable U is assigned to the perturbation data 82, and for each state variable U Then, the musculoskeletal model 3 and the eye movement model 4 are perturbed. The skeleton link posture, the eyeball posture, and the burden B obtained as a result of each perturbation are stored in the perturbation data 82, and the eyeball posture is reflected on the camera 55 of the visual field determination model 50, and the visibility R calculated from the visual field image is obtained. Is stored in the perturbation data 82.

Next, the conditions that result in the minimum burden (B _min ) and the maximum visibility (R _max ) are stored in the behavior data 81, the behavior process is performed in accordance therewith, and the skeleton link posture and the eyeball posture are registered as key frame information. . By accumulating such key frame information, it is possible to generate a 3D CG moving image (and a 3D CG moving image viewed from an arbitrarily set viewpoint) of the visual field image corresponding to the confirmation operation. Further, along with the execution of the behavior process, the integrated values of the minimum burden (B _min ) and the maximum visibility (R _max ) acquired in the previous behavior process are updated and stored in the behavior data 81. These integrated values serve as basic information for the final easy-visibility evaluation in the simulation.

2. Details of visual field judgment model The visual field judgment model 50 is a 3D coordinate system (virtual data space), a human body corresponding to the evaluation object data 2 having the visual target 20 and a biomechanical model (musculoskeletal model 3, eyeball movement model 4). The model 53 and the camera 55 are defined, and the body motion and the eyeball movement (body posture and eyeball posture) generated by the biomechanical model are reflected on the human body model 53 and the camera 55, and the visual target point 20 in that state is reflected. This is for acquiring visibility.

  This camera 55 is a viewpoint (corresponding to a human eye) set in a 3D coordinate system, and a camera image is a field of view of a human (human body model 53). By registering the posture of the human body model 53 in a key frame every moment, a moving image can be finally created. If you set an external viewpoint, you can also visually observe your body behavior.

  Hereinafter, the visibility calculation process by the visual field determination model 50 will be described. Note that all the processes are automatically performed by a program. Since it is based on image processing in the 3D data space, a general-purpose 3DCG application can be used.

  As described above, the evaluation system 1 of the present embodiment uses 3D CAD data of an instrument panel provided in front of a cockpit of a vehicle as evaluation target data 2 and corresponds to a driver as shown in FIGS. 2 and 3. A simulation of a confirmation operation for visually recognizing the visual recognition target point 20 is performed using the biomechanical model 3 to evaluate the visibility including its easiness, and the visual field judgment model 50 performs the confirmation operation (perturbation). And behavior) with respect to the viewing target point 20 at each time point.

  In the illustrated example, a switch located on the lower right side of the instrument panel 2 is assumed as the visual recognition target point 20. This switch is located at a position hidden by the handlebar 7 and is difficult to directly recognize from the viewpoint (5) of the driver (3) sitting on the seat 6, and as shown in FIG. The confirmation operation involving the movement and the rotation of the head gradually makes it visually recognizable.

  Therefore, in order to obtain the visibility of the viewing target point 20 that is not directly visible at the initial position or is difficult to visually recognize, the gradation sphere 21 is defined as a solid body obtained by radially expanding the viewing target point 20. This makes it possible to calculate the visibility by at least partially entering the gradation sphere 21 into the visual field image.

  Furthermore, in order to weight the visibility at the center of the gradation sphere 21, that is, in the vicinity of the viewing target point 20 and at the periphery, only specific pixel information is gradually reduced according to the distance from the center of the gradation sphere 21. In the present embodiment, the R value indicating the intensity (luminance) of red is used as specific pixel information, and the gradation sphere 21 has a radial direction in which red becomes darker on the center side and red becomes lighter on the peripheral side. Has gradation.

The pixel information used in the visual field determination model 50 is based on an “RGBA” color model in which an alpha channel (A) of transparency is added to three primary colors of red (R), green (G), and blue (B). RBGA is a numerical value from 0 to 1.0, and is represented by [R, B, G, A]. In the gradation sphere 21 installed at the viewing target point 20, the R value gradually decreases between the center R value = 1.0 (maximum) and the surface R value = 0 (minimum), and the G value and the B value are 0, the A value is 1.0 because the whole is based on a transparent sphere, so the RGBA of the gradation sphere 21 is
[R, G, B, A] = [0 to 1.0, 0, 0, 1.0]
Will be expressed in the range.

  When calculating the visibility in the visual field judgment model 50, all solids (evaluation data 2, the handle 7, the sheet 6, the human model 53, etc.) other than the gradation sphere 21 are black and opaque (do not reflect the light source, [R, G, B, A] = [0, 0, 0, 0]), the view image of the camera 55 is in a state where only the gradation sphere 21 partially hidden by a black solid (such as the handle 7) is visible. Has become.

  In this state, by taking the sum of the R values of all the pixels of the image of the field of view of the camera 55, it is possible to quantify how much the red gradation sphere 21 can be seen, and to visually recognize the viewing target point 20 based on that. You can judge the degree.

  Since the visibility (total R value) uses the gradation sphere 21, it can be extended and quantified to the case where the viewing target point 20 is not directly visible, and is approaching the state where it is not directly visible but visible. Since the situation can be acquired as an improvement in the visibility, it is optimal for evaluation of the visibility (easy visibility) including the confirmation operation, and is fed back to the biomechanical model 10 as will be described later in detail. It is also significant in that it becomes a part of the exercise reference potential 14 that directs the movement (confirmation operation) of the movement.

  Note that as specific pixel information of the gradation sphere 21 used in the visual field determination model 50, luminance of green or blue other than red can be used, and information other than luminance can also be used. For example, red dots may be defined inside the transparent gradation sphere at a density corresponding to the distance from the center, and the R values may be summed.

3. Details of Biomechanical Model The biomechanical model 10 includes a musculoskeletal model 3 and an eye movement model 4. Basically, a body movement is generated in the musculoskeletal model 3, and the eye movement is coupled with the body movement. Be evoked. Therefore, first, a flow of generating the body motion for the operation of confirming the visual recognition target point 20 in the musculoskeletal model 3 will be described.

3.1 Definition of Motion Representative Point It is preferable that the musculoskeletal model 3 is biomechanically valid in reproducing various body motions, is simple, and has a low calculation cost. Therefore, a motion representative point is set for each part of the body so that the motion trajectory of each motion representative point can be defined. For example, since the movement is a body movement for the confirmation operation of the viewing target point 20, the eyeball coordinates of the head 31 (the middle coordinates of the left and right eyes 5 or the center coordinates of the head 31) are defined as the movement representative points. .

  The three-dimensional position of the motion representative point is described by a spatial coordinate system, and the body motion is defined as a movement from the initial position of the motion representative point to the target (end) position. This coordinate position can also be defined as a relative coordinate value based on another motion representative point. For example, as shown in FIG. 3, when the center coordinate of the head 31 is set as the movement representative point, the confirmation operation can be defined as the movement of this point, and the eyeball table is defined based on the movement representative point. You can also.

3.2 Inverse model-based motion command generation In the case of an ideal trajectory given in body motion, a calculation theory is proposed in which a motion command for realizing this motion is obtained by a module corresponding to an inverse model of body dynamics. (Wang, et al., 2001). Also in the present embodiment, a motion command (joint driving moment) is generated by considering an inverse model of the body dynamics.

In the study of computational theory models of the brain, it may be debated how to obtain the model of the body dynamics itself, but here we can calculate the body dynamics to avoid complexity Assuming that the module already exists in the brain, the module uses the inverse dynamics calculation routine of the body dynamics model. Assuming that the movement (joint) displacement of the body such as the joint angle is q, the inverse model of the body dynamics system is given by the following equation (3.1).
Here, M is an inertia matrix, h () is a vector representing the influence of Coriolis force, centrifugal force, and external force, f _ext is an external force acting on the body, and n is a joint driving moment by a muscle.

  3, in the initial posture of the musculoskeletal model 3, the head 31 is directed forward in the traveling direction of the vehicle, and the line of sight of the eyeball 5 is also directed forward as indicated by reference numeral 5a in the figure. In this initial posture, the visual recognition target point 20 is outside the central visual field and is hidden by the handle 7, so that it is not possible to directly recognize the visual recognition target point 20. As shown in FIG. 7, the driver turns his / her gaze in that direction or rotates the head 31.

  These operations are related to the eye movement model 4, which will be described later. However, basically, the driver moves the head 31 in the direction of the viewing target point 20 so that the driver can move the head 31 toward the viewing target point 20. Get the idea that visibility will improve and take action. Instead of making the mechanism of such a movement command inherent in the biomechanical model 10 itself, in the present invention, the mechanism is obtained by coupling with the visual field judgment model 50.

  That is, as described above, the approximate azimuth of the visual recognition target point 20 in the visual field image can be acquired by the gradation sphere 21 obtained by radially expanding the visual recognition target point 20, and the deviation between the azimuth and the gaze direction 5a in the initial posture is obtained. , The direction of the initial operation as shown by the dashed arrow in FIG. 3 can be obtained.

  Furthermore, since the movement of the driver in the confirmation operation is basically in a lateral direction (y direction) and / or a vertical direction (z direction) substantially perpendicular to the front of the vehicle (x direction), the movement of the driver with respect to the gaze direction 5a in the initial posture is performed. Assuming the displacement of the displacement on the yz plane or the y-direction component as the trajectory of the body movement in the confirmation operation, in order to obtain a joint driving moment capable of realizing this trajectory, the trajectory and the current body posture Consider a virtual force corresponding to the difference between the two and consider this a kind of external force acting on the internal inverse model. This is called willpower.

The will power f _int is obtained by the following equation (3.2).
Here, P _cur is the position coordinate of the motion representative point at the present time, P _des is the position coordinate of the motion representative point obtained by the trajectory planning of the body motion in the confirmation operation, and A _int is a weight matrix (diagonal matrix). The importance is determined according to the importance of the plurality of exercise representative points and the importance of the direction of the exercise. k ₁ and k ₂ are coefficients for determining the weights of the position information and the speed information.

If this intentional force is treated as an external force equivalent to a reaction force acting on another body and given to an inverse model as in the following equation (3.3), a joint driving moment capable of realizing the trajectory of the body movement in the confirmation operation n (f _int ) is obtained.

  However, the intentional trajectory and the target trajectory of the physical motion based on the will power do not always strictly define the motion to be realized. For example, when a plurality of motion representative points are defined, even if there is some inconsistency in the mutual motion, the definition of the physical motion is facilitated by allowing it. In the first place, willpower is a virtual force that does not actually act on the body, and therefore does not usually satisfy the mechanical balance.

  There is also a problem in that exercise generation based on biomechanical load such as minimization of muscle load has not been realized. Furthermore, when the inverse model is simply calculated, not only the degrees of freedom of the joints but also the driving force / moment for the degrees of freedom between the body and the space coordinate system are calculated.

  On the other hand, a model that autonomously acquires the natural movement of a living body by considering the dynamics of the living body and body as some potential and considering the dynamics to minimize it has been proposed (Umedachi, T., et al., 2010). Therefore, in the biomechanical model 10 according to the present embodiment, a potential that can be a norm of body movement as described below is defined, and a mechanism for minimizing the potential is introduced.

That is, as shown in the following formulas (3.4) to (3.6), the state variable u corresponding to the will power is set as the state variable of the motion reference potential U _inv , The dynamics for obtaining the state variable u that reduces the sum of the driving moments is defined.

Here, n (u) is a joint driving moment obtained when a state variable u corresponding to a willing force is applied to the inverse model. k ₃ and k ₄ are coefficients, and A ₁ is also a weight coefficient matrix. This is a weight coefficient corresponding to the joint driving moment and driving force corresponding to the degree of freedom between the spatial coordinate system and the musculoskeletal model 3. I made it bigger. Further, since it is difficult to analytically calculate the potential gradient (inU _inv / ∂u) in the equation (3.4), in the actual calculation, it is obtained by a perturbed difference equation.

The above is only the body movement in the musculoskeletal model 3, and does not consider the coupling with the eye movement model 4. When considering the coupling with the eye movement, as shown in the following equation (3.7), the eye movement load f _eye is added to the movement potential U _inv in the body movement, and this may be newly set as the movement reference potential. .
The eye movement load f _eye is obtained from an eye movement model 4 described later. This enables coupling between the musculoskeletal model 3 and the eye movement model 4 and generation of a movement that reduces both the physical load and the eye movement load.

  In the control of a robot such as an articulated manipulator, a method of using a Jacobi matrix or its pseudo inverse matrix to convert between task space coordinates and joint (configuration) space coordinates is generally performed. It has a high degree of mathematical abstraction as a biological control model. On the other hand, in this model, the joint driving force can be obtained directly from the motion trajectory in the task space by assuming the inverse model of the body dynamics system and the will power, and the advanced and abstract mathematical No processing required. In addition, although there is some ambiguity in how to determine will power and how to determine its coefficient, there is an advantage that it is easy to generate motion of a large-scale model with multiple degrees of freedom such as a whole body model.

3.3 Motion Command Adjustment Based on Forward Model Based on the above-described inverse model based motion command generation module 12, it is possible to generate a motion according to a trajectory plan and with a small muscle burden. However, the musculoskeletal model 3 has many degrees of freedom and is complicated. In addition, it is difficult to achieve both trajectory planning and biomechanically valid motion by itself, because it uses the willpower that may cause a mechanical inconsistency with the inverse model-based calculation.

  Therefore, the forward model of body dynamics is considered as a model of the movement control, and a forward model-based movement control mechanism is added. By using the forward model, foreseeable motion control with the time axis advanced to some extent can be performed, and a skillful movement of a living body can be reproduced.

Here, a state variable v corresponding to the joint driving moment is assumed, a motion reference potential U _fwd (v) is defined by the state variable v, and the dynamics for reducing this is defined as the following equation (3.8).
Here, n (u) is a joint driving moment obtained by the inverse model, and k ₅ and k ₆ are coefficients.

The motion reference potential U _fwd is obtained by a forward model as follows. First, when the state variable v corresponding to the joint driving moment is given, the acceleration at the present time (time t) can be estimated by the forward model as in the following equation (3.9).

From this acceleration, the motion displacement and velocity of the body at time t + Δt can be estimated by simple time integration as in the following equations (3.10) and (3.11).

From these, the velocity and position of the motion representative point at time t + Δt can be estimated as in the following equations (3.12) and (3.13).

The above f ₁ and f ₂ are functions for obtaining the position and speed of the motion representative point from the joint displacement and speed using the forward kinematics model of the body structure. The motion reference potential U _fwd is defined by the following equation (3.14) based on the difference between the predicted motion representative point position P _pre obtained from the forward model and the trajectory position P _des in the trajectory planning.

Here, A ₂ and A ₃ are weight coefficient matrices. The above equation (3.14) does not consider the coupling with the eye movement as in the aforementioned equation (3.7). Therefore, by adding the visibility (R value) output from the visual field judgment model to the above equation (3.14), the motion reference potential that realizes the coupling between the body movement and the eye movement is expressed by the following equation (3.14). .15).

  Although the R value increases as the visibility of the target increases, the R value takes the reciprocal to minimize the motion reference potential.

  Considering the dynamics of the neural control system as described in the above formulas (3.4) and (3.8), the autonomous motion objectives such as the achievement of the target motion trajectory and the minimization of the muscle burden are considered. In order to achieve this. Minimizing muscle strain is an optimization problem, and its solution generally requires iterative search calculations. By changing the control variable over time in the direction of decreasing the objective function expressed as a potential as in the above equation (3.15), it is possible to achieve both the generation of the motion and the purpose of the motion autonomously without repeatedly calculating. it can.

  On the other hand, the above-described equations (3.4) and (3.8) have a first-order lag characteristic as a control system, and a time delay occurs. The forward model has the function of compensating for this time delay, but autonomously balances opposing motion goals, such as achieving the target motion trajectory and minimizing muscle strain, which are difficult to achieve with the inverse model alone. It also has the function to achieve. In other words, by resolving the dynamic contradiction that cannot be solved by the inverse model by the forward model, there is an advantage that it is possible to easily achieve the reciprocal motion objectives.

The joint driving moment n _real finally acting on the body dynamics system is obtained by adding a damping term to the state variable v of the forward model of the above-mentioned equation (3.8) in consideration of compensation of calculation convergence and joint impedance. Thus, the following equation (3.16) is obtained.
Here, k ₇ is a coefficient. This attenuation term is based on a neural control mechanism as a body model, and a structural attenuation characteristic of the soft tissue of the joint is separately defined by a musculoskeletal model 3 described below.

3.4 Musculoskeletal model (body mechanics model)
The body motion is generated by giving the joint driving moment n _real obtained by the above motion control model to the musculoskeletal model 3 and performing forward dynamics calculation. The musculoskeletal model 3 is an extension of the existing walking simulation model (Hase and Yamazaki, 2002), and is a three-dimensional model having 20 links and 43 joint degrees of freedom for the whole body as shown in FIG.

  That is, the musculoskeletal model 3 includes a head 31, a cervical vertebra 32, a chest (upper thoracic vertebra) 33a, a lower thoracic vertebra 33b, a lumbar vertebra 34, a pelvis 35, a thigh 36a, a lower leg 36b, a foot 36c, Of the clavicle 37a, upper arm 37b, left and right arms 37c, and hand 37d. Each joint is schematically shown by one to three cylinders (rotational axes). For example, the elbow joint between the upper arm 37b and the forearm 37c has one degree of freedom of bending around one cylinder, and the wrist joint between the forearm 37c and the hand part 37d has a combination of three cylinders, whereby the front and rear and inside and outside bending and screwing are performed. The three degrees of freedom are displayed.

  In the embodiment, the values of the body parameters such as the mass and the moment of inertia of each body segment are assumed to be those of a general adult male as a standard, but are based on the height and weight of the input data (simulation data). , The body parameters can be scaled. In addition, a passive resistance moment due to a joint structure such as soft joint tissue acts on each joint. This is defined by a nonlinear elastic property function (Yamazaki et al., 2006) and a linear viscous damping.

3.5 Definition of Contact Force The body and the sheet 6 basically come into surface contact. The skin tissue of the body has non-linear viscoelastic properties, and the seat 6 is provided with a cushion material on both the seat cushion 61 and the seat back 62, and has viscoelastic properties. Although a calculation method such as the finite element method is desirable for accurately modeling these characteristics, the calculation cost is large and a large number of physical characteristic values are required.

  Therefore, in the present embodiment, in order to more easily express the contact force, surface contact and distributed contact are not assumed, and a plurality of body segments (32 to 36a) as shown in FIG. It is assumed that a contact force acts on a contact point. Here, first, the shapes of the seat cushion 61 (seat surface) and the seat back 62 are represented by a plurality of planes, and are defined as contact planes (61a, 61b, 62a, 62b, 62c). If the contact point defined on the body segment sinks below the contact plane, a viscoelastic element is assumed and a reaction force corresponding to the contact point displacement and the contact point velocity is returned. Also, assuming friction, if the tangential component of the contact surface exceeds the frictional force, slippage occurs.

Specifically, the contact surface coordinate system is defined as the origin on the contact surface, the x and y axes in the tangential direction, and the z axis in the normal direction (positive in the spatial direction). .17) (3.18) defines the contact force.

Here, the temporary component force, k _s tangential modulus of formula (3.17) is tangential (x or y), c _s viscosity coefficient, [s] tangential coordinates (x or y), [s _0] is the reference point of the elastic element, it holds first the position coordinates contact point is in contact with the contact surface as [s _0].

F _z is the component force in the normal direction, max ₀ (x) is a function that returns zero when x is negative and returns the value as it is when x is positive, k _{z and cz} are the viscoelastic coefficients in the normal direction, e nonlinear coefficient, [z] are the normal _{coordinates, step (x, x 0,} y 0, x 1, y 1) is step function is approximated by a cubic curve in order to avoid discontinuities, i.e., the variable x Returns y ₀ if x <x ₀ , returns y ₁ if x ₁ <x, and if x ₀ ≦ x ≦ x ₁ , interpolates between y ₀ and y _{1 with} a cubic curve and returns a value corresponding to it return it. This is a device for changing the viscosity coefficient depending on the amount of penetration of the contact point. That is, the viscosity coefficient when the [z] = 0 in formula (3.18) becomes zero, than a specified amount of intrusion d [-z]> d the viscosity coefficient is c _z.

  Several typical models for determining the frictional force generated in the tangential direction are known.However, in the present embodiment, a complicated friction model is not considered in order to easily reproduce the body behavior, The friction is expressed by updating the reference point position of the tangential elastic element.

That is, when the absolute value of the provisional tangential component obtained by the equation (3.17) is larger than the frictional force, the friction is kinetic friction, and the tangential component is expressed by the following equation (3.19).
Here, μ is a friction coefficient, and sgn (x) is a function that returns −1 when x is negative and +1 when x is positive.

When the absolute value of the provisional tangential component force obtained by the equation (3.17) is equal to or less than the frictional force, the apparatus is in a stationary state, and is determined by the viscoelastic element as shown by the following equation (3.21). The temporary tangential component becomes the tangential component as it is.

  As shown in FIG. 4B, 24 contact points for the whole body are defined as the contact point positions that come into contact with the sheet 6 in consideration of the body shape and the contact state with the sheet 6. Each contact point position coordinate is described as a node coordinate on the body segment (32 to 36a) and is fixed on the node coordinate. In addition, the contact point is also defined as the contact between the foot 36c and the floor surface, and the hand 37d and the grip of the handle 7.

  To summarize the above, first, in the inverse model-based motion command generation module 12, a virtual force corresponding to the difference between the target posture and the current posture is considered, and this is defined as the will force acting on the internal inverse model. . If this intentional force is treated as an external force equivalent to a reaction force acting on another body and applied to an inverse model, a joint driving moment for realizing the target posture can be obtained. A perturbation is applied to this will, and a joint driving moment is calculated from the will and the external force acting on the body by inverse dynamics calculation. This is repeated for the number of state variables.

  Next, in the forward model-based motion control module 13, a perturbation is applied to the joint driving moment, and the current acceleration is calculated by forward dynamics calculation. The posture after Δt seconds (for example, 0.1 seconds) is estimated by integrating the acceleration with time. The difference between the visual field evaluation and the target posture is calculated from the estimated posture. This is repeated for the number of state variables. Thereby, the joint driving moment for minimizing the physical load and maximizing the visibility is determined.

4. Eye Movement Model As shown in FIG. 6, the biomechanical model 10 includes three models: a vestibular oculomotor reflex model (VOR) 41, a tracking eye movement model (Smooth Pursuit) 42, and a saccade model (Saccade) 43. The eyeball movement models 4 are incorporated, and the eyeballs 5 are moved by complementing each other. Hereinafter, each eye movement model will be described with reference to the drawings.

4.1 Vestibular oculomotor reflex (VOR)
The vestibular oculomotor reflex is a reflective eye movement that functions to keep the direction of the line of sight 5b constant when the head 31 or the body 30 is exercising. That is, when the head 31 rotates, the eyeball 5 smoothly moves in a direction opposite to the rotation direction so as to cancel the rotation. A known model can be used for modeling the vestibular oculomotor reflex. In the present embodiment, a vestibular oculomotor reflection model 41 of Merfeld and Haslwanter et al. As shown in FIG. 7 is employed.

  The vestibular oculomotor reflex model 41 can be roughly divided into I: Movement in space, II: Transmission by otoliths and semicircular canals (Transduction by otoliths and canals), III: Internal processing, IV: Eyeball It consists of four phases of eye movement generation. The details of each phase are shown below.

I: Movement in space
This phase is a physical stimulus to the head 31, that is, an input phase to the vestibular system. The motion of the head 31 is represented by a translational acceleration α expressed by the following equation (4.1) and an angular velocity ω expressed by the following equation (4.2), and both are three-dimensional vectors. Input to vestibular oculomotor reflex model.
Here, a block 411 of g (ω) updates the gravitational acceleration vector g caused by the inclination of the head 31 at the current time using the angular velocity vector ω, and the following differential equation (4.3) It is represented by

The input to the semicircular canal is directly ω, and the physical quantity perceived by the otolith organ is the combined vector f of the inertial force vector and the gravitational acceleration g generated by the translational acceleration α, and is calculated as in the following equation (4.4). You.

II: Transduction by otoliths and canals
This phase represents the transfer characteristic S _oto of the otolith organ for sensing head movement and the transfer characteristic S _{scc of the} semicircular canal.

The otolith organ is inclined about 30 degrees upward with respect to the horizontal plane, and is considered to be pulled backward and downward by gravity. In consideration of this characteristic, the transfer function S _oto of the otolith organ indicated by the block 412 is expressed by the following equation (4.5) by applying a unit matrix and a constant force in the z-axis direction.

The transfer function S _{scc of the} semicircular canal shown by the block 413 is a 3 × 3 diagonal matrix, and all of its elements are given by the following equation (4.6). Here, τ _d and τ _a are time constants, respectively.

As described above, the combined vector f of the inertial force direction acceleration α and the gravitational acceleration g considers the transfer characteristic S _oto of the otolith, and the angular velocity vector ω considers the transfer characteristic S _{scc of the} semicircular canal. Sensory information α _oto and α _scc transmitted to the system.

III: Internal processing
In this phase, the central nervous system processes sensory information transmitted from sensory organs (otoliths and semicircular canals) into perceptual information, and assumes that an internal model of the body and sensory organs is included. By feeding back the error between the estimated value of the sensory information of the internal model and the actual sensory information, the difference between the recognized information and the actual information is reduced.

The transfer functions of the internal models (sensory organs and semicircular canals) of the sensory organs are all 3 × 3 diagonal matrices, and their elements are represented by the following equations (4.7) and (4.8).

The combined vector f ^ of the inertial force direction acceleration and the gravitational acceleration estimated by the internal model becomes the estimated value α ^ _oto of the otolith sensory information via the block 414, and the angular velocity vector ω ^ estimated by the internal model is _calculated by the block 415. After that, it becomes an estimated value α ^ _scc of the sensory information of the semicircular canal.

A block 416 (dump) of a loop portion of the internal model of the semicircular canal represents the attenuation characteristics of the eyeball, and is represented by a nonlinear function of the following equation (4.9).
Here, a and b are coefficients, and the argument ω is not a vector but a norm of the vector.
Through these loops, the estimated values α ^ and ω ^ of the translational acceleration and the angular velocity are calculated, respectively.

A block 417 (X) represents an outer product, which is converted into rotation information by taking an outer product of each normalized vector of the otolith sensory information α _oto and the estimated value α ^ _oto by the internal model, and the difference in the direction, In other words, it represents the error signal between the otolith sensory information and the estimated value of the sensory information of the internal model of the otolith, fed back to the internal model of the body, further transmitted to the internal model loop of the semicircular canal, and the mutual Represents the action. Due to the interaction between the otolith and the semicircular canal, the translational acceleration and the gravitational acceleration of the head are distinguished and perceived, and the posture of the head can be recognized from the information of the vestibular system. For example, in the state of lying on the floor, the otolith is pulled backward by the gravitational acceleration, and therefore, the acceleration is applied in the backward direction of the head, but it is not perceived as the translational acceleration.

  Block 418 represents the internal model of the body and calculates an estimate g ^ of the gravitational acceleration vector.

IV: Eye movement generation
In this phase, the motion command ω _VOR to the eyeball of the vestibular oculomotor reflex is calculated using the perceived translational acceleration α ^ and angular velocity ω ^. The motion command ω _VOR of the vestibular oculomotor reflex VOR is composed of a rotation VOR which is an element of a rotation component and a translation VOR which is an element of a translation component, and is calculated by adding both.

The perceived angular velocity vector ω ^ is inverted by the gain and becomes a rotation VOR motion command ω _R-VOR . The perceived translational acceleration vector α ^ is integrated into a translational velocity vector by a leaky integration circuit 419 (Leaky Integrator), converted into an angular velocity vector using a target line-of-sight direction, and passed through a high-pass filter 420 to execute a translational VOR motion command. ω _T-VOR .

The angular velocity ω ^ _V based on the translation speed can be calculated by the following equation (4.10) using the position vector d of the target line of sight.

The transfer function of the high-pass filter 420 is represented by the following equation (4.11).

  The details of the vestibular oculomotor reflection model 41 have been described above. This model is calculated in a frequency domain such as a transfer function. In the present embodiment, the vestibular oculomotor reflection model is converted into a time domain, and a mathematical model including 18 first-order ordinary differential equations is used. Since the variables are three-dimensional vectors, there are six types of differential equations × 3. The formula is shown below.

α _scc can be solved by the differential equation of the following equation (4.12) using the parameters α _scc2 , α _scc21 , and α _scc22 .

Here, the relationship between these parameters α _scc2 , α _scc21 , and α _scc22 is as follows (4.13).

Since α _oto and α ^ _oto are transfer functions each of which is a constant diagonal matrix, the following equations (4.14) to (4.16) are obtained from the relationship of signal flow.

alpha _oto, alpha ^ _oto of the error signal e _f is because it is the outer product between the normalized vector is expressed by the following equation (4.17).

“Dump” is as shown in the following equation (4.18).
Here, dump (ω) is a scalar because it is a function using the norm of a vector as a variable.

ω ^ is _given by the following equation (4.19) from the signal flow and using the parameter α ^ _scc2 of α ^ _scc .
The relationship between α ^ _scc and α ^ _scc2 will be described later.

Since ω ^ _d is simply a constant multiple, it is expressed by the following equation (4.20).

α ^ _scc can be solved by the following differential equation (4.21) using the parameter α ^ _scc2 .
The relationship between α ^ _scc and α ^ _scc2 is shown in the following equation (4.22).

g ^ can be solved by the following differential equations (4.23) and (4.24).

α ^ is represented by the following equation (4.25), and v ^ is the integral of α ^ by the leaky integrator, so that the differential equation of the following equations (4.25) to (4.27) I can solve it.

ω _T-VOR can be solved by the following differential equation (4.28) using the parameter ω _T-VOR2 .
The relationship between ω _T-VOR and ω _T-VOR2 is shown in the following equation (4.29).

Since ω _R-VOR is obtained by simply _multiplying ω ^ by a gain and inverting it, it is expressed by the following equation (4.30).

Finally, the output ω _VOR of the vestibular oculomotor reflex is expressed by the following equation (4.31) as the addition of ω _T-VOR and ω _R-VOR .

  In addition, the method and order of the equation transformation were set as described above from the relation of solving a differential equation by a computer. As described above, by treating the vestibular oculomotor reflection model 41 as a mathematical model in the time domain, it becomes a format applicable in the present embodiment.

4.2 Pursuit Eye Movement (Smooth Pursuit)
The pursuit eye movement is a smooth eye movement that occurs when the viewpoint follows a slowly moving visual object. The stimulus that causes the movement is the speed of the moving object, and is an optional movement that holds the object on the retina by matching the eye movement speed with the object. Although it cannot follow fast movements and can only smoothly follow up to about 50 deg / s, it can follow up to about 90 deg / s when the trajectory of the index can be predicted.

  In the present embodiment, Robinson et al.'S model was used in modeling the tracking eye movement. The tracking eye movement model 42 receives a target velocity and outputs an eye velocity that follows the target velocity. It is composed of three feedback loops, and takes into account the delay characteristics of the tracking eye movement, and uses time delay constants at various places. In this tracking eye movement model 42, the signal is handled as a three-dimensional vector similarly to the vestibular movement reflection model 41, and the same processing is performed for each element. A block diagram of the tracking eye movement model 42 is shown in FIG.

4.2.1 Near CNS (Central Nervous System) Here, the eyeball angular velocity (time differential of E) based on the actual tracking eyeball movement and the tracking eyeball movement command which is an efferent copy of the eyeball angular velocity based on the tracking eyeball movement are fed back. Will be. The target angular velocity is copied to the central nervous system, that is, the perceived information of the target angular velocity is reproduced by the error signal of the target angular velocity and the eyeball angular velocity and the tracking eye movement command fed back. Each block indicated by a numeral is a coefficient representing a time delay of the numeral (msec), and τ ₁ and P ₂ are also coefficients representing a time delay.

Block 421 (Plant) represents the ocular muscle and orbital tissue, and CP (Central Processing) shown by block 422 represents the transmission characteristics of the central nervous system, respectively, as expressed by the following equations (4.32) and (4.33). It is represented by a transfer function.
Here, τ _e2 is a time constant of Plant and τ _c is a time constant of CP.
Through the processing by these blocks 421 and 422, the perceived information of the angular velocity of the target becomes the target eyeball angular velocity.

4.2.2 PMC (Premotor circuity)
This represents the premotor circuit. The target eyeball angular velocity is input, and a motion command of the eyeball angular velocity at the current time is output. The motion command of the eyeball angular velocity is fed back, compared with the target eyeball angular velocity, forms an error signal, and serves to bring the motion command of the eyeball angular velocity closer to the target value. Here, the error signal is proportional to a change in the target eyeball angular velocity, and thus is also proportional to the target eyeball angular acceleration. This error signal becomes a target eyeball angular acceleration through a block 423 (AS: Acceleration Saturation). A block 423 (AS) is a part for processing acceleration saturation, and is divided into a case where the threshold value is larger than the threshold value and a case where the acceleration value is smaller than the threshold value, and is represented by the following equation (4.44).

The target eyeball angular acceleration calculated by the block 423 (AS) passes through a time delay constant τ _2, and is integrated by a block 424 (NI: Neural Integrator) to become an eyeball angular velocity movement command. The block 424 (NI) is represented by the following equation (4.45). Where A is a coefficient.

4.2.3 Near eyeball angular velocity output
The movement command of the eyeball angular velocity calculated by the PMC becomes a movement command of the eyeball angular velocity with a delay after passing through the time delay constant P1 and the block 425 (Velocity Saturation). However, block 425 (VS) has no effect below 90 deg / s. Actually, it is difficult to imagine an eyeball angular velocity exceeding 90 deg / s in tracking eyeball movement. Therefore, the processing of block 425 (VS) is not performed. The movement command of the eyeball angular velocity with a delay is fed back to the CNS as an efferent copy, and at the same time, becomes the eyeball angular velocity via a block 426 (Plant). The transfer function of the block 426 (Plant) is obtained by multiplying the equation (4.32) related to the block 421 (Plant) by a gain.

  The details of the tracking eye movement model 42 have been described above. Since this model is also a model in the frequency domain like the vestibular oculomotor reflection model 41, it is converted to a time domain applicable to the present embodiment, and a mathematical model including 12 first-order ordinary differential equations is used. However, since the signal is a three-dimensional vector, there are four types of differential equations × 3. The equations are shown below.

The target eyeball angular velocity and the eyeball angular velocity movement command are both variables of a differential equation, as described later. Therefore, the computer is used to solve the differential equation, so that the PMC feedback loop can be solved first using these. The error signal is as shown in the following equation (4.46).

Next, a target eyeball angular acceleration is obtained by a block 423 (AS) as in the following equation (4.47).

The integration of the target eyeball angular acceleration by the block 424 (NI) is solved by the following differential equation (4.48), and a motion command for the eyeball angular velocity is obtained.

As described above, since the block 425 (VS) is not processed, it is simply represented by the following equation (4.49).

The feedback of the eyeball angular velocity and its motion command to the CNS can be solved by the differential equations of the following equation (4.50).

Here, the error signal between the angular velocity of the target and the eyeball angular velocity and the perception information of the angular velocity of the target are obtained by the following equations (4.52) and (4.53), respectively.

Since the perceived information of the target angular velocity has been obtained, the target angular velocity can be solved by the differential equation of the following equation (4.54).

  As described above, the method and order of the equation transformation are as described above from the relation of solving a differential equation by a computer, and as described above, the tracking eye movement model is treated as a mathematical model in the time domain. This is a format applicable in the present embodiment.

4.3 Saccade
Saccades are rapid eye movements that occur when the user moves the point of gaze to see an object at will. Saccades are characterized in that once eye movements occur, they cannot be stopped arbitrarily until the saccade ends. The speed of movement is dependent on the angle of rotation of the eyeball, cannot be adjusted at will, and the duration increases in proportion to the amplitude. The maximum speed also increases in proportion to the amplitude of the motion and reaches about 700 deg / s, but is about 350 to 500 deg / s at an amplitude of 10 to 20 degrees.

  In modeling the saccade, a saccade model by Tweed was adopted. The input is a target position described in the eyeball coordinate system, that is, a line-of-sight error. The saccade is obtained by driving the eye 5 and the head 31 according to the saccade motion rule so as to minimize the line-of-sight error. It is a model that realizes. In addition, the feature of this model is that the variables are mainly handled by quaternions, and are represented by a 4 × 4 quaternion matrix. FIG. 9 shows a block diagram of the saccade model 43. Hereinafter, each calculation method will be described.

4.3.1 sight comparator this saccade model 43, the target position is written in the eye coordinate system T _e, i.e. the line of sight errors as input, alters this as the attitude of the eye coordinate system and the target is coincident . The change rule is a cross product of the angular velocity vector omega _ese of eye 5 against the space described by the target position T _e and the eyeball coordinate system, represented by the differential equation of the following equation (4.55). Note that ω _ese converts this variable into a usable form, not as it is.

Here, quaternion q _eh is eyeball orientation in the head coordinate system, omega _hsh is angular velocity vector with respect to the space of the head 31 in the head coordinate system. The juxtaposed ω _hsh q _eh is a quaternion product of a vector and a quaternion, and the vector is treated as a quaternion with a scalar part element of 0.

4.3.2 Donders and head pulse generator P _h
First, in block 431 (Donders), the target head posture is obtained from the following equations (4.56) to (4.58).

Formula (4.56) by using a head posture q _h in the spatial coordinate system, and the eyeball position q _eh in the head coordinate system with the coordinate conversion on the eyeball position q _es in the spatial coordinate system. Next, the coordinate transformation into the target T _s in the spatial coordinate system of the target T _e in the eye coordinate system using the eyeball position q _es. Using this T _s , a target head posture is calculated via Donders.

The role of the block 431 (Donders) is to define a head posture that makes it easy to see the target. Normally, the head 31 tends to rotate more easily on the horizontal plane than on the sagittal plane. In consideration of these factors, the conversion rule from the target direction to the head posture that makes it easier to see the target is called the Donders rule. Are defined in three steps of the following equations (4.59) to (4.61).

Here, i, j, and k are unit vectors along three axis directions of the coordinate system, respectively, where i indicates forward, j indicates leftward, and k indicates upward. x ₂ and x ₃ are elements along the j-axis and k-axis of the vector x, respectively, and “•” represents an inner product.

Regarding equation (4.59), the quaternion product of −T _s and i is the quaternion square root. The quaternion square root means that, for example, when p is a quaternion square root of q, a quaternion product pp = q holds. Here, the rotation axis of p is equal to q, but has the characteristic that the amplitude of rotation is half. Therefore, the meaning of the quaternion x indicates the shortest rotation by taking the i direction with respect to the target.

The target head posture is determined by using the calculated y by performing scaling by the coefficients δ _T , δ _V , and δ _H in the above equation (4.60).

Next, the head pulse generator 432, the head 31 is the time change of the head posture q _h (time derivative), i.e. is controlled by the speed command is calculated as a function of the following equation (4.62).

Here, V _q is a function that takes a vector of the quaternion, the re expressed as v, P _h is defined by the following equation (4.63).

Although the pulse generator 432 is usually defined by an exponential function, an approximate expression is used here for simplicity. Here, the temporal change of the head posture represented by Expression (4.62) is a quaternion velocity, not an angular velocity vector. The relationship between the angular velocity vector and the quaternion velocity is expressed by the following equation.

4.3.3 Listing
Block 433 (Listing) is a portion for calculating a target ocular orientation in the head coordinate system from the target T _s, is expressed by the following equation (4.64).

The argument to the function of the block 433 (Listing) indicates the relative position direction of the target when the head 31 is in the target head posture. In block 433 (Listing) is first performed quaternion product of arguments and g _v. g _v represents the main gaze direction in the head coordinate system, and g _v is i. Thereafter, a quaternion square root is obtained, and a target eye posture in the head coordinate system is obtained.

4.3.4 Eye saturation and eye pulse generator The eye has an effective eye movement range (EOMR), and its effects need to be considered in saccades. If the target is located beyond the movable limit, the eyeball must be saturated, that is, the target eyeball posture must be within EOMR. These processes are performed in three steps represented by the following equations.

First, a target eyeball posture in a space coordinate system is obtained. Equation (4.65) shows the target eye posture with respect to the target head posture. Thereafter, using the current head posture q _h , the target eye posture in the head coordinate system at the present time is expressed by the following formula. It is calculated by (4.66).

  Block 434 (Sat) projects the excessively eccentric current target eye posture into the EOMR. That is, the effective target eyeball posture represented by Expression (4.67) is located on the boundary of the EOMR as shown in the schematic diagram of FIG.

The process of block 434 (Sat) is as shown in the following equation (4.68).

When the square horizontal and vertical eccentricity α of the current target eye posture q _eh ⁺ is larger than the square of the radius EOMR radius (= sin (40 ° / 2)), solve a quadratic equation with coefficients a, b, and c Thus, q _eh ⁺ is projected on the horizontal and vertical planes of the EOMR. Since the projected point is still outside the EOMR with respect to the torsional dimension here, the effective tolerance of the torsion element is now defined. signum is a function that returns 1 if its argument is positive, -1 if its argument is negative, and 0 if it is 0. The equation in the last row converts q _eh ^s from a vector to a unit quaternion, which is the output of block 434 (Sat).

Next, the effective target ocular posture q _eh ^s calculated becomes a speed commanded by the eyeball pulse generator 435 (P _e). Ocular pulse generator 435 (P _e) is represented by the following formula similar to the P _h (4.69) and (4.70). However, it is ^{_{v = q eh s q eh -1}} .

4.3.5 VOR Shutoff
The vestibular oculomotor reflex (VOR) is a reflexive eye movement that also affects saccade-induced eye movements. That is, when a saccade occurs, the vestibular oculomotor reflex (VOR) is also activated, but the saccade is shut out by a certain signal, and the saccade causes the eye movement to become dominant. Here, it is assumed that the vestibular oculomotor reflex (VOR) is shut out by the error signal of the eyeball in the current head coordinate system.

It should be noted that the “VOR” mentioned here takes into account only the interference part of the VOR during the saccade, and is different from the vestibular oculomotor reflection model 41 (VOR) described above.
In block 436, the VOR interference signal is processed according to the following equation (4.71).

Here, M is a 3 × 3 matrix, I is a 3 × 3 unit matrix, and uu ^T is a 3 × 3 matrix by a matrix product of a column vector u and its transpose. x is an eye movement error signal in the head coordinate system in which the eye is not saturated at the present time, C is a coefficient defined as cos (A / 2), and A is a threshold value of VOR shut-off. If the error signal exceeds A, VOR is completely shut off. Then, the eye movement error signal converges from A to 0. That is, the scalar part x ₀ converges from C to 1. Then VOR recovers gradually. The processing of block 436 expresses the eye movement of the saccade affected by the VOR.

Finally, the eyeball speed command in the head coordinate system of the saccade is calculated by the following equation (4.72) from the sum of the eyeball speed command considering the effective eyeball movement range and the interfering eyeball speed command of the VOR through the block 436. Is done.

5. Integration of eye movement model and musculoskeletal model The above-mentioned three eye movements are appropriately selected according to the conditions such as the index (target) and head movement, and are optimal eyeballs for accurately recognizing the index while interpolating each other. Exercise is performed. Therefore, when constructing the eye movement model 4, these eye movement models are integrated so that final eye movement is generated, and then integrated with the musculoskeletal model 3.

Basically, as shown in FIG. 6, the acceleration and angular velocity of the head 31 are input to the vestibular oculomotor reflection model 41, the eyeball angular velocity ω _VOR by the vestibular oculomotor reflection is output, and the tracking eye movement model 42 is The target speed and the error between the target and the eye line are input to the _saccade model 43, and the eyeball angular velocity ω _SmPu by tracking eye movement and the eyeball angular velocity ω _Sac by _saccade are output. Then, these are integrated and the final eyeball angular velocity ω _eye is output, but these eyeball models 41, 42, and 43 do not have a simple addition relationship. FIG. 11 shows a block diagram of a model in which the characteristics of each eye movement model are taken into account and the musculoskeletal model 3 (31) is integrated.

  First, the saccade model 43 does not have a time dimension because it receives as input a unit vector in the direction of the target in the eyeball coordinate system, that is, an error with an index on the retina. Therefore, saccades cannot be generated when saccades are required within the simulation time by simple addition. Therefore, in FIG. 11, a condition for saccade activation is defined in the process of arriving at the saccade block 43 so that the occurrence of saccade can be controlled.

  That is, a switch 44 that is turned on under a predetermined condition is defined on the input side of the saccade 43, and the switch 44 is turned off at the start to leave no line-of-sight error for saccade input. Perform calculations. By doing so, the output of the saccade model 43 is always 0, and does not affect the final eyeball angular velocity.

When the predetermined condition is satisfied and the switch 44 is turned ON, the value of the line-of-sight error at that time is input to the _saccade model 43, and the eyeball angular velocity ω _Sac is output. There are two conditions under which the switch 44 is turned on.
Sac_trg == ON && Sac_flg == OFF
It is. Sac_trg is a saccade input trigger, that is, a trigger that is turned ON when the norm of the target angular velocity vector in the head coordinate system exceeds a certain threshold. On the other hand, Sac_flg is a flag that is turned on during the occurrence of a saccade when the eyeball speed commands by the saccade are 1.0 (deg / s) or more.

  The reason for setting such a flag is that if only the trigger is used, the saccade motion will be updated if the input trigger becomes ON during the occurrence of the saccade, and the saccade motion will not be consistent with the original saccade motion. That's why. The moment when the index is discrete is one step, and the flag is turned on in the next step, so that the trigger is turned on for only one step.

  Next, with respect to the tracking eye movement, three processes of subtraction of the output of the vestibular ocular movement reflection block 41, SMA 45, and low-pass filter 46 are added to the input side of the tracking eye movement block 42.

(About SMA (Simple Moving Average))
The pursuit eye movement is a movement that follows the smooth movement of the target, and therefore cannot correspond to a pulse input or the like. The saccade works when the input cannot be handled by the tracking eye movement, but if the input to the tracking eye movement or the output from the tracking eye movement is set to 0 while the saccade is working, the saccade ends. Still shows unstable behavior. Therefore, a simple moving average (SMA) is used as a method for smoothly interpolating an input during the occurrence of a saccade, instead of setting the input to 0. That is, by adding the switch 47 which is turned on by the flag of the saccade and switching to the SMA 45, the pulse-like input is smoothed, which is synonymous with the absence of the pulse-like input in the tracking eye movement block 42. In other words, saccades 43 are provided for discrete inputs, and pursuit eye movements 42 are provided for smooth inputs.

(About subtraction of vestibular oculomotor reflex output)
The interpolation of the eye movement with respect to the movement of the head is performed by the vestibular oculomotor reflex, whereby the angular velocity of the target in the head coordinate system which is the input of the tracking eye movement is affected by the head movement. For example, if the head vibrates at a high frequency due to a disturbance, the vibration affects the angular velocity of the target, becomes an input signal in a high-frequency band that cannot be tracked originally, and outputs an unnecessary eyeball angular velocity. Therefore, when the head vibrates at a high frequency, it is necessary to cut off the high-frequency input to the tracking eye movement. Therefore, there is a path between the tracking eye movement and the vestibular oculomotor reflex, which transmits an efferent copy of the vestibular oculomotor reflex eye velocity signal. It is assumed that the high frequency components of the input have been removed. Since the output of the vestibular oculomotor reflection is an eyeball angular velocity that cancels the shaking of the head, high frequency components can be removed by simply subtracting the output of the vestibular oculomotor reflection 41 from the index velocity input.

(About low pass filter)
By subtracting the output of the vestibular oculomotor reflection 41 as described above, the low-frequency component of the target angular velocity that should originally be tracked by the tracking eye movement becomes clear, and by applying a low-pass filter 46 to this signal, Only the input signal from which the high-frequency component has been removed is input to the tracking eye movement model 42.

(Correction of position error by saccade)
By performing the processing described above, the eye movement models complement each other even when the head is vibrating or the saccade is working, so that the target can be tracked. Shoots may occur. Since the tracking eye movement is controlled based on the velocity error, it cannot respond to the position error. Therefore, the positional deviation between the index and the line of sight can be corrected using the saccade 43 having the position error as an input. For example, a trigger different from the above-described trigger is defined, and the saccade 43 is turned on when the length of the position vector of the target in the eyeball coordinate system becomes larger than a threshold value (for example, 0.1 m). .

6. Integration of Eye Movement Model and Musculoskeletal Model In the block diagram of FIG. 11, the output from the eye movement model 4 is roughly divided into an angular velocity ω _{eye of} the eyeball and an angular velocity command ω _ha to the head. Hereinafter, a coupled mechanism of eye movement and body movement based on each output will be described.

6.1 Head angular velocity command from eye movement model The angular velocity command to the head 31 output from the tracking eye movement model 42 and the saccade model 43 is added, and an active angular velocity command accompanying the eye movement of the head 31 is added. as ω _ha, it is input to the head 31 of the musculoskeletal model 3. Therefore, by taking this movement command _ωha as a target value and incorporating it into the calculation of the inverse model as a willingness, an active head movement accompanying the eyeball movement can be generated.

Since these are rotational motions, they are calculated as the will moments by the following equation (6.1).
Therefore, the head 31 of the musculoskeletal model 3 can generate an active head movement command by the eyeball movement model 4 in addition to the movement command for maintaining the posture corresponding to the movement of the body 30. Become.

  From the head 31 of the musculoskeletal model 3 to the vestibular oculomotor reflex model 41 based on the passive head movement (31) for the body movement (30) and the active head movement accompanying the eye movement (4) Of the head, such as acceleration and angular velocity, can be obtained. However, in this state, the vestibular oculomotor reflex occurs for the head movements operated by the other eye movement models 42 and 43, and the desired gaze rotation direction is canceled. Therefore, by assuming that the efferent copy of the active head movement command issued from the other eye movements 42 and 43 is transmitted to the vestibular oculomotor reflex model 41, the active head movement is calculated from the actual head movement. The vestibular oculomotor reflex occurs only for the part excluding the head movement, that is, only for the passively generated head movement, thereby avoiding interference with active head movement.

6.2 Angular Velocity of Eyeball Next, the magnitude of the angular velocity ω _eye output from the eyeball movement model 4 to the eyeball 5 directly corresponds to the movement load of the eyeball movement. Therefore, the eye movement load f _eye of the musculoskeletal model 3 shown in the equation (3.7) can be defined as the following equation (6.2).
Here, omega _eye uses a norm of a vector, T _g is the position vector of the target as seen from the eye coordinates, e _x is the x-axis vector of the eyeball coordinate _{(e x = (1 0 0} ) T), λ is a weighting factor It is.

The first term of the above equation (6.2) has a function of suppressing eye movement, and the second term represents a function of catching a target as close as possible to the gazing point. By adding this eye movement load f _eye to the body movement load as shown in the above-described formula (3.7) in the inverse model-based movement command generation, a coupled movement of the eye movement and the body movement can be generated. Become.

7. Process Flow of Visibility Evaluation System As described above, the evaluation system 1 according to the present invention simulates the operation of confirming the visibility target point 20 in the evaluation target data 2 using the biomechanical model 10, and performs the confirmation operation. FIG. 12 is a flowchart illustrating a processing flow of the evaluation system 1 according to the embodiment, which evaluates visibility in consideration of ease (burden).

  Further, as described above, the evaluation system 1 is composed of the biomechanical model 10 and the visual field determination model 50 that are coupled while operating independently, and FIG. 13 is a processing flow of the evaluation system 1 according to the embodiment. Are divided into a process in the biomechanical model 10 (confirmation operation simulation) and a process in the visual field determination model 50 (visibility determination).

  The lower two digits of each step shown in the 100s in FIG. 12 correspond to the lower two digits of each step shown in the 200s and 300s in FIG. The processing flow of the evaluation system 1 will be described with reference to FIG.

  At the time of the visibility evaluation, first, the computer constituting the evaluation system 1 is activated, and the biomechanical model 10 and the visual field judgment model 50 are activated in a state where the main program (1) for controlling the simulation and the database 8 are activated. (Steps 100, 200, and 300).

  Next, in accordance with the input information of the simulation data 80, the predetermined evaluation target data 2 is read into the 3D data space of the visual field determination model 50, and the viewing target point 20 of the evaluation target data 2 is acquired by the biomechanical model 10 (Step 101). , 201, 301).

  Next, in the visual field determination model 50, the gradation sphere 21 is set at the visual recognition target point 20 of the evaluation target data 2 (Steps 102 and 302). The data of the gradation sphere 21 itself is prepared in the visual field determination model 50 in advance, and the center coordinates of the gradation sphere 21 are set to the viewing target point 20.

Next, posture information is read from the simulation data 80, an initial posture is generated in the musculoskeletal model 3 and the eye movement model 4 of the biomechanical model 10, and the initial posture is reflected in the human body model 53 of the visual field judgment model 50, and The initial position of the camera 55 is obtained (steps 103, 203, 303). In this state, a field-of-view image is acquired from the camera 55, and an initial value R _{0 of} visibility is calculated.

  When the process shifts to the perturbation / behavior process through the above preparation, first, the state variables are calculated in the inverse model-based motion command generation module 12 and the forward model-based motion control module 13 (step 110, step 110). 210, 310).

  Next, the joint drive moment is calculated by the inverse dynamics calculation in the motion command generation module 12, the acceleration is calculated by the forward dynamics calculation in the motion control module 13, and the acceleration is integrated over time to obtain a unit time (in the embodiment, for example). The posture after 0.1 second) is estimated. The perturbation is given to the biomechanical model 10 in this way, and the obtained posture information (the skeleton link posture and the eyeball posture, all of which are quaternions) and the joint driving moment (body load) acting on the body mechanical system are the burden B Are stored in the perturbation data 82 together with the state variables (steps 111 and 211).

  Further, the posture information obtained by the perturbation is reflected on the human body model 53 of the visual field judgment model 50 and the camera 55 (FIG. 13, step 311), and the visibility R is calculated as the sum of the R values of the visual field images acquired by the camera 55. Is calculated (steps 112 and 312) and stored in the perturbation data 82.

  The above-described perturbation process is repeated by the number of state variables, and when the perturbation of all the state variables is given (steps 113 and 213), the process proceeds to the behavior process.

First, among the state variables for which the burden B and the visibility R have been acquired in the perturbation process, the state variables that provide the minimum burden B _min (minimization of physical load) and the maximum visibility R _max are calculated and corresponded. A joint drive moment is determined (steps 120, 220).

  Next, the biomechanical model 10 (the musculoskeletal model 3 and the eye movement model 4) is actually driven by the determined state variables and the joint driving moment, and a posture based on the behavior is generated. , Eyeball posture, quaternion) are stored (steps 121 and 221).

  Further, the posture information based on the behavior is reflected on the human body model 53 of the visual field judgment model 50 and the camera 55 (FIG. 13, step 321), and the visibility R is calculated as the sum of the R values of the visual field images acquired by the camera 55. (Steps 122 and 322) are stored in the behavior data 81. In addition, the posture information of the human body model 53 is registered in the key frame (FIG. 13, step 324).

  In parallel with this, the integrated value of the burden B including the behavior process is calculated in the database 8, and the visibility is calculated with reference to the visibility R in consideration of the load B (integrated value). (Step 222).

  Next, the visibility R is compared with a preset threshold (steps 130, 230, 330), and if it is determined that the threshold has not been reached, the perturbation / behavior process as described above is repeated.

  When the visibility R is equal to or greater than a preset threshold, the simulation using the biomechanical model 10 is terminated, and the visibility is easily considered in consideration of the visibility R and the burden B (integrated value) calculated in the final behavior process. The sex evaluation is recorded in the simulation data 80 as a result of the simulation (trial No) (FIG. 13, step 231).

  At the same time, in the visual field judgment model 50, the rendering is executed based on the data of the key frames (0.1 second interval, 10 fps in the embodiment) registered up to the final behavior process, and an evaluation image is created (FIG. 13, FIG. Step 331).

  At the time of rendering, the body motion between key frames is interpolated by the moving image creation function. Alternatively, the evaluation target data 2 (instrument panel) or the surface information of the human body model 53 may be reflected, or the position of the viewing target point 20 may be clarified by using a gray scale or the like. In addition to the visual field image obtained by the camera 55 (human body model 53), an image can be created from an external viewpoint set at an appropriate position such as a position obliquely above the human body model 53.

  In the perturbation process (steps 111 to 112, 211, 311 to 312) of the above embodiment, the posture information obtained in the biomechanical model 10 is reflected on the human body model 53 and the camera 55 of the visual field judgment model 50 for each perturbation. Then, instead of calculating the visibility R from the visual field image, the posture information for all (or a part of) the state variables is obtained in the biomechanical model 10 and then transferred to the visual field determination model 50 collectively. Obtaining the view image and calculating the visibility R may be performed.

In the perturbation process (steps 111 to 112, 211, 311 to 312) of the above-described embodiment, a case where perturbation is given to the biomechanical model 10 by the number of state variables has been described. By giving a perturbation, it is also possible to estimate a state variable having a minimum burden B _min (minimizing physical load) and a maximum visibility R _max, and determine a joint driving moment for behavior.

  Further, in the above-described embodiment, the case where the perturbation / behavior process is repeated until the visibility R becomes equal to or greater than a preset threshold has been described. However, an upper limit is provided for the number of repetitions (or repetition time) of the perturbation / behavior process. The simulation by the biomechanical model 10 may be ended when a predetermined number of times (predetermined time) is reached.

  On the other hand, by improving the visibility R by repeating the perturbation / behavior process, the joint drive moment calculated by the motion command generation module 12 gradually decreases, and the acceleration calculated by the motion control module 13 gradually decreases. When the state value (or the rate of change thereof) becomes equal to or less than the setting, the confirmation operation by the biomechanical model 10 may be regarded as autonomously converging.

  In the above-described embodiment, the case of simulating the operation of confirming one viewing target point 20 from the initial posture in the biomechanical model 10 has been described. High visibility can also be evaluated.

In a preferred aspect of the present invention, the general control unit (11) includes:
A behavior in which a perturbation is given to the biomechanical model, the degree of visibility of the biomechanical model associated with the perturbation is calculated in the posture, and the degree of burden of the biomechanical model is calculated in accordance with the perturbation. Has a function of performing an operation of generating a posture, and calculates the visibility from the integrated value of the burden and the final value of the visibility.

In a preferred aspect of the present invention, the general control unit (11) includes:
Exercise command generating means for perturbing the biomechanical model, and means for calculating the burden of the biomechanical model accompanying the perturbation, a plurality of trials perturbation process including the calculation of the visibility and burden And performing a repetitive operation of a perturbation / behavior process for generating a posture with a behavior having a minimum burden and a maximum visibility.

In a preferred aspect of the present invention, the general control unit (11) includes:
A perturbation process including the steps of: providing a perturbation to the biomechanical model; calculating visibility with respect to the viewing target point in a posture caused by the perturbation; and calculating a burden of the biomechanical model accompanying the perturbation. Is performed a plurality of times, and a repetitive operation of a perturbation / behavior process for generating a posture with a behavior having a minimum burden and a maximum visibility is performed.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications and changes can be made based on the technical idea of the present invention.

  For example, in the above embodiment, the case where the instrument panel of the vehicle is used as the evaluation target data 2 and the switch at the lower right side is used as the visual recognition target point 20 has been described. However, the present invention is not limited to this. The viewing target point may be another position, or data other than the instrument panel may be used as evaluation target data. In addition, the present invention can be applied to evaluation of easy visibility and design support of various articles which are expected to be visually recognized with a confirming operation, such as a standing posture other than a sitting posture, a walking posture, and a confirming operation of a visually recognizable target point when getting on and off the vehicle. .

1 Evaluation system 2 Evaluation target data (instrument panel)
3 Musculoskeletal model (body model)
4 Eye movement model 5 Eye 6 Support structure (sheet)
7 Support structure (handle)
Reference Signs List 8 Database 10 Biomechanical model 11 Overall control unit 12 Reverse model-based motion command generation module 13 Forward model-based motion control module 14 Motion reference potential 15 Visibility 16 Burden 17 Easy-visibility comprehensive evaluation unit 20 Visual target point 21 Gradation Sphere 30 body 31 head 50 visual field judgment model 51 judgment processing unit 53 human body model 55 camera

Claims (7)

A visibility evaluation system for an object by 3D CAD,
A biomechanical model for simulating a confirmation operation on the visual target in the virtual 3D space in which the object and the visual target are defined;
A visual field determination model for calculating the visibility of the visual target in the visual field image in the line of sight of the biomechanical model,
A behavior in which a perturbation is given to the biomechanical model, the degree of visibility of the biomechanical model associated with the perturbation is calculated in the posture, and the degree of burden of the biomechanical model is calculated in accordance with the perturbation. An overall control unit that couples the biomechanical model and the visual field determination model by performing an operation of generating a posture at
And a visibility evaluation system configured to calculate easy visibility from the integrated value of the burden and the final value of the visibility.   The biomechanical model includes a musculoskeletal model that models the skeleton, joints, and skeletal muscles of a human body, and an eyeball movement model that determines an eyeball posture in conjunction with head movement in the musculoskeletal model, The visibility evaluation system according to claim 1, wherein the line of sight is given by a posture.   The visibility evaluation system according to claim 1, wherein the perturbation process is configured to be tried by the number of state variables of the biomechanical model.   The visual field judgment model is a transparent solid body in which the visual target point is radially enlarged in a virtual 3D space, and a gradation solid body in which only a specific pixel information value is reduced or gradually reduced according to a distance from the visual target point. The visibility evaluation system according to any one of claims 1 to 3, wherein the visibility evaluation system is configured to define and calculate a sum of the specific pixel information values in the visual field image as the visibility.   The gradation solid has a single color tone, the specific pixel information value is the luminance of the single color tone, and the gradation solid has a large luminance on the center side and a luminance from the center side toward the peripheral side. 5. The visibility evaluation system according to claim 4, wherein the visibility evaluation system decreases or gradually decreases.   The visual field determination model acquires the visual field image in a state where all objects other than the gradation solid are black and opaque, and calculates the total visibility value of the specific colors in the visual field image, thereby obtaining the visibility. The visibility evaluation system according to claim 5, wherein the visibility evaluation system is configured to calculate.   The visibility evaluation system according to any one of claims 4 to 6, wherein the gradation solid is a gradation sphere centered on the viewing target point.