JP2014095557A - Motion tracker device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a motion tracker device capable of calculating an angle of an object with respect to a camera coordinate system with high accuracy.
An actual measurement position calculation unit for calculating an actual measurement position of each optical marker with respect to a camera coordinate system, and a memory for storing reference information indicating a reference position of each optical marker with respect to the camera coordinate system. and parts 42, a motion tracker device 1 and a angle calculator 27, based on the measured positions of the angular velocity and the time _{t n} from the time _{t (n-1)} to time _{t n,} the time _{t (n-1 )} Includes a true position calculation unit 25 that calculates the true position of each optical marker 7, and an update unit 26 that changes the reference information using the true position at time t _(n−1). Calculates the angle of the object 10 with respect to the reference coordinate system at time t _{(n + 1)} based on the actually measured position at time t _{(n + 1)} and the updated reference information.
[Selection] Figure 1

Description

  The present invention relates to a motion tracker device (hereinafter also referred to as an MT device) that measures a current angle of an object with respect to a camera coordinate system set in a camera device. The present invention relates to a head motion tracker device (hereinafter also referred to as an HMT device) that measures the current angle (that is, the current head angle) of a helmet with a head-mounted display device used in a moving body such as a game machine or a vehicle. Etc.).

  Techniques for accurately measuring the current angle of an object that changes from moment to moment are used in various fields. For example, in a game machine, in order to realize virtual reality (VR), an image is displayed by using a helmet with a head-mounted display device. At this time, it is necessary to change the image according to the current angle of the helmet with a head-mounted display device, and an HMT device is used to measure the current angle of the helmet with a head-mounted display device. Yes.

  Also, in the rescue operation by the rescue flying boat (aircraft), it is locked when the aiming image displayed by the helmet with a head mounted display device corresponds to the rescue target so as not to lose sight of the discovered rescue target. By doing so, the position of the locked rescue target is calculated. At this time, in order to calculate the position of the rescue target, in addition to the latitude, longitude, altitude, and attitude of the flying object, it is necessary to measure the current angle of the pilot's head with respect to the coordinate system set for the flying object For this, an HMT apparatus is used.

  As such an HMT device, there is disclosed an optical type device in which a plurality of light emitters are attached as an optical marker group on the outer peripheral surface of a helmet with a head-mounted display device, and light from the light emitter is monitored by a camera device. (For example, refer to Patent Document 1). Specifically, on the outer peripheral surface of the head-mounted display device-equipped helmet, as an optical marker group, LEDs (light-emitting diodes) are attached to three locations so as to be separated from each other. The relative positions (reference positions) Cb of these three LEDs are stored in advance using design information (design drawings of a helmet with a head-mounted display device). These three LEDs are photographed simultaneously by two cameras (camera devices) that can be viewed stereoscopically and have a fixed installation location, so that the current three LEDs can be obtained according to the so-called triangulation principle. Position (actually measured position) Co is calculated. If three positions (positions of three LEDs) Co fixed to the helmet with a head-mounted display device can be identified by the camera device, the head-mounted display device can be obtained by referring to the stored design information Cb. It is possible to specify how much the attached helmet has rotated, and as a result, the current angle of the helmet with a head-mounted display device is calculated.

JP 2006-284442 A

However, when an LED is used as an optical marker, there is a problem in storing design information as initial information. Here, FIG. 6 is a cross-sectional view showing a schematic configuration of a general LED. The LED 50 includes a semiconductor unit 51, a reflection plate 52, and a lens 53. As shown in FIG. 6, there may be a difference produced during manufacture between the center obtained from the outer shape of the LED and the center where the LED actually emits light (physical light emission center of the LED). Even if the center (design information) Cb obtained from the outer shape of the LED is stored, a measurement error may occur. Also, when a pilot wears a helmet with a head-mounted display device, it is fixed to the head by tightening the chin strap of the helmet with a head-mounted display device. While the helmet deformed, the LED sometimes deformed. Furthermore, when the LED is used for a long period of time, deterioration with time and deterioration may occur.
Accordingly, an object of the present invention is to provide a motion tracker device that can calculate the angle of an object with respect to a camera coordinate system with high accuracy.

In order to solve the above problems, the motion tracker device of the present invention includes an optical marker group composed of three or more optical markers attached to an object, a camera coordinate system, and a light beam from the optical marker. A camera device that detects in stereoscopic view, an actual position calculation unit that calculates an actual position of each optical marker with respect to the camera coordinate system based on the detected light beam, and each optical marker with respect to the camera coordinate system A storage unit for storing reference information indicating a reference position, and an angle calculation unit for calculating an angle of the object with respect to the camera coordinate system at time t _n based on the actually measured position at time t _n and the reference information. A motion tracker device that detects an angular velocity acting on the object from time t _(n-1) to time t _n. And Sa, based on the measured positions of the angular velocity and the time t _n from the time t _(n-1) to time t _n, the true position calculation for calculating the respective true position of each optical marker in the time t _(n-1) And an update unit that changes the reference information using the true position at time t _(n−1) , and the angle calculation unit is based on the measured position at time t _{(n + 1)} and the updated reference information. The angle of the object with respect to the camera coordinate system at time t _{(n + 1)} is calculated.

Here, the “angular velocity sensor” means a sensor having three axes defined in the sensor itself and capable of detecting an angular velocity with reference to the three axes. For example, a gyro sensor or the like is used. A gyro sensor can generally ensure high accuracy in a short time, but cannot ensure high accuracy in a long time. Therefore, in the present invention, the reference information is used for updating (calibration).
The “angle of the object relative to the camera coordinate system” means an angle RL in the roll direction (rotation of the camera coordinate system with respect to the X axis), an angle EL in the elevation direction (rotation of the camera coordinate system with respect to the Y axis), and The angle AZ in the azimuth direction (rotation with respect to the Z axis of the camera coordinate system).

  According to the MT apparatus of the present invention, three or more optical markers are attached to the object. Then, the reference information indicating the reference positions Cb of the three or more optical markers with respect to the camera coordinate system is stored using design information (such as a design drawing of a helmet with a head-mounted display device). At this time, in the reference information Cb, there is a difference that occurs at the time of manufacture between the center obtained from the outer shape of the optical marker and the center where the optical marker actually emits light. An error occurs because the helmet deforms when tightening.

Then, when measuring the current angle of the object relative to the camera coordinate system, first, the camera apparatus, at time t _1, detects the light beam from the optical markers stereoscopic. The actual measurement position calculation unit calculates the position (measurement position) Co ₁ of the optical marker with respect to the camera coordinate system at time t ₁ based on the detected light beam. The angle calculation unit calculates the angle of the object with respect to the camera coordinate system at time t ₁ based on the measured position Co _{1 at} time t ₁ and the reference information Cb. At this time, since the reference information Cb as described above has assumed that the error has occurred, for performing calibration of the reference information Cb, the angular velocity sensor is subject to the measurement start time t ₀ to time t ₁ Detect the angular velocity acting on the. The true position calculation unit calculates the position (true position) Cg ₀ of the optical marker at the measurement start time t ₀ based on the angular velocity from the measurement start time t ₀ to the time t ₁ and the measured position Co _{1 at the} time t _1. To do. Then, the stored reference information Cb is changed to new reference information Cb ₀ using the true position Cg ₀ at the measurement start time t ₀ . At this time, in the new reference information Cb ₀ , there is a difference that occurred during the manufacture between the center obtained from the outer shape of the optical marker and the center where the optical marker actually emits light, There is no error due to deformation of the helmet when tightening the chin strap or the like.

Next, the camera apparatus, at time t _2, detects the light beam from the optical markers stereoscopic. The actual measurement position calculation unit calculates the position (measurement position) Co ₂ of the optical marker with respect to the camera coordinate system at time t ₂ based on the detected light beam. The angle calculation unit calculates the angle of the object with respect to the camera coordinate system at time t ₂ based on the actual measurement position Co _{2 at} time t ₂ and the new reference information Cb ₀ . At this time, since the reference information Cb ₀ has an error because the optical marker is deteriorated or deteriorated over time, the calibration of the reference information Cb ₀ is performed. detecting an angular velocity acting on the object from ₁ to time t _2. The true position calculation unit calculates the position (true position) Cg ₁ of the optical marker at time t ₁ based on the angular velocity from time t ₁ to time t ₂ and the measured position Co _{2 at} time t ₂ . Then, the stored reference information Cb ₀ is changed to new initial information Cb ₁ using the true position Cg ₁ at time t ₁ . At this time, with this new reference information Cb ₁ , there is no error due to aging or deterioration of the optical marker that has occurred until time t ₁ .

Next, the camera device detects the light beam from the optical marker by stereoscopic vision at time t ₃ . The actual measurement position calculation unit calculates the position (measurement position) Co ₃ of the optical marker with respect to the camera coordinate system at time t ₃ based on the detected light beam. Angle calculation unit based on the measured positions Co ₃ and new reference information Cb ₁ time t _3, and calculates the angle of the object relative to the camera coordinate system at time t _3. At this time, the reference information Cb ₁ has an error because the optical marker has deteriorated or deteriorated over time from the time t ₁ to the time t _3, so that the calibration of the reference information Cb ₁ is performed. Therefore, the angular velocity sensor detects the angular velocity acting on the object from time t ₂ to time t ₃ . The true position calculation unit calculates the position (true position) Cg ₂ of the optical marker at time t ₂ based on the angular velocity from time t ₂ to time t ₃ and the measured position Co _{3 at} time t ₃ . Then, the stored reference information Cb ₁ is changed to new reference information Cb ₂ using the true position Cg ₂ at time t ₂ . At this time, the new reference information Cb ₂ has no error due to deterioration or deterioration of the optical marker that has occurred until time t ₂ .

As described above, according to the MT apparatus of the present invention, the reference information is changed using the true position Cg _(n−1) at time t _(n−1) . That is, the calibration of the reference information is executed in real time. Thereby, the angle calculation part can calculate the angle of the target object with respect to the camera coordinate system with high accuracy.

(Means and effects for solving other problems)
The motion tracker device of the present invention is a camera device in which an optical marker group composed of three or more optical markers attached to an object and a camera coordinate system are set, and light rays from the optical marker are detected in a stereoscopic view. And an actual measurement position calculation unit for calculating an actual measurement position of each optical marker with respect to the camera coordinate system based on the detected light beam, and reference information indicating a reference position of each optical marker with respect to the camera coordinate system. a storage unit for storing, based on the measured position and the reference information of the time t _n, a motion tracker device and a angle calculator that calculates an angle of the object relative to the camera coordinate system at time t _n, an acceleration sensor for detecting an acceleration applied from the time _{t (n-1)} on the object until the time _{t n,} or time _{t (n-1)} Based on the measured positions of the acceleration and the time t _n to time t _n, the true position calculation unit for calculating a respective true position of each optical marker in the time t _(n-1), the reference information time t _{(n- 1)} , and the angle calculator is configured to change the camera coordinate system at time t _{(n + 1)} based on the actually measured position at time t _{(n + 1)} and the updated reference information. It is characterized in that the angle of the object with respect to is calculated.

Here, the “acceleration sensor” means a sensor in which three axes are defined in the sensor itself, and acceleration in a triaxial direction with reference to the three axes can be detected. For example, an acceleration sensor or the like is used. An acceleration sensor can generally ensure high accuracy in a short time, but cannot ensure high accuracy in a long time. Therefore, in the present invention, the reference information is used for updating (calibration).
The “angle of the object relative to the camera coordinate system” means an angle RL in the roll direction (rotation of the camera coordinate system with respect to the X axis), an angle EL in the elevation direction (rotation of the camera coordinate system with respect to the Y axis), and The angle AZ in the azimuth direction (rotation with respect to the Z axis of the camera coordinate system).

As described above, according to the MT apparatus of the present invention, the reference information is changed using the true position Cg _(n−1) at time t _(n−1) . That is, the calibration of the reference information is executed in real time. Thereby, the angle calculation part can calculate the angle of the target object with respect to the camera coordinate system with high accuracy.

  In the above invention, the object may be a helmet worn on a wearer's head, and the optical marker may be a light emitting diode.

The figure which shows schematic structure of the HMT apparatus which is one Embodiment of this invention. The perspective view of the helmet with a head-mounted display apparatus shown in FIG. The figure explaining the setting of a camera coordinate system (XYZ coordinate system). The figure explaining the movement operation | movement of the helmet with a head mounting | wearing type display apparatus. The flowchart explaining the measurement operation | movement by an HMT apparatus. Sectional drawing which shows schematic structure of general LED.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below, and includes various modes without departing from the spirit of the present invention.

FIG. 1 is a diagram showing a schematic configuration of an HMT device according to an embodiment of the present invention, and FIG. 2 is a perspective view of the helmet with a head-mounted display device shown in FIG. FIG. 3 is a diagram for explaining setting of the camera coordinate system (XYZ coordinate system). In the present embodiment, the current angle of the helmet 10 with a head mounted display device worn by the pilot P who rides on the vehicle 30 is calculated. That is, the HMT device 1 has a helmet coordinate system (X′Y ′) set in the helmet 10 with a head mounted display device worn by the pilot P with respect to the camera coordinate system (XYZ coordinate system) set in the vehicle 30. The angle (RL _n , EL _n , AZ _n ) of the Z ′ coordinate system) is calculated at a predetermined time interval Δto. The camera coordinate system (XYZ coordinate system) is based on a camera device 31 described later, and is stored in advance in the camera coordinate system storage unit 43, while the helmet coordinate system (X'Y'Z 'coordinate system). ) Is stored in the helmet coordinate system storage unit 44 using design information (such as a design drawing of a helmet with a head-mounted display device).
The HMT device 1 includes a helmet 10 with a head-mounted display device that is mounted on the head of the pilot P, a camera device 31 that is mounted on the vehicle body 30, and a control unit 20 that includes a computer. .

The helmet 10 with a head-mounted display device includes a display (not shown), a combiner 8 that guides the eyes of the pilot P by reflecting image display light emitted from the display, and an LED (optical marker) group. 7, a three-axis gyro sensor (angular velocity sensor) 9, and a chin strap 6. Note that the pilot P wearing the helmet 10 with a head-mounted display device can visually recognize the display image by the display and the front actual thing of the combiner 8.
The LED group 7 includes three LEDs 7a, 7b, and 7c that emit infrared light having different wavelengths and are separated from each other.

  Here, the helmet coordinate system (X′Y′Z ′ coordinate system) can arbitrarily determine the origin and the direction of each coordinate axis. In this embodiment, as shown in FIG. 2, the origin is the position of the LED 7a. Design information so that the forward direction is the X′-axis direction, the direction perpendicular to the forward direction is the Y′-axis direction, and the direction perpendicular to the X′-axis direction and the Y′-axis direction is the Z′-axis direction. The information is stored in the helmet coordinate system storage unit 44 according to (a design drawing of a helmet with a head-mounted display device). Further, the positions of the three LEDs 7a, 7b, 7c on the helmet coordinate system (X'Y'Z 'coordinate system) are also determined based on the design information (design drawing of the helmet with a head-mounted display device). It is stored in the storage unit 44.

When the helmet 10 with a head-mounted display device is attached to the pilot P, first, at a first (measurement start time t ₀ ), a predetermined position (for example, XYZ) on the camera coordinate system (XYZ coordinate system). The pilot P is instructed to place the head of the pilot P at a position where the coordinate system and the X′Y′Z ′ coordinate system coincide with each other, and helmet coordinates relative to the camera coordinate system (XYZ coordinate system) as an initial state (reference information) The angle (RL ₀ , EL ₀ , AZ ₀ ) of the system (X′Y′Z ′ coordinate system) is set to the reference angle (0, ₀ , ₀ ). At this time, the LEDs 7a, 7b, and 7c are arranged at the respective reference positions Cb, and the reference vector is the direction perpendicular to the plane formed by the LEDs 7a, 7b, and 7c with the origin as the position of the LED 7a. V (Cb) is also arranged in the reference direction. FIG. 4A is a perspective view showing the helmet 10 with a head-mounted display device in an initial state.

The three-axis gyro sensor 9 detects an angular velocity acting on the head mounted helmet 10 with a display device. The triaxial gyro sensor 9 is attached to the helmet 10 with a head-mounted display device so that the axis is aligned with the helmet coordinate system (X′Y′Z ′ coordinate system). Therefore, angular velocities (ω ′ _RL , ω ′ _EL , ω ′ _AZ ) in the roll direction (rotation with respect to the X ′ axis), the elevation direction (rotation with respect to the Y ′ axis), and the azimuth direction (rotation with respect to the Z ′ axis) are detected. The
The three-axis gyro sensor 9 may be attached to the origin of the helmet coordinate system (X′Y′Z ′ coordinate system) or at a position other than the origin of the helmet coordinate system (X′Y′Z ′ coordinate system). when attached multiplies the offset matrix M0 in order to determine the helmet coordinate system 'angular velocity at the position of the origin of the (coordinate system _{(ω X'Y'Z)' RL, ω} 'EL, ω' AZ) It is converted by a general calculation method or the like.

The vehicle body 30 is a cockpit on which the pilot P is boarded, and includes a seat 32 on which the pilot P is seated and a camera device 31.
The camera device 31 includes a first camera 31a and a second camera 31b. The shooting direction is directed to the helmet 10 with a head-mounted display device, and stereoscopic viewing of the helmet 10 with a head-mounted display device is possible. It is installed on the ceiling of the vehicle body 30 via a fixed shaft 31c so as to be separated by a possible constant distance d. As a result, a first image photographed by the first camera 31a is created, and a second image photographed by the second camera 31b is created.
Then, the direction angle α from the first camera 31a and the direction angle β from the second camera 31b are calculated from the positions of the LEDs 7a, 7b, and 7c displayed in the first image and the second image, and the first By using the distance d between the camera 31a and the second camera 32b, the positions of the LEDs 7a, 7b, and 7c with respect to the camera device 31 can be calculated by a so-called triangulation method (see FIG. 3). .

  At this time, a camera coordinate system (XYZ coordinate system) set in the camera device 31 is used so that the positions of the LEDs 7a, 7b, and 7c can be expressed in spatial coordinates. The camera coordinate system (XYZ coordinate system) can arbitrarily determine the origin and the direction of each coordinate axis. In this embodiment, as shown in FIG. 3, the direction from the second camera 31b to the first camera 31a is set to X. The axis direction is defined as the Y-axis direction that is perpendicular to the X-axis direction and leftward, and the Z-axis direction is defined as the direction that is perpendicular to the X-axis direction, perpendicular to the ceiling, and upward. It is preset in the camera coordinate system storage unit 43 so as to be defined as a midpoint between 31a and the second camera 31b.

Thus, three LED 7a on the camera coordinate system at time _{t n} (XYZ coordinate system), 7b, to calculate the actual position _{C n} of 7c, the origin and location of the LED 7a, formed by the LED 7a and LED7b and LED7c An actual measurement vector V (C _n ) in a direction perpendicular to the measured plane can be calculated. FIG. 4B is a perspective view showing the helmet 10 with a head-mounted display device at time t _(n−1) , and FIG. 4C shows the helmet with a head-mounted display device at time t _n . FIG. Then, by calculating how much the measured vector V (C _n ) has been rotated from the reference vector V (Cb) and the measured vector V (C _(n−1) ), the current state of the helmet 10 with the head mounted display device is calculated. The angles (RL _n , EL _n , AZ _n ) are calculated.

The control unit 20 is configured by a computer including a CPU 21, a memory 40, and the like, and performs various controls and arithmetic processes. Processing executed by the CPU 21 will be described separately for each functional block. An image acquisition unit 28, an angular velocity acquisition unit 23, an actual position calculation unit 24, a true position calculation unit 25, an update unit 26, and an angle calculation unit. 27 and a helmet coordinate system storage control unit 22. The memory 40 is formed with an area for storing various data necessary for the control unit 20 to execute processing, and a camera coordinate system storage unit 43 that stores a camera coordinate system (XYZ coordinate system) in advance. , A helmet coordinate system storage unit 44 for storing a helmet coordinate system (X′Y′Z ′ coordinate system) and a reference information storage unit 42 for storing reference information Cb _n .

The helmet coordinate system storage control unit 22 performs control to store the helmet coordinate system (X′Y′Z ′ coordinate system) in the helmet coordinate system storage unit 44 by an input operation of the operator. Specifically, the operator performs an input operation using design information (such as a design drawing of a helmet with a head-mounted display device), thereby changing the helmet coordinate system (X'Y'Z 'coordinate system) to the helmet coordinate system. The information is stored in the storage unit 44, and the design information (design of the helmet with a head-mounted display device) is also provided for the positions of the three LEDs 7a, 7b, and 7c on the helmet coordinate system (X'Y'Z 'coordinate system). The helmet coordinate system storage unit 44 stores it by performing an input operation using a diagram or the like.
The helmet coordinate system memory control unit 22, the measurement starting time t _0, an instruction to place the head of the pilot P to a predetermined position on the camera coordinate system (XYZ coordinate system). As a result, as the initial state (reference information) Cb, the angle (RL ₀ , EL ₀ , AZ ₀ ) of the helmet coordinate system (X′Y′Z ′ coordinate system) with respect to the camera coordinate system (XYZ coordinate system) is the reference angle ( 0, 0, 0), and by calculating from design information (design drawings of a helmet with a head-mounted display device, etc.), the LEDs 7a, 7b, 7c are arranged at the respective reference positions Cb. The vector V (Cb) is also arranged in the reference direction (see FIG. 4A).

The image acquisition unit 28 outputs a command signal for turning on the LED group 7, and acquires the first image and the second image at a predetermined time interval Δto by photographing the LED group 7 with the camera device 31. Take control.
The angular velocity acquisition unit 23 performs control to acquire angular velocities (ω ′ _RL , ω ′ _EL , ω ′ _AZ ) from the three-axis gyro sensor 9 at a predetermined time interval Δtg.

Measured position calculation unit 24, based on the second image of the first image and the time t _n of the time t _n, calculating the camera coordinate system with respect to the (XYZ coordinate system), each measured position Cn of the LED group 7, Control is performed to calculate the measured vector V (C _n ). Specifically, LED 7a that are displayed in a second image of the first image and the time _{t n} of the time _{t n,} 7b, the position of 7c, the direction angle α and the second camera 31b from the first camera 31a Is calculated, and the distance d between the first camera 31a and the second camera 32b is used to determine the measured position C _n of the LEDs 7a, 7b, 7c at time t _n by the triangulation method. calculate. Then, LED 7a at time _{t n,} 7b, from the measured position _{C n} of 7c, and calculates the measured vector V _{(C n)} at time _{t n} (see FIG. 4 (c)).

Based on the measured vector V (C _n ) calculated by the measured position calculating unit 24 and the reference information Cb _(n−2) stored in the reference information storage unit 42, the angle calculating unit 27 at time t _n . angle of the helmet coordinate system (X'Y'Z 'coordinate _{system) (RL n, EL n,} AZ n) the control of calculating the conduct. Specifically, although the details will be described later, the latest reference information Cb _(n−2) is called from the reference information storage unit 42. That is, the reference position Cb _(n-2) of the LEDs 7a, 7b, and 7c is called and the reference vector V (Cb _(n-2) ) is called. Then, how much the measured vector V (C _n ) is rotated is calculated from the reference vector V (Cb _(n−2) ). Thereby, how much it rotated from the angle (RL _(n-2) , EL _(n-2) , AZ _(n-2) ) of the helmet 10 with a head mounted display apparatus in time t _(n-2) . calculated, to calculate the angle of the head-mounted display device with the helmet 10 at time _{t n (RL n, EL n} , AZ n) a. In addition, when calculating the angle (RL ₁ , EL ₁ , AZ ₁ ) of the helmet coordinate system (X′Y′Z ′ coordinate system) at time t ₁ , the reference information Cb ₋₁ does not exist. The reference position Cb stored using the information is used.

The true position calculation unit 25 uses the angular velocities (ω ′ _RL , ω ′ _EL , ω ′ _AZ ) from time t _(n−1) to time t _n and the actual measurement vector V (C _n ), control is performed to calculate each true position Cg _(n-1) of the LED group 7 at time t _(n-1) . Specifically, the time _{t (n-1)} from to time _{t n} angular velocity _{(ω 'RL, ω' EL} , ω 'AZ) by integrating the, from to time _{t n} time _{t (n-1)} The angle movement amount (ΔRL, ΔEL, ΔAZ) of the helmet coordinate system (X′Y′Z ′ coordinate system) is calculated. Next, the true vector V (Cg _(n−1) ) at time t _(n−1) is returned by returning the measured vector V (C _n ) at time t _{n by} the amount of angular movement (ΔRL, ΔEL, ΔAZ). Is calculated. Then, from the true vector V (Cg _n) at time _{t (n-1),} calculates LED 7a, 7b, true position _Cg of 7c the _(n-1) at time _{t (n-1)} (see FIG. 4) .

The updating unit 26 uses the true position Cg _(n−1) at the time t _(n−1) as the new reference information Cb _{(n −1)} from the reference information Cb _(n−2) stored in the reference information storage unit 42. Control to change to _-1) is performed. Specifically, when the angle (RL _(n-1) , EL _(n-1) , AZ _(n-1) ) of the helmet coordinate system (X'Y'Z 'coordinate system) is used, the LEDs 7a, 7b, 7c Is stored in the reference information storage unit 42 as reference information Cb _{(n-1) so} as to exist at the true position Cg _(n-1) . That is, at time t _(n−1), it is assumed that the LEDs 7a, 7b, and 7c exist at the measured position C _(n−1), and the angle (RL _{(n (n} )) of the helmet coordinate system (X′Y′Z ′ coordinate system). _-1) , EL _(n-1) , AZ _(n-1) ) were calculated, but calibration was performed so that the LEDs 7a, 7b, 7c were actually located at the true position Cg _(n-1) . To do.

Next, the measurement operation for measuring the angles (RL _n , EL _n , AZ _n ) of the helmet 10 with a head-mounted display device by the HMT device 1 will be described. FIG. 5 is a flowchart for explaining the measurement operation by the HMT apparatus 1.
First, in the process of step S101, the operator performs an input operation using design information (such as a design drawing of a helmet with a head-mounted display device), whereby the helmet coordinate system (X′Y 'Z' coordinate system) and the positions of the three LEDs 7a, 7b, 7c on the helmet coordinate system (X'Y'Z 'coordinate system) are stored.

Next, in the process of step S102, the pilot P is wearing the head-mounted display device with the helmet 10, the time t _n and a measurement start time t _0, a predetermined position on the camera coordinate system (XYZ coordinate system) Place the head on.
Next, in the process of step S103, time t _{n is set} to time t _{(n + 1)} .

Next, in the process of step S104, the image acquisition unit 28 outputs the command signal for lighting the LED group 7, by taking LED group 7 by the camera device 31, the first image and the time of time _{t n} A second image of t _n is acquired.
Next, in the process of step S105, the angular velocity acquisition unit 23 acquires the angular velocities (ω ′ _RL , ω ′ _EL , ω ′ _AZ ) from the time t _(n−1) to the time t _n from the three-axis gyro sensor 9. To do.

Next, in the process of step S106, the measured position calculating unit 24 on the basis of the second image of the first image and the time _{t n} of the time _{t n,} the camera coordinate system with respect to the (XYZ coordinate system), LED 7a, 7b, 7c and of calculating the measured position _{C n,} to calculate the actual measurement vector V _{(C n).}
Next, in the process of step S _<b> 107, the angle calculation unit 27 calls the latest reference information Cb _(n−2) from the reference information storage unit 42. That is, the reference position Cb _(n-2) of the LEDs 7a, 7b, and 7c is called and the reference vector V (Cb _(n-2) ) is called. When calculating the angle (RL ₁ , EL ₁ , AZ ₁ ) of the helmet coordinate system (X′Y′Z ′ coordinate system) at time t ₁ , since the reference information Cb ₋₁ does not exist, the reference position Cb is used.

Next, in the process of step S108, the angle calculation unit 27 calculates how much the measured vector V (C _n ) has been rotated from the reference vector V (Cb _(n−2) ), and time t _(n−2). The amount of rotation from the angle (RL _(n-2) , EL _(n-2) , AZ _(n-2) ) of the helmet 10 with a head-mounted display device is calculated, and the head-mounted display device The angle (RL _n , EL _n , AZ _n ) of the attached helmet is calculated.
Next, in the processing of step S109, the true position calculation unit 25 integrates the angular velocities (ω ′ _RL , ω ′ _EL , ω ′ _AZ ) from time t _(n−1) to time t _n , An angular movement amount (ΔRL, ΔEL, ΔAZ) of the helmet coordinate system (X′Y′Z ′ coordinate system ₎ from t _(n−1) to time t _n is calculated.

Next, in the process in step S110, the true position calculating section 25 calculates the true vector _{V (Cg (n-1)} ) at time _{t (n-1),} LED7a at time _{t (n-1),} 7b 7c, the true position Cg _(n-1) is calculated.
Next, in the process of step S111, the updating unit 26 uses the reference information Cb _(n-2) stored in the reference information storage unit 42 as the true position Cg _(n-1) at time t _(n-1). It is changed to new reference information Cb _(n-1) and stored.

  Next, in the process of step S112, it is determined whether or not to end the measurement. When it is determined that the measurement is to be terminated, this flowchart is terminated. On the other hand, when it is determined not to end the measurement, the process returns to step S103. That is, the processes in steps S103 to S111 are repeated until it is determined that the measurement is to be ended.

As described above, according to the HMT device 1 of the present invention, the reference information Cb _(n-2) is changed using the true position Cg _(n-1) at time t _(n-1) . That is, the calibration of the reference information Cb _n is executed in real time. Thereby, the angle calculation unit 27 accurately calculates the angle (RL _n , EL _n , AZ _n ) of the helmet coordinate system (X′Y′Z ′ coordinate system) with respect to the camera coordinate system (XYZ coordinate system). Can do.

<Other embodiments>
(1) Although the three-axis gyro sensor 9 is used in the HMT device 1 described above, a configuration in which an acceleration sensor is used may be used. Note that the angular movement amounts (ΔRL, ΔEL, ΔAZ) are calculated from the acceleration sensor using a known calculation method.

(2) In the HMT device 1 described above, the LED group 7 emits infrared light having different wavelengths. However, all the LEDs emit light having the same wavelength, for example, It is possible to identify each other by using a method of individually tracking and distinguishing LEDs (for example, the LED identification method described in Patent Document 1) or by changing the shape of each LED. Good. The LED group may be configured to emit light other than infrared light.

(3) In the HMT device 1 described above, the LED group 7 is configured such that the three LEDs 7a, 7b, 7c are separated from each other. For example, the ten LEDs are separated from each other. It is good also as a structure attached.

  The present invention can be used for a head motion tracker device or the like for measuring a head angle of a crew member or the like with respect to a camera coordinate system set on a vehicle.

1 Head Motion Tracker (HMT) 7 LED (Optical Marker)
9 3-axis gyro sensor (angular velocity sensor)
10 Helmet with head mounted display (object)
24 actual measurement position calculation unit 25 true position calculation unit 26 update unit 27 angle calculation unit 31 camera device 42 reference information storage unit

Claims (3)

An optical marker group composed of three or more optical markers attached to an object;
A camera device in which a camera coordinate system is set and detects the light beam from the optical marker in a stereoscopic view;
Based on the detected light beam, an actual position calculation unit that calculates an actual position of each optical marker with respect to the camera coordinate system;
A storage unit for storing reference information indicating a reference position of each optical marker with respect to the camera coordinate system;
A motion tracker device comprising: an angle calculation unit that calculates an angle of an object with respect to the camera coordinate system at time t _n based on an actually measured position at time t _n and reference information;
An angular velocity sensor for detecting an angular velocity acting on the object from time t _(n−1) to time t _n ;
Based on the measured positions of the angular velocity and the time t _n from the time t _(n-1) to time t _n, each of the true position calculation unit for calculating the true position of each optical marker in the time t _(n-1),
An update unit that changes the reference information using the true position at time t _(n−1) ,
The angle calculation unit calculates an angle of the object with respect to the camera coordinate system at time t _{(n + 1)} based on the actually measured position at time t _{(n + 1)} and the updated reference information. . An optical marker group composed of three or more optical markers attached to an object;
A camera device in which a camera coordinate system is set and detects the light beam from the optical marker in a stereoscopic view;
Based on the detected light beam, an actual position calculation unit that calculates an actual position of each optical marker with respect to the camera coordinate system;
A storage unit for storing reference information indicating a reference position of each optical marker with respect to the camera coordinate system;
A motion tracker device comprising: an angle calculation unit that calculates an angle of an object with respect to the camera coordinate system at time t _n based on an actually measured position at time t _n and reference information;
An acceleration sensor that detects acceleration acting on the object from time t _(n−1) to time t _n ;
Based on the measured positions of the acceleration and the time t _n from the time t _(n-1) to time t _n, each of the true position calculation unit for calculating the true position of each optical marker in the time t _(n-1),
An update unit that changes the reference information using the true position at time t _(n−1) ,
The angle calculation unit calculates an angle of the object with respect to the camera coordinate system at time t _{(n + 1)} based on the actually measured position at time t _{(n + 1)} and the updated reference information. . The object is a helmet to be worn on a wearer's head,
The motion tracker device according to claim 1, wherein the optical marker is a light emitting diode.

Images