KR101686585B1 - A hand motion tracking system for a operating of rotary knob in virtual reality flighting simulator - Google Patents

Abstract

The present invention relates to a hand motion tracking system for driving a rotary knob in a flight virtual simulator environment, which includes a hand motion tracker (100). The hand motion tracker (100) includes: a finger tip position tracker (110) for measuring a three-dimensional position of a contact point of a thumb and an index finger; a hand orientation tracker (120) for measuring posture recognition of a hand (position information of the hand); and a finger joint angle tracker (130) for measuring a motion angle of the thumb and a joint of the index finger.

Description

TECHNICAL FIELD [0001] The present invention relates to a hand motion tracking system for driving a rotatable tap handle in a flight simulator environment,

The present invention relates to a hand motion tracking system, and more particularly, to a hand motion tracking system for driving a rotatable tap handle in a flight simulator environment.

As the computing environment is improved and various terminal devices can be electrically connected, studies on user friendly input / output devices are increasing. Unlike conventional computer monitors and TV screens, for example, input / output devices such as Galaxy Gear, Apple Watch, Google Glass and the like, which are recently worn on a wearer's wrist, glasses or the like and display data, have been developed. In addition, unlike conventional keyboards and mice, which are typical input devices, touch screens that provide intuitive input by touching the display screen with fingers are becoming popular. In this trend, input devices that receive inputs based on user's body movements, such as Nintendo Wii, Microsoft's Xbox Kinect, and the like, are being developed and deployed. These input devices may include a motion tracker for sensing the motion of the user.

In the case of the conventional motion tracker, the input signal can be provided by sensing the movement of a large joint such as the pelvis or the knee of a human body, or sensing the motion of the entire arm, using the whole body of the user as a sensing target. However, in the conventional motion tracker, the motion of each of the fingers included in the user's hand is detected, the fingers are bent and disappear, and the precise movements represented by the hands such as the overlapping and falling of two fingers can not be detected . On the other hand, in a virtual environment-based flight simulator, the user operates the airplane by operating buttons, switches, touch screens, and rotary tap handle. In order to operate buttons, switches, levers, etc. of a flight simulator configured in a virtual screen in such a three-dimensional virtual space, a simulator is operated through three-dimensional position recognition in which a user's finger touches a button. In the conventional motion tracker, There is a problem that it is difficult to integrate the movement of the hand and the movement of the finger joint simultaneously with the recognition of the three-dimensional position generated when the tap handle is operated, thus limiting the operation of the rotary tap handle in the flight simulator .

The finger tip position tracker measures the three-dimensional position through the contact point between the thumb and the index finger of the user. The finger tip position tracker measures the rotation of the user's hand. The hand motion tracker is composed of a hand direction tracker that can measure the movement angle of the finger and a finger joint angle tracker that can measure the movement angle of the finger node. The present invention provides a hand movement tracking system for driving a rotatable tap handle in a virtual flight simulator environment, which enables operation of a rotatable tap handle.

In addition, the present invention provides a precise simulation of various hand movements such as buttons, switches, levers, and rotary tap handle manipulation of a virtual environment-based flight simulator through integrated measurement of three-dimensional position recognition, Another object of the present invention is to provide a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment.

According to an aspect of the present invention, there is provided a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment,

A hand movement tracking system for driving a rotatable tap handle in a flight simulator environment,

In order to operate the rotary tap handle provided on the flight simulator simulator display, the user is required to recognize the three-dimensional position recognition of the user's hand caused by the rotation of the virtual rotary tap handle provided in the display screen And a hand motion tracker for performing an attitude recognition for measuring the rotation of the user's hand and an angle measurement recognition for measuring the movement of the finger joint.

Advantageously, the hand motion tracker comprises:

A finger tip position tracker for measuring a three-dimensional position of a thumb and index finger contact point for operating the rotatable tap handle;

A Hand Orientation Tracker for measuring the hand posture recognition (hand position information) according to the rotation of the user's hand to recognize the operation of rotating the rotatable tap handle; And

And a finger joint angle tracker for measuring a movement angle of the thumb and forefinger knobs to implement the operation of touching the rotatable tap handle.

More preferably, the finger tip position tracker comprises:

A reflective ball mounted on a fingertip of a user's thumb and index finger, respectively;

Two or more infrared (IR) cameras for detecting infrared rays reflected by infrared rays on reflection balls mounted on fingertips of the user's thumb and index finger, and sensing the reflection balls at different angles; And

And a finger tip tracker control unit for obtaining three-dimensional position information of the thumb and index finger of the user by using infrared rays detected from the infrared camera.

Still more preferably, the finger tip position tracker comprises:

The infrared camera can detect the three-dimensional position of the reflective balls within an error of 5 mm in a virtual space of 1 m (H) x 1 m (W) x 0.5 m (D).

More preferably, the hand direction tracker comprises:

A three-axis acceleration sensor, a three-axis angular velocity sensor, a three-axis geomagnetic sensor, and a signal processing embedded device for detecting changes in hand rotation along the yaw axis, roll axis and pitch axis of the user's hand And a small AHRS (Attitude Heading Reference System) including a processor.

Even more preferably, the hand direction tracker comprises:

The roll axis may be configured to have an accuracy of ± 7 °, the pitch axis to be ± 7 °, and the yaw axis to have an accuracy of ± 10 °.

More preferably, the finger joint angle tracker comprises:

(PIP) joint and an MCP (MetaCarpoPhalangeal) joint so as to measure the degree of bending of the fingers of the thumb and the index finger, and using the angle sensors, You can track the movement of the node.

Preferably,

The three-dimensional position information of the finger tip position tracker of the hand motion tracker, the hand position information of the hand direction tracker according to the rotation of the user's hand, and the motion angle information of the finger node of the finger joint angle tracker, And a system controller for analyzing the hand motion and providing the analyzed hand motion to the display screen.

According to a hand movement tracking system for driving a rotatable tap handle in a flight virtual simulator environment proposed in the present invention, a finger tip position tracker for measuring a three-dimensional position through a contact point between a thumb and a forefinger of a user, A hand direction tracker capable of measuring the rotation of the hand, and a finger joint angle tracker capable of measuring a movement angle according to the movement of the finger node, thereby constituting a hand motion tracker. In the virtual environment based flight simulator, It is possible to operate the rotary tap handle through precise tracking of the movement.

Further, according to the present invention, it is possible to realize a variety of hand motion precision such as buttons, switches, levers, and rotary tap handle operation of a virtual environment based flight simulator through integrated measurement of three-dimensional position recognition, Simulation can be enabled.

1 is a block diagram of a functional block of a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an embodiment of the present invention;
2 is a block diagram of a functional block of a finger tip position tracker in a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment in accordance with an embodiment of the present invention;
3 is a block diagram of a functional block of a hand direction tracker in a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an embodiment of the present invention.
4 is a diagram illustrating a configuration of a hand motion tracker in a hand movement tracking system for driving a rotatable tap handle in a flight virtual simulator environment according to an embodiment of the present invention.
5 is a view illustrating a configuration of a contact point of a rotatable tap handle in a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an exemplary embodiment of the present invention.
FIG. 6 is a diagram illustrating an installation structure of a finger tip position tracker in a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an exemplary embodiment of the present invention; FIG.
FIG. 7 illustrates an operational configuration and measurement values of a hand motion tracker in a hand motion tracking system for driving a rotatable tap handle in a flight simulator environment according to an exemplary embodiment of the present invention; FIG.
FIG. 8 is a diagram illustrating an operational configuration of a finger joint angle tracker in a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an exemplary embodiment of the present invention; FIG.
FIG. 9 is a diagram illustrating an overall operation configuration of a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an embodiment of the present invention; FIG.
10 is a view illustrating operation of a rotatable tap handle in a virtual reality-based flight simulator display screen in a hand movement tracking system for driving a rotatable tap handle in a flight virtual simulator environment according to an embodiment of the present invention;

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, in order that those skilled in the art can easily carry out the present invention. In the following detailed description of the preferred embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In the drawings, like reference numerals are used throughout the drawings.

In addition, in the entire specification, when a part is referred to as being 'connected' to another part, it may be referred to as 'indirectly connected' not only with 'directly connected' . Also, to "include" an element means that it may include other elements, rather than excluding other elements, unless specifically stated otherwise.

1 is a block diagram of a functional block of a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an embodiment of the present invention. Figure 3 is a block diagram of a functional block of a finger tip position tracker in a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment, FIG. 4 is a view showing a configuration of a functional block of a hand direction tracker in a hand movement tracking system for driving a rotatable tap handle; and FIG. 4 is a block diagram of a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an embodiment of the present invention. FIG. 5 is a block diagram illustrating a configuration of a hand motion tracker according to an embodiment of the present invention. In the hand motion tracking system for driving a rotatable tap handle in the flight simulator virtual environment, according to time is it shows the configuration of the contact point of the rotary type tap handle. 1 to 5, a hand movement tracking system for driving a rotatable tap handle in a flight virtual simulator environment according to an embodiment of the present invention includes a hand motion tracker 100, 200).

The hand motion tracker 100 includes a virtual rotatable tap handle 10 provided by a user through a display screen to operate the rotatable tap handle 10 provided on the flight simulator display screen The three-dimensional position recognition of the user's hand caused by the rotation in the state of being held in contact, and the angular measurement recognition for measuring the rotation of the user's hand and the movement of the fingertip . This hand motion tracker 100 may be implemented in the form of a glove that is worn on at least a portion of the user's hand incorporating finger tip position tracker 110, hand direction tracker 120, and finger joint angle tracker 130 . That is, it can be configured to measure the bending of the fingertips and the fingertips of the back and thumb and forefinger in the form of gloves.

The finger tip position tracker 110 of the hand motion tracker 100 is a configuration for measuring the three-dimensional position of the thumb and forefinger contact points for operating the rotatable tap handle 10. The finger tip position tracker 110 includes a reflective ball 111 mounted on the fingertips of the user's thumb and index finger, a reflective ball 111 mounted on the fingertip of the user's thumb and index finger, Two or more infrared (IR) cameras 112 for detecting infrared rays reflected by the infrared cameras 111 and detecting the reflected balls 111 at different angles, And a finger tip tracker control unit 113 for obtaining three-dimensional position information of the user's thumb and index finger using infrared rays. Here, the finger tip position tracker 110 detects the three-dimensional position of the reflective balls 111 in the virtual space of 1 m (H) x 1 m (W) x 0.5 m (D) As shown in FIG.

The hand direction tracker 120 of the hand motion tracker 100 is configured to measure the hand posture recognition (hand position information) according to the rotation of the user's hand in order to recognize the operation of rotating the rotatable tap handle 10 to be. The Hand Orientation Tracker 120 is disposed on the user's hand and measures the rotation of the hand when the virtual rotatable tap handle 10 is held and rotated. Here, the hand direction tracker 120 includes a three-axis acceleration sensor 121 for detecting a yaw axis of the user's hand, a change in hand rotation along a roll axis and a pitch axis, An attitude heading reference system (AHRS) including an angular velocity sensor 122, a three-axis geomagnetic sensor 123, and a signal processing embedded processor 124. In addition, the hand direction tracker 120 can be configured to have a precision of ± 7 ° for the roll axis, ± 7 ° for the pitch axis, and ± 10 ° for the yaw axis.

The finger joint angle tracker 130 of the hand motion tracker 100 is a structure for measuring the movement angles of the thumb and forefinger joints to implement the operation of touching the rotatable tap handle 10. [ The Finger Joint Angle Tracker 130 includes an angle sensor 131 at a PIP (Proximal Interphalangeal) joint and an MCP (MetaCarpoPhalangeal) joint so as to measure the degree of bending of the fingertip of the thumb and index finger And is configured to track the movement of the finger joints within an error range of less than 5 degrees using the angle sensors 131. [

The system control unit 200 receives the three-dimensional position information of the finger tip position tracker 110 of the hand motion tracker 100, the hand position information according to the rotation of the user's hand of the hand direction tracker 120, The hand movement information of the finger joint of the tracker 130 is received and the hand posture in the virtual environment of the flight simulator is detected, and the analysis of the hand motion is performed on the detected hand posture to provide it to the display screen. The system controller 200 may include a motion detector 201, a motion analyzer 202, and a display unit 203, as shown in FIG.

FIG. 6 is a diagram illustrating an installation configuration of a finger tip position tracker in a hand movement tracking system for driving a rotatable tap handle in a flight virtual simulator environment according to an embodiment of the present invention. 6 (a) shows the installation structure of the finger tip position tracker 110 integrated into the hand motion tracker 100, and FIG. 6 (b) shows the measurement result of the finger tip position tracker 110. FIG. That is, the finger tip position tracker 110 irradiates infrared rays from two or more infrared cameras with respect to the reflective balls 111 mounted on the fingertips of the wearer's thumb and index finger of the user, respectively, The infrared rays are provided to the finger tip tracker control unit 113 so that the three-dimensional position information of the thumb and the index finger of the user is displayed as a measurement result as shown in FIG. 6 (b). Here, the infrared camera 112 of the finger tip position tracker 110 detects the position of the reflection ball 111 in a virtual space of height, horizontal and vertical virtual space, i.e., 1 m (H) x 1 m (W) x 0.5 m (D) Dimensional position of the object can be detected with an error within 5 mm.

FIG. 7 is a diagram illustrating an operation configuration and measurement values of a hand motion tracker in a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an exemplary embodiment of the present invention. 7, the hand motion tracker 120 is disposed on the user's hand or the like to recognize the operation of rotating the rotatable tap handle 10, and the yaw axis of the user's hand and the roll Axis acceleration sensor 121, a 3-axis angular velocity sensor 122, a 3-axis geomagnetic sensor 123, and a signal processing embedded processor 124 for detecting changes in the rotation of the hand along the axis and the pitch axis, , And is configured to have an accuracy of ± 7 ° for the roll axis, ± 7 ° for the pitch axis, and ± 10 ° for the yaw axis.

FIG. 8 is a diagram illustrating an operational configuration of a finger joint angle tracker in a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an exemplary embodiment of the present invention. Referring to FIG. 8, the finger joint angle tracker 130 includes a PIP (Proximal Interphalangeal) joint and an MCP (MetaCarpoPhalangeal) joint angle so as to measure the degree of bending of the finger joint of the user's thumb and index finger. The movement angle of the fingertip is tracked within an error range of 5 degrees or less using the sensors 131. FIG. Here, the finger joint angle tracker 130 is provided with a torque driver at the position of the angle sensors 131 installed on the user's finger joint, and measures the driving bending of the torque driver by the angle sensor 131 at the movement angle.

FIG. 9 is a diagram illustrating an overall operation of a hand movement tracking system for driving a rotatable tap handle in a flight simulator environment according to an exemplary embodiment of the present invention. FIG. Figure 2 illustrates the operation of a rotatable tap handle in a virtual reality based flight simulator display in a hand movement tracking system for driving a rotatable tap handle in a virtual simulator environment. 9 shows a state in which the finger tip position tracker 110 and the hand direction detecting unit 110 are in a state in which the user holds the hand-motion tracker 100 and touches the rotating type faucet handle provided on the flight simulator display screen of the display unit 203, Hand motion information provided to the system controller 200 by the hand motion tracker 100 that incorporates the tracker 120 and the finger joint angle tracker 130 can be generated by the motion detector 201 of the system controller 200 in a virtual environment Direction, speed, and acceleration information to the display unit 203 through the motion analysis processing in the motion analysis unit 202, and the display unit 203 provides the hand, The graphical object of the user's hand motion is visually rendered through the visual rendering process in the environment and provided to the flight simulator display screen. 10 (a) shows a process of contacting the rotatable tripod knob 10 in a virtual reality-based flight simulator, and FIG. 10 (b) shows a process of contacting the rotatable tripod knob 10 in a virtual reality- And shows an operating state in which it is rotated while being held in contact.

As described above, the hand movement tracking system for driving the rotatable tap handle in the flight virtual simulator environment according to an embodiment of the present invention includes a finger for measuring the three-dimensional position through the contact point between the thumb and the index finger of the user, By configuring the hand motion tracker by integrating the tip position tracker, the hand direction tracker which can measure the rotation of the user's hand, and the finger joint angle tracker which can measure the movement angle according to the movement of the finger node, It is also possible to operate the rotary tap handle by precisely tracking the movement of the user 's finger in the flight simulator. In addition, it is possible to use the virtual environment - based flight simulator Buttons, switches, levers, and rotary tap handle operations. It becomes possible to perform precise simulation of hand motion.

The present invention may be embodied in many other specific forms without departing from the spirit or essential characteristics of the invention.

100: Hand motion tracker 110: Finger tip position tracker
111: reflective ball 112: infrared camera
113: finger tip tracker control unit 120: hand direction tracker
121: 3-axis acceleration sensor 122: 3-axis angular velocity sensor
123: 3-axis geomagnetic sensor 124: signal processing embedded processor
130: finger joint angle tracker 131: angle sensor
200: system controller 201: motion detector
202: motion analysis unit 203: display unit

Claims (8)

A hand movement tracking system for driving a rotatable tap handle in a flight simulator environment,
A user operates the rotating type faucet handle 10 provided on the flight simulator display screen by holding the virtual rotating type faucet handle 10 provided on the display screen, And a hand motion tracker (100) for performing a three-dimensional position recognition of the user's hand, an orientation recognition for measuring the rotation of the user's hand, and an angle measurement recognition for measuring the movement of the finger joint, However,
The hand motion tracker (100)
A finger tip position tracker 110 for measuring a three-dimensional position of a thumb and index finger contact point for operating the rotatable tap handle 10;
A Hand Orientation Tracker 120 for measuring the hand posture recognition (hand position information) according to the rotation of the user's hand to recognize the operation of rotating the rotatable tap handle 10; And
And a Finger Joint Angle Tracker 130 for measuring a movement angle of the thumb and forefinger knuckles to implement the operation of touching the rotatable tap handle 10,
The hand direction tracker (120)
A three-axis acceleration sensor 121, a three-axis angular velocity sensor 122, and a three-axis geomagnetism sensor 122 for sensing a change in hand rotation along the yaw axis, the roll axis, and the pitch axis of the user's hand, A sensor 123, and a small AHRS (Attitude Heading Reference System) including a signal processing embedded processor 124,
The hand direction tracker (120)
Characterized in that the roll axis is configured to have an accuracy of +/- 7 degrees, the pitch axis to +/- 7 degrees, and the yaw axis to have an accuracy of +/- 10 degrees. Hand Motion Tracking System.
delete The apparatus of claim 1, wherein the finger tip position tracker (110)
A reflective ball 111 mounted on the fingertips of the thumb and forefinger of the user, respectively;
A plurality of reflection balls 111 mounted on the fingertips of the user's thumb and index finger are irradiated with infrared rays to detect infrared rays reflected from the reflection balls 111 and two or more infrared rays IR) camera 112; And
And a finger tip tracker control unit (113) for acquiring three-dimensional position information of the thumb and index finger of the user using infrared rays detected from the infrared camera (112) Hand motion tracking system for driving a rotatable tap handle.
4. The apparatus of claim 3, wherein the finger tip position tracker (110)
Characterized in that the infrared camera (112) senses the three-dimensional position of the reflective balls (111) within an error of 5 mm in a virtual space of 1 m (H) x 1 m (W) x 0.5 m (D) A Hand Motion Tracking System for Driving a Rotating Tap Handle in a Virtual Simulator Environment.
delete delete The apparatus of claim 1, wherein the finger joint angle tracker (130)
(Proximal Interphalangeal) joint and an MCP (MetaCarpoPhalangeal) joint so that the degree of bending of the thumb and forefinger of the index finger can be measured. Wherein the movement of the finger joint is tracked within a tolerance range within a predetermined range of the fingertip movement.
The method according to any one of claims 1, 3, 4, and 7,
The three-dimensional position information of the finger tip position tracker 110 of the hand motion tracker 100, the hand position information according to the rotation of the user's hand of the hand direction tracker 120, Further comprising a system controller (200) for receiving hand movement angle information of a node and detecting a hand posture in a virtual environment of the flight simulator, analyzing the hand movement according to the detected hand posture, and providing the analysis result to a display screen. A hand motion tracking system for driving a rotatable tap handle in a flight simulator environment.

Images