KR102522142B1 - Wearable augmented reality device that inputs operation signal using a two-handed gesture and method of operating a wearable augmented reality device using a two-handed gesture - Google Patents

Abstract

The present invention relates to a wearable augmented reality device that inputs a manipulation signal using a gesture of both hands, and more specifically, to a wearable augmented reality device, comprising: a setting unit for setting gestures and manipulation events in correspondence with each other; a hand motion detection unit including proximity sensors mounted on both sides of the wearable augmented reality device and detecting a left hand motion and a right hand motion within a recognition range of the proximity sensor; a gesture detector detecting a gesture composed of a combination of a left hand gesture and a right hand gesture using the left hand gesture and the right hand gesture detected by the hand gesture detector; and a control unit extracting a manipulation event corresponding to the detected gesture according to a setting of the setting unit and controlling an augmented reality image according to the extracted manipulation event, wherein the left hand motion and the right hand motion are within the recognition range. It includes the presence or absence of a hand within the recognition range, and is defined by further combining at least one of a hand shape of a hand existing within the recognition range, a change in the presence or absence of the hand, and a hand movement within the recognition range. to be
According to the wearable augmented reality device for inputting a manipulation signal using a two-handed gesture and the method for operating the wearable augmented reality device using a two-handed gesture proposed in the present invention, a proximity sensor is mounted and the left hand motion and the right hand motion are within the recognition range. By detecting a gesture composed of a combination of and performing control according to a manipulation event corresponding to the detected gesture, real-time recognition can be quickly performed in an embedded state using a proximity sensor, and a simple operation using a combination of two hand gestures. It is possible to implement various functions with just one, and it is easy to learn and use with a manipulation method similar to the existing mouse input method by using manipulation events such as cursor movement, object selection and execution.

Description

A wearable augmented reality device inputting a manipulation signal using both hand gestures and a method of operating the wearable augmented reality device using both hand gestures DEVICE USING A TWO-HANDED GESTURE}

The present invention relates to a wearable augmented reality device and a method for operating the wearable augmented reality device, and more specifically, to a wearable augmented reality device for inputting a manipulation signal using a gesture of both hands and a wearable augmented reality device using a gesture of both hands. It's about how to operate.

Various wearable devices are being developed according to the trend of light weight and miniaturization of digital devices. A head mounted display, which is a kind of wearable device, refers to various devices that can be worn on a user's head to receive multimedia contents and the like. Here, the head mounted display (HMD) is worn on the user's body and provides images to the user in various environments as the user moves. Head-mounted displays (HMDs) are classified into see-through and see-closed types. The see-through type is mainly used for Augmented Reality (AR), and the closed type is mainly used for virtual reality (VR). reality, VR).

Since it is difficult to use an input method such as a conventional touch screen in a head-mounted display, an input method optimized for user interaction is required. As an input method that can be used in the augmented reality of the head-mounted display, there may be a button provided in the HMD, a separate input device connected to the HMD, or gesture recognition. Among them, gesture recognition can be said to be a suitable input method that can be used in augmented reality of a head-mounted display.

However, the method using gestures is completely different from the input methods of existing computers or portable terminals, and thus has limitations in that it is inconvenient for users to learn and difficult to remember various types of gestures.

On the other hand, as a prior art related to the present invention, Patent Registration No. 10-1700569 (Title of Invention: HMD device capable of gesture-based user authentication and gesture-based user authentication method of the HMD device, registration date: January 2017 23), etc. have been disclosed.

The present invention is proposed to solve the above problems of the previously proposed methods, and a proximity sensor is mounted, a gesture consisting of a combination of a left hand motion and a right hand motion is detected within a recognition range, and the detected gesture is responded to. By performing control according to the manipulation event that can be performed, it is possible to perform real-time recognition quickly in the embedded state using a proximity sensor, and since it uses a combination of two hand gestures, it is possible to implement various functions with simple actions, such as cursor movement, object selection and execution. A wearable augmented reality device that inputs a manipulation signal using two-handed gestures that can be easily learned and used in a manipulation method similar to the existing mouse input method using manipulation events such as and a method of operating the wearable augmented reality device using two-handed gestures It aims to provide

A wearable augmented reality device for inputting a manipulation signal using a two-handed gesture according to the features of the present invention for achieving the above object is,

As a wearable augmented reality device,

a setting unit that sets gestures and manipulation events in correspondence with each other;

a hand motion detection unit including proximity sensors mounted on both sides of the wearable augmented reality device and detecting a left hand motion and a right hand motion within a recognition range of the proximity sensor;

a gesture detector detecting a gesture composed of a combination of a left hand gesture and a right hand gesture using the left hand gesture and the right hand gesture detected by the hand gesture detector; and

A control unit extracting a manipulation event corresponding to the detected gesture according to a setting of the setting unit and controlling an augmented reality image according to the extracted manipulation event;

The left hand motion and the right hand motion,

It includes the presence or absence of a hand within the recognition range, and is defined by further combining at least one of a hand shape of a hand existing within the recognition range, a change in the presence or absence of the hand, and a hand movement within the recognition range. It is a structural feature.

Preferably, the manipulation event,

It can include cursor movement, object selection, and execution events.

Preferably, the hand shape is

It may be any one of a first shape in which a hand is spread out so that a palm is visible and a second shape in which a fist is clenched.

More preferably, the gesture detection unit,

Detecting a gesture composed of a combination of the left hand motion consisting of a hand member within the recognition range and the right hand motion consisting of the presence of a hand within the recognition range, the first shape, and the direction of the hand motion;

The control unit,

A movement event, which is a manipulation event corresponding to the detected gesture, may be extracted, and a control of moving a cursor by a predetermined amount in a direction of a hand movement constituting the right hand movement may be performed.

More preferably, the gesture detection unit,

Detecting a gesture composed of a combination of the hand presence within the recognition range, the left hand motion consisting of the first shape, and the right hand motion consisting of the hand presence within the recognition range, the first shape, and the direction of the hand movement; ,

The control unit,

A movement event, which is a manipulation event corresponding to the detected gesture, may be extracted, and a control may be performed to move a cursor while the left hand motion is maintained in the direction of the hand motion constituting the right hand motion.

More preferably, the gesture detection unit,

Detecting a gesture composed of a combination of the left hand motion consisting of the presence of a hand within the recognition range, the second shape, and a change in whether the hand exists, and the right hand motion consisting of a hand member within the recognition range;

The control unit,

A selection event, which is a manipulation event corresponding to the detected gesture, may be extracted, and a control of selecting an object at a cursor position may be performed.

More preferably, the gesture detection unit,

Detecting a gesture composed of a combination of the left hand motion and the right hand motion, each composed of the presence of the hand within the recognition range, the second shape, and a change in the presence or absence of the hand;

The control unit,

An execution event, which is a manipulation event corresponding to the detected gesture, may be extracted, and control of executing an object located at a cursor position may be performed.

In addition, a method for operating a wearable augmented reality device using a two-handed gesture according to the features of the present invention for achieving the above object is,

As a method of operating a wearable augmented reality device,

(1) setting gestures and manipulation events in correspondence with each other;

(2) detecting a left hand gesture and a right hand gesture within a recognition range of the proximity sensor using proximity sensors mounted on both sides of the wearable augmented reality device;

(3) detecting a gesture composed of a combination of the left hand motion and the right hand motion using the left hand motion and the right hand motion detected in step (2); and

(4) extracting a manipulation event corresponding to the detected gesture according to the setting in step (1), and controlling an augmented reality image according to the extracted manipulation event;

The left hand motion and the right hand motion,

It includes the presence or absence of a hand within the recognition range, and is defined by further combining at least one of a hand shape of a hand existing within the recognition range, a change in the presence or absence of the hand, and a hand movement within the recognition range. It is a structural feature.

According to the wearable augmented reality device for inputting a manipulation signal using a two-handed gesture and the method for operating the wearable augmented reality device using a two-handed gesture proposed in the present invention, a proximity sensor is mounted and the left hand motion and the right hand motion are within the recognition range. By detecting a gesture composed of a combination of and performing control according to a manipulation event corresponding to the detected gesture, real-time recognition can be quickly performed in an embedded state using a proximity sensor, and a simple operation using a combination of two hand gestures. It is possible to implement various functions with just one, and it is easy to learn and use with a manipulation method similar to the existing mouse input method by using manipulation events such as cursor movement, object selection and execution.

1 is a diagram showing the configuration of a wearable augmented reality device for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention.
2 is a view showing, for example, a manipulation event corresponding to a defined gesture in a wearable augmented reality device for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention.
3 to 6 are diagrams illustrating, for example, control of an augmented reality image according to a gesture in a wearable augmented reality device that inputs a manipulation signal using a two-handed gesture according to an embodiment of the present invention.
7 is a diagram showing a detailed configuration of a hand motion detection unit in a wearable augmented reality device for inputting a manipulation signal using a gesture of both hands according to an embodiment of the present invention;
8 is a view showing the overall configuration of a wearable augmented reality device for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention.
9 is a flowchart illustrating a method of operating a wearable augmented reality device using a two-handed gesture according to an embodiment of the present invention.

Hereinafter, preferred embodiments will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, in describing a preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the same reference numerals are used throughout the drawings for parts having similar functions and actions.

In addition, throughout the specification, when a part is said to be 'connected' to another part, this is not only the case where it is 'directly connected', but also the case where it is 'indirectly connected' with another element in between. include In addition, 'including' a certain component means that other components may be further included, rather than excluding other components unless otherwise specified.

1 is a diagram showing the configuration of a wearable augmented reality device 10 for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention. As shown in FIG. 1, the wearable augmented reality device 10 for inputting a manipulation signal using a gesture of both hands according to an embodiment of the present invention includes a setting unit 110, a hand motion detection unit 120, a gesture detection unit ( 130) and the controller 400.

That is, the setting unit 110, the hand motion detection unit 120, and the gesture detection unit 130 constitute the gesture input device 100, and the gesture input device 100 is one of the components of the wearable augmented reality device 10. As such, the detected gesture may be transmitted to the control unit 400 so that control of the wearable augmented reality device 10 according to the gesture may be implemented. The gesture input device 100 may detect a gesture in an embedded environment on the wearable augmented reality device 10 .

Here, the wearable augmented reality device 10 is worn on the head or head of the user, and the real world that the user wearing the wearable augmented reality device 10 is transmitted through the optical system and output from the display are displayed to the user through the optical system. It may be a device that allows users to experience augmented reality by receiving image information delivered to the pupil of the child. Here, the wearable augmented reality device 10 may be of various types such as glass type and goggles type, and if it is worn on the user's body to experience augmented reality through the user's field of view, the present invention regardless of its specific form or type It can serve as a wearable augmented reality device 10 of.

The wearable augmented reality device 10 for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention is equipped with a proximity sensor 121 and a gesture composed of a combination of a left hand motion and a right hand motion within a recognition range. By detecting and performing control according to the manipulation event corresponding to the detected gesture, real-time recognition can be quickly performed in an embedded state using the proximity sensor 121, and various functions can be performed with simple hand motions because a combination of two hand motions is used. can be implemented, and it can be easily learned and used with a manipulation method similar to the existing mouse input method using manipulation events such as cursor movement, object selection and execution.

Hereinafter, with reference to FIG. 1 , each component of the wearable augmented reality device 10 for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention will be described in detail.

The setting unit 110 may set gestures and manipulation events in correspondence with each other. Here, the gesture is composed of a combination of a left hand motion and a right hand motion, and the left hand motion and the right hand motion include whether or not the hand exists within the recognition range of the proximity sensor 121, and the hand shape and hand of the hand existing within the recognition range. It may be defined by further combining at least one of a change in the presence or absence of and a hand movement within the recognition range. At this time, a change in the existence of a hand or a hand movement within a recognition range may be detected as a change for a predetermined time.

2 is a diagram showing, for example, a manipulation event corresponding to a defined gesture in the wearable augmented reality device 10 that inputs a manipulation signal using a gesture of both hands according to an embodiment of the present invention. As shown in FIG. 2, the setting unit 110 of the wearable augmented reality device 10 for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention lists gestures and manipulation events corresponding to each other in a table. can be saved as Here, the hand shape may be any one of a first shape in which the hand is spread out so that the palm is visible and a second shape in which the hand is clenched into a fist.

Also, manipulation events may include cursor movement, object selection, and execution events. That is, by setting operation events such as cursor movement, click, double-click, etc. similar to mouse operation on the actual desktop, such as cursor movement, object selection, and execution, the user can separately learn or memorize operation events according to various gestures. No need, just a few simple hand gestures, you can understand the relationship between gestures and manipulation events and use them easily.

The hand motion detection unit 120 includes proximity sensors 121 mounted on both sides of the wearable augmented reality device 10, and can detect left hand motions and right hand motions within a recognition range of the proximity sensors 121. . That is, the hand motion detection unit 120 is configured to recognize hand motions of both hands using the proximity sensor 121, where the proximity sensor 121 is a photoelectric proximity sensor, a capacitive proximity sensor ( It may be at least one of a capacitive proximity sensor and an ultrasonic proximity sensor.

3 to 6 are diagrams illustrating, for example, control of an augmented reality image according to a gesture in the wearable augmented reality device 10 that inputs a manipulation signal using a two-handed gesture according to an embodiment of the present invention. As shown in FIGS. 3 to 6 , the hand motion detector 120 may include a left proximity sensor 121a that detects a left hand motion and a right proximity sensor 121b that detects a right hand motion. While wearing the wearable augmented reality device 10, the user may input a gesture using a combination of left and right hand motions of placing a hand on the proximity sensor 121 in front of the ear next to the face.

FIG. 7 is a diagram showing a detailed configuration of the hand motion detection unit 120 in the wearable augmented reality device 10 that inputs a manipulation signal using a gesture of both hands according to an embodiment of the present invention. As shown in FIG. 7 , the hand motion detection unit 120 of the wearable augmented reality device 10 for inputting a manipulation signal using a gesture of both hands according to an embodiment of the present invention includes a proximity sensor 121 as well as a proximity sensor. It may be configured to further include a proximity switch 122 for activating a gesture input using 121 and a motion recognition module 123 for recognizing a hand shape and/or hand movement when the gesture input is activated.

That is, when the user brings his hand close to the proximity sensor 121 on the side of the wearable augmented reality device 10, the gesture input mode is activated by the proximity switch 122, and in the gesture input mode, the proximity sensor 121 moves the hand The presence or absence of the hand and a change in the presence or absence of the hand may be recognized, and the motion recognition module 123 may recognize the hand shape and hand movement.

More specifically, the motion recognition module 123 may recognize a hand shape and/or a hand movement based on a degree of reflection of light emitted from the proximity sensor 121 . In the case of recognizing a hand gesture with only the proximity sensor 121, it can be quickly processed in an embedded environment without an additional sensor, and for efficient and accurate hand shape detection by the proximity sensor 121, the palm is visible rather than the complex shape of the hand. The gesture may be composed of a first shape, which is an open shape, and a second shape, which is a closed fist shape.

According to embodiments, the motion recognition module 123 may recognize a hand shape and/or a hand movement by processing a hand image captured by a side camera using a side camera that captures a hand image. In other words, the hand shape and direction of hand movement can be accurately detected by analyzing the hand image captured in the gesture input mode using image processing technology. At this time, the image processing technology may be processed through communication with a server. In this way, if the image processing technology is used, the accuracy limit of the proximity sensor 121 can be overcome and an accurate hand shape and hand movement direction can be detected.

The gesture detector 130 may use the left hand motion and the right hand motion detected by the hand motion detector 120 to detect a gesture composed of a combination of the left hand motion and the right hand motion.

The controller 400 may extract a manipulation event corresponding to the detected gesture according to the setting of the setting unit 110 and control the augmented reality image according to the extracted manipulation event.

Hereinafter, with reference to FIGS. 3 to 6 , as shown in FIG. 2 , control of an augmented reality image by gestures and manipulation events set in correspondence with each other will be described in detail as an example.

For the basic movement of the cursor movement as shown in FIG. 3, the gesture detection unit 130 consists of the left hand motion consisting of the hand member within the recognition range and the presence of the hand within the recognition range, the first shape, and the direction of the hand movement. A gesture composed of a combination of right hand motions may be detected. The controller 400 may extract a movement event, which is a manipulation event corresponding to the detected gesture, and perform control to move the cursor by a predetermined amount in the direction of the hand movement constituting the right hand movement. That is, the left hand is located outside the recognition range (left hand motion), and the right hand is spread out and gathered so that the palm is visible (first shape) to the right side of the wearable augmented reality device 10 (recognition range of the right proximity sensor 121b) and when a gesture is detected that moves the cursor once in the desired direction (right hand motion), the cursor may be moved by a predetermined number of pixels in accordance with the right hand motion direction on the augmented reality screen and then stopped.

For compound movement among cursor movements as shown in FIG. 4 , the gesture detection unit 130 includes the left hand motion consisting of the hand presence within the recognition range and the first shape, and the presence of the hand within the recognition range, the first shape and the hand movement. A gesture composed of a combination of right hand gestures composed in the direction of may be detected. The controller 400 may extract a movement event, which is a manipulation event corresponding to the detected gesture, and perform control to move the cursor while the left hand motion is maintained in the direction of the hand motion constituting the right hand motion. That is, based on the basic movement of the cursor shown in FIG. 4, the left hand is also placed on the left side of the wearable augmented reality device 10 (recognition range of the left proximity sensor 121a) in the same first shape as the right hand, and The hand remains motionless, and only the right hand moves. When a gesture is detected according to the basic movement method, the cursor is displayed on the edge of the screen in the input direction while the left hand is maintained in line with the right hand movement direction on the augmented reality screen. You can move the cursor until you reach it. In addition, if the direction of the movement of the right hand changes during cursor movement, the cursor can move in the corresponding direction, and if the left hand disappears outside the sensor recognition range during cursor movement, the moving cursor can stop moving.

For object selection as shown in FIG. 5 , the gesture detection unit 130 uses the left hand motion consisting of the presence of a hand within the recognition range, the second shape, and a change in whether or not the hand exists, and the right hand consisting of a hand member within the recognition range. A gesture composed of a combination of hand gestures may be detected. The controller 400 may extract a selection event, which is a manipulation event corresponding to the detected gesture, and perform control to select an object at the cursor position. That is, as shown in FIG. 5, the right hand is located outside the recognition range (right hand motion), and the left hand is clenched into a fist (second shape) on the left side of the wearable augmented reality device 10 (left proximity). When a gesture of moving a hand with a clenched fist within the recognition range to outside the recognition range (hand movement) is detected, a selection input can be received at the current position of the cursor on the augmented reality screen. . A manipulation event corresponding to such object selection may function as a single click of a mouse, such as inputting a selection at the current position of the cursor, selecting a window, or selecting an app icon.

In order to execute the object as shown in FIG. 6 , the gesture detection unit 130 includes a hand presence within the recognition range, a second shape, and a gesture composed of a combination of a left hand motion and a right hand motion each composed of a change in existence or nonexistence of the hand. can be detected. The controller 400 may extract an execution event, which is a manipulation event corresponding to the detected gesture, and perform control to execute the object at the cursor position. That is, as shown in FIG. 6, when a gesture in which both hands leave the recognition range in the second shape and then return is detected, a manipulation event corresponding to the current position of the cursor on the augmented reality screen is displayed in a window (windows). ), or double-clicking the mouse, such as opening or closing the app icon.

8 is a diagram showing the overall configuration of the wearable augmented reality device 10 for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention. As shown in FIG. 8 , the wearable augmented reality device 10 for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention includes a gesture input device 100 and a controller 400. and may further include an HMD frame 200, an optical display unit 300, a GPS module 500, a camera 600, a sensor unit 700, and a communication unit 800.

The HMD frame 200 is a frame configuration of the wearable augmented reality device 10 that can be worn on the head or head by the user. The HMD frame 200 may be configured in the form of a helmet or goggles having a frame structure through which light may enter while worn on the head by a user. Here, when the HMD frame 200 is formed in the form of a helmet, it may be composed of a helmet (not shown) worn on the user's head and a display frame (not shown) disposed in front of the helmet. In addition, when the HMD frames 200 and 200 are configured in the form of goggles, they are composed of a band frame (not shown) that can be worn on the user's head and a goggle frame (not shown) fastened and fixed to the band frame. can

The optical display unit 300 may be disposed in front of both eyes of the wearer to provide augmented reality or mixed reality to the wearer. That is, the optical display unit 300 is a component corresponding to the AR, VR or MR glasses of the wearable augmented reality device 10, and displays and wears an OLED that outputs image light so that image information can be provided to the wearer. It may be configured to include an optical system that is disposed in front of both eyes of the wearer wearing the type augmented reality device 10 and provides augmented reality by combining real world light and image light. In addition, the optical system is composed of a plurality of lenses and mirrors and may be implemented in various ways, such as an optical diffraction method, a beam splitter method, a pin mirror method, and the like.

The GPS module 500 may provide location information of the user, and the camera 600 may capture an image in the direction of the user's gaze. The control unit 400 generates image information to be provided to the user based on information collected from the GPS module 500, the camera 600, and other various sensors, and controls the image information to be transmitted to the display so that the user can enjoy augmented reality. Through this, it is possible to receive additional information about the external environment through an optimized screen.

The sensor unit 700 may include at least one or more sensors, and more specifically, an iris recognition sensor or the like. According to the embodiment, when a user wears the wearable augmented reality device 10 and authenticates the user, the controller 400 configures various settings such as illuminance, inter-pupillary distance (IPD), input interface, etc., stored according to each user. can be applied automatically. In this case, user authentication may use an iris recognition sensor included in the sensor unit 700 .

The communication unit 800 is installed on one side of the wearable augmented reality device 10 and can transmit and receive various signals and data to and from other wearable augmented reality devices 10 or servers. Here, the network used by the communication unit 800 is a wired network or a mobile communication network such as a Local Area Network (LAN), a Wide Area Network (WAN), or a Value Added Network (VAN). radio communication network), satellite communication network, Bluetooth, Wibro (Wireless Broadband Internet), HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution), 3/4/5G (3/4/5th Generation Mobile Telecommunication ) can be implemented in all types of wireless networks, such as

9 is a flow diagram illustrating a method of operating the wearable augmented reality device 10 using a two-handed gesture according to an embodiment of the present invention. As shown in FIG. 9, the method of operating the wearable augmented reality device 10 using a two-handed gesture according to an embodiment of the present invention includes setting a gesture and a manipulation event in correspondence with each other (S110), wearable Using the proximity sensors 121 mounted on both sides of the augmented reality device 10, detecting the left hand motion and the right hand motion within the recognition range of the proximity sensor 121 (S120), the left hand motion detected in step S120. and detecting a gesture composed of a combination of the left hand motion and the right hand motion using the right hand motion (S130), and extracting a manipulation event corresponding to the detected gesture according to the settings in step S110, and extracting a manipulation event according to the extracted manipulation event. It may be implemented by including the step of performing control of the augmented reality image (S140).

Since the details related to each step have been sufficiently described in relation to the wearable augmented reality device 10 for inputting a manipulation signal using a two-handed gesture according to an embodiment of the present invention, detailed descriptions will be omitted. .

As described above, according to the wearable augmented reality device 10 for inputting a manipulation signal using a gesture of both hands and the method of operating the wearable augmented reality device 10 using a gesture of both hands proposed in the present invention, the proximity sensor ( 121), detects a gesture consisting of a combination of a left hand motion and a right hand motion within the recognition range, and performs control according to a manipulation event corresponding to the detected gesture. Real-time recognition is possible, and since a combination of two hand gestures is used, various functions can be implemented with simple actions. can be used

The present invention described above can be variously modified or applied by those skilled in the art to which the present invention belongs, and the scope of the technical idea according to the present invention should be defined by the claims below.

10: wearable augmented reality device
100: gesture input device
110: setting unit
120: hand motion detector
121: proximity sensor
122 proximity switch
123: motion recognition module
130: gesture detection unit
200: HMD frame
300: optical display unit
400: control unit
500: GPS module
600: camera
700: sensor unit
800: communication department
S110: Setting gestures and manipulation events in correspondence with each other
S120: Detecting left hand gestures and right hand gestures within the recognition range of the proximity sensor
S130: Detecting a gesture composed of a combination of a left hand motion and a right hand motion
S140: Extracting a manipulation event corresponding to the detected gesture and performing control of the augmented reality image according to the extracted manipulation event

Claims (8)

As a wearable augmented reality device 10,
a setting unit 110 that sets gestures and manipulation events in correspondence with each other;
A hand motion detection unit 120 including proximity sensors 121 mounted on both sides of the wearable augmented reality device 10 and detecting a left hand motion and a right hand motion within a recognition range of the proximity sensor 121;
a gesture detector 130 that detects a gesture composed of a combination of a left hand gesture and a right hand gesture using the left hand gesture and the right hand gesture detected by the hand gesture detector 120; and
A control unit 400 extracting a manipulation event corresponding to the detected gesture according to the setting of the setting unit 110 and controlling an augmented reality image according to the extracted manipulation event,
The left hand motion and the right hand motion,
It includes whether the hand exists within the recognition range, and if the hand exists within the recognition range, at least one of a hand shape of the hand existing within the recognition range, a change in the presence or absence of the hand, and a hand movement within the recognition range. A wearable augmented reality device 10 that inputs a manipulation signal using a two-handed gesture, characterized in that it is defined by combining one more.
The method of claim 1, wherein the manipulation event,
A wearable augmented reality device (10) for inputting a manipulation signal using a two-handed gesture, characterized in that it includes cursor movement, object selection, and execution events.
The method of claim 1, wherein the hand shape,
A wearable augmented reality device 10 that inputs a manipulation signal using a two-handed gesture, characterized in that any one of a first shape in which the hand is open so that the palm is visible and a second shape in which the fist is clenched.
The method of claim 3, wherein the gesture detection unit 130,
Detecting a gesture composed of a combination of the left hand motion consisting of a hand member within the recognition range and the right hand motion consisting of the presence of a hand within the recognition range, the first shape, and the direction of the hand motion;
The controller 400,
Extracting a movement event, which is a manipulation event corresponding to the detected gesture, and performing control to move a cursor by a predetermined amount in the direction of the hand movement constituting the right hand gesture, characterized in that, a manipulation signal using a two-handed gesture Wearable augmented reality device 10 to input.
The method of claim 3, wherein the gesture detection unit 130,
Detecting a gesture composed of a combination of the hand presence within the recognition range, the left hand motion consisting of the first shape, and the right hand motion consisting of the hand presence within the recognition range, the first shape, and the direction of the hand movement; ,
The controller 400,
A two-handed gesture characterized by extracting a movement event, which is a manipulation event corresponding to the detected gesture, and performing control to move a cursor while the left hand gesture is maintained in the direction of the hand movement constituting the right hand gesture. A wearable augmented reality device 10 that inputs a manipulation signal using.
The method of claim 3, wherein the gesture detection unit 130,
Detecting a gesture composed of a combination of the left hand motion consisting of the second shape of a hand existing within the recognition range and a change in the presence or absence of the hand and the right hand motion consisting of a hand member within the recognition range;
The controller 400,
A wearable augmented reality device (10) for inputting a manipulation signal using a two-handed gesture, characterized in that a selection event, which is a manipulation event corresponding to the detected gesture, is extracted and control for selecting an object at a cursor position is performed.
The method of claim 3, wherein the gesture detection unit 130,
Detecting a gesture composed of a combination of the left hand motion and the right hand motion each composed of the second shape of a hand existing within the recognition range and a change in whether the hand exists,
The controller 400,
A wearable augmented reality device (10) for inputting a manipulation signal using a two-handed gesture, characterized in that an execution event, which is a manipulation event corresponding to the detected gesture, is extracted and a control for executing an object at a cursor position is performed.
As a method of operating the wearable augmented reality device 10,
(1) setting gestures and manipulation events in correspondence with each other;
(2) detecting a left hand gesture and a right hand gesture within a recognition range of the proximity sensor 121 using the proximity sensors 121 mounted on both sides of the wearable augmented reality device 10;
(3) detecting a gesture composed of a combination of the left hand motion and the right hand motion using the left hand motion and the right hand motion detected in step (2); and
(4) extracting a manipulation event corresponding to the detected gesture according to the setting in step (1), and controlling an augmented reality image according to the extracted manipulation event;
The left hand motion and the right hand motion,
It includes whether the hand exists within the recognition range, and if the hand exists within the recognition range, at least one of the hand shape of the hand existing within the recognition range, a change in the presence or absence of the hand, and a hand movement within the recognition range. A method of operating the wearable augmented reality device 10 using a two-handed gesture, characterized in that defined by combining one more.

Images