JP4172307B2 - 3D instruction input device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To execute a function of a two-dimensional mouse such as click, double click or drag by use of three-dimensional information without using a mechanical mechanism. <P>SOLUTION: A space extending in the front of a screen is divided into three areas. When an LED 10 worn by a user goes and returns through the boundary between a semi-pressing layer and a full-pressing layer once in a certain fixed time while making the LED recognize the passage of the boundary of each area, the click operation of the two-dimensional mouse is executed, and when it goes and returns twice in the fixed time, the double click operation of the two-dimensional mouse is executed. When it is not returned to the semi-pressing layer for a fixed time after moved from the semi-pressing layer to the full-pressing layer, an object such as an icon is dragged, and when the LED 10 is returned to the semi-pressing layer, the operation of dropping the dragged object is executed. <P>COPYRIGHT: (C)2005,JPO&amp;NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional instruction input device, and more particularly to a three-dimensional instruction input device that inputs an instruction based on three-dimensional information of a light emitting means.
[0002]
[Prior art]
Conventionally, a two-dimensional mouse has been widely used as a device for inputting instructions to a personal computer or the like. The two-dimensional mouse instructs and inputs the xy coordinate value as a relative coordinate value to the personal computer. In recent years, a technique for inputting a third coordinate value using the two-dimensional mouse has been proposed.
[0003]
For example, a three-dimensional mouse can be easily achieved by attaching an LED as a light emitter to a finger and attaching a light receiving unit that receives light from the LED to the upper part of the screen of a personal computer (see, for example, Patent Document 1). . The sensor used here obtains the angle of the light source with respect to the sensor and obtains the three-dimensional position of the light source based on the principle of triangulation. The LED receives power as a light source from a personal computer, and a signal such as a click, which is the basic operation of a mouse, is sent to the basic software of the personal computer through a wire. That is, a switch that can be pressed with a finger is provided near the light emitter.
[0004]
In addition, by rotating two two-dimensional balls, it is possible to input not only the movement amount in the x and y axis directions, but also the angle and the rotation angle, and a pointing device that can input in the z axis direction ( For example, see Patent Document 2.), a first photosensor array in which light conversion elements are arranged in a first direction, and a second in which light conversion elements are arranged in a second direction different from the first direction. When the optical sensor array moves, a signal corresponding to the change in the output of the both is generated, thereby determining the amount of movement (such as a cursor) and the direction of movement (for example, Patent Document 3) See.) Is known.
[0005]
Furthermore, a remote control device (for example, refer to Patent Document 4) that recognizes the index with a CCD camera and interprets it as a command for a personal computer or the like, and a rotary switch that is rotated with a finger in an electronic pen are provided. A coordinate input electronic pen (see, for example, Patent Document 5) that determines the degree of change in graphical parameters (line thickness, color, shading, gray scale) and movement of an RFID attached to a user's wrist After being recognized by the antenna, the RFID movement pattern is interpreted as an input command to another device. Similarly, when there is one antenna, an instruction input system (for example, refer to Patent Document 6) that interprets a time-series pattern in which an RFID is not recognized as a command is known.
[0006]
In addition, Gyromouse of Gyration Inc. in the United States is a pointing device that has a built-in auto gyro and can change the direction of the laser by aerial operation, but its weight is relatively large and its size is relatively small for the gyro. Large and somewhat expensive.
[0007]
In recent years, a computer keyboard that applies the concept of half-pressing in camera shutter operation has been prototyped, taking advantage of the fact that characters cannot be input unless it is pressed hard like a camera shutter. It was done.
[0008]
[Patent Document 1]
JP-A-10-9812
[Patent Document 2]
JP-A-5-150900
[Patent Document 3]
JP-A-11-24832
[Patent Document 4]
JP-A-11-110125
[Patent Document 5]
JP 2000-47805 A
[Patent Document 6]
JP 2001-306235 A
[0009]
[Problems to be solved by the invention]
However, the above-described conventional three-dimensional mouse has the same operability as the two-dimensional mouse, as is clear from the fact that a signal such as a click is sent to the basic software of a personal computer by wire. In addition, other instruction input devices such as the above-described conventional pointing device and instruction input system have the same operability as a two-dimensional mouse.
[0010]
In addition, Gyromouse Presenter of Gyration Inc. of the United States also relies on mechanical operations with fingers, as with normal two-dimensional mice, for operations such as clicking. Further, even in a device in which a keyboard and a note pad are overlapped, the operation of the cursor is restricted by two-dimensional information, and does not use three-dimensional information.
[0011]
That is, there is a problem that the operability of the conventional instruction input device is restricted in two dimensions.
[0012]
The present invention has been made to solve the above-described problem, and performs functions of a two-dimensional mouse such as click, double click, and drag using three-dimensional information without using a mechanical mechanism. An object of the present invention is to provide a three-dimensional instruction input device capable of
[0013]
[Means for Solving the Problems]
  To achieve the above object, a three-dimensional instruction input device of the present invention receives light emitted from the light emitting means, light emitting means that can be worn on the hand or finger of the instruction input person,A measuring means for measuring the three-dimensional position of the light emitting means from the received light, and a plane substantially coinciding with a screen displaying predetermined information is defined as an xy plane, and a space in front of the screen is defined as the xy plane. 3 of the light emitting means obtained by the measuring means obtained by dividing into three areas of the first area, the second area, and the third area in the order close to the screen in the z-axis direction perpendicular to the plane. The way of passing the light-emitting means with respect to each boundary dividing the three regions is determined from the z-coordinate value of the three-dimensional position, and differs depending on the way of passing and appeals to at least one of the five senses of the instruction input person Execution means for executing functions of a two-dimensional mouse including selection, click, double-click, drag and drop so that feedback is applied, and feedback that appeals to the visual input of the instruction input person; A small window is displayed on the screen at a position on the xy plane of the light emitting means, and an image displayed on the screen on the xy plane of the light emitting means is displayed in the small window. An image of a predetermined area including the position is enlarged and displayed, and the enlargement ratio of the image displayed in the small window is changed according to the z coordinate value of the light emitting means, and the image displayed in the small window is changed. Execution means for changing at least one of hue, saturation, and brightnessAnd.
[0014]
  Further, the three-dimensional instruction input device of the present invention receives light emitted from the light emitting means and light emitting means that can be worn on the hand or finger of the instruction input person, and receives the three-dimensional position of the light emitting means. A plane that substantially coincides with a measuring unit that measures from the light and a screen that displays predetermined information is an xy plane, and a space in front of the screen is in a z-axis direction perpendicular to the xy plane. The three areas of the first area, the second area, and the third area are divided into three areas in order from the screen, and the three coordinates are obtained from the z coordinate value of the three-dimensional position of the light emitting means obtained by the measuring means. Judgment of how to pass the light emitting means with respect to each boundary that divides the region, and selection and click so that feedback varies depending on the way of passing and appeals to at least one of the five senses of the instruction input person , Double chestnut Click, execution means for executing the function of the two-dimensional mouse comprising a drag-and-dropIn this case, a small window is displayed at a position on the xy plane of the light emitting means on the screen as feedback appealing to the visual input of the instruction input person, and an image displayed on the screen is displayed in the small window. Among these, an image of a predetermined area including the position of the light emitting means on the xy plane is enlarged and displayed, and an enlargement ratio of the image displayed in the small window according to the z coordinate value of the light emitting means is set. Execution means for changing and changing the uneven state of the image displayed in the small windowAnd can also be configured.
  Furthermore, the three-dimensional instruction input device of the present invention comprises a light-emitting means that can be worn on the hand or finger of the instruction input person, light emitted from the light-emitting means, and a three-dimensional position of the light-emitting means. A measuring means for measuring from the received light and a plane substantially coincident with a screen for displaying predetermined information is defined as an xy plane, and a space in front of the screen is in a z-axis direction perpendicular to the xy plane. Are divided into three areas, a first area, a second area, and a third area, in order from the screen, and are obtained from the z coordinate value of the three-dimensional position of the light emitting means obtained by the measuring means. Determining how to pass the light emitting means with respect to each boundary dividing one region, and depending on the way of passing, and selecting feedback to appeal to at least one of the five senses of the instruction input person, Click, dub Execution means for executing functions of a two-dimensional mouse including click, drag and drop, and as a feedback appealing to the visual input of the instruction input person, a small amount is placed on the screen at a position on the xy plane. A window is displayed, and within the small window, an image of a predetermined area including a position on the xy plane of the light emitting means is enlarged among the images displayed on the screen, and the light emitting means is displayed. And executing means for changing the size or / and shape of the small window while changing the enlargement ratio of the image displayed in the small window in accordance with the z-coordinate value of the small window.
[0015]
The light emitting means can be worn by an instruction input person, and is preferably worn by a finger or a hand. The light emitting means is not particularly limited as long as it emits light, and may be, for example, an LED. The measuring means receives the light from the light emitting means and measures the three-dimensional position of the light emitting means from the received light.
[0016]
On the other hand, predetermined information is displayed on the screen visually recognized by the instruction input person. The execution means sets the plane substantially coincident with the screen as the xy plane, and sets the space in front of the screen in the order of the first area and the second area in the z-axis direction perpendicular to the xy plane from the screen. The area and the third area are divided into three areas. Further, the execution means determines how to pass the light emitting means with respect to each boundary of the three regions from the z coordinate value of the three-dimensional position of the light emitting means obtained by the measuring means, and differs depending on the way of passing, and indicates Provide feedback that appeals to at least one of the five senses of the input person.
[0017]
That is, by measuring the three-dimensional position of the light emitting means that changes according to the movement of the finger or hand of the instruction input person by the measuring means, the execution means can determine how to pass through each boundary of the light emitting means. Furthermore, since the execution means can make the instruction input person recognize the three-dimensional position of the light emitting means by feedback, the instruction input person confirms the position of the light emitting means, that is, the space in front of the screen is divided into three areas. While confirming the passing state with respect to each boundary to be divided, it is possible to move the finger or hand wearing the light emitting means.
[0018]
The execution means executes the function of the two-dimensional mouse including selection, click, double click, and drag and drop according to the way of passing through each boundary of the light emission means while applying feedback in this way.
[0019]
Therefore, according to the present invention, the function of the two-dimensional mouse can be executed by three-dimensional information without using a mechanical mechanism.
[0020]
The execution means of the present invention displays a small window at a position on the xy plane of the light emitting means on the screen as the feedback, and within the small window, among the images displayed on the screen, An image of a predetermined area including the position of the light emitting means on the xy plane is enlarged and displayed, and an enlargement ratio of the image displayed in the small window is changed according to the z coordinate value of the light emitting means. Therefore, it is possible to appeal to the vision of the instruction input person.
[0021]
The execution means displays a small window on the screen at a position on the xy plane as feedback. The shape of the small window is not particularly limited, and may be a rectangle or a circle. The executing means enlarges and displays an image of a predetermined area including the position of the light emitting means on the xy plane among the images displayed on the screen in the small window. Furthermore, the magnification of the image displayed in the small window is changed according to the z coordinate value of the light emitting means to appeal to the instruction input person's vision.
[0022]
In this way, by changing the enlargement ratio of the image displayed in the small window in accordance with the z coordinate value of the light emitting means, the instruction input person can recognize the position of the light emitting means in the z-axis direction.
[0023]
When the light emitting means approaches the boundary between the first area and the second area from the second area side, the execution means reduces the magnification rate as the boundary approaches the boundary. When the boundary passes, the instruction input person can recognize the boundary by setting the enlargement ratio to 1.
[0024]
Thereby, the instruction input person can recognize the state where the light emitting means is approaching the boundary due to the change in the enlargement ratio. When the enlargement ratio is 1, it can be recognized that the light emitting means is passing the boundary. Therefore, the instruction input person can clearly recognize the position of the light emitting means in the z-axis direction, and can appropriately move the finger or hand wearing the light emitting means to execute the function of the two-dimensional mouse.
[0025]
The execution means of the present invention can change at least one of hue, saturation, and brightness of an image displayed in the small window according to the z coordinate value of the light emitting means.
[0026]
Thereby, the instruction input person can recognize the position of the light emitting means in the z-axis direction.
[0027]
Furthermore, the execution means of the present invention continuously increases the brightness in the small window according to the movement of the light emitting means when the light emitting means moves from the third area toward the second area. When the brightness becomes a predetermined magnitude due to the movement of the light emitting means, the value is kept constant. When the light emitting means moves and exceeds a predetermined z coordinate value, the second The brightness is reduced again as it approaches the boundary between the area and the first area, and the brightness in the small window is made the same as the brightness of the corresponding partial image on the screen in a predetermined area before and after the boundary including the boundary. The image displayed on the screen can be seen as it is, and after the light emitting means moves to the first area through the boundary between the second area and the first area, Before the hue of The hue is changed to a hue different from that of the second area, and the brightness is rapidly increased as the light emitting means moves away from the boundary between the second area and the first area and approaches the screen, and the hue in the small window is apparent. It can be made to appeal to the vision of the instruction input person.
[0028]
Thereby, the attributes (hue, lightness, saturation, etc.) of the small window change according to the z coordinate value of the light emitting means, but the state where the image on the screen can be seen in the small window is maintained.
[0029]
Specifically, when the light emitting means moves from the third area toward the second area, the brightness of the small window is continuously and rapidly increased according to the movement of the light emitting means. Thus, the instruction input person can recognize that the light emitting means is moving toward the second area.
[0030]
Furthermore, the execution means maintains the lightness value constant when the lightness reaches a predetermined level due to the movement of the light emitting means. Thus, the instruction input person can recognize that the second area is moving.
[0031]
Further, when the light emitting means moves and exceeds a predetermined z coordinate value, the executing means decreases the brightness again as it approaches the boundary between the second area and the first area. Thus, the instruction input person can recognize that the light emitting means is approaching the first area.
[0032]
Further, the execution means makes the brightness in the small window the same as the brightness of the corresponding partial image on the screen when the light emitting means is in a predetermined area before and after the boundary including the boundary between the second area and the first area. To make the image displayed on the screen visible. Thus, the instruction input person can recognize that the light emitting means is located near the boundary.
[0033]
Further, the execution means changes the hue in the small window to a hue different from the second area after the light emitting means moves to the first area through the boundary between the second area and the first area. . Thereby, the instruction input person can recognize that the light emitting means has moved from the second area to the first area.
[0034]
Further, the execution means rapidly increases the brightness as the light emitting means moves away from the boundary between the second area and the first area and approaches the screen so that the hue in the small window becomes clear. Thus, the instruction input person can recognize that the light emitting means is moving in the first area.
[0035]
The execution means of the present invention can change the uneven state of the image displayed in the small window according to the z coordinate value of the light emitting means. In addition, the execution means of the present invention can change the size or / and shape of the small window according to the z coordinate value of the light emitting means.
[0036]
Note that the size of the small window is at least locally when exceeding the boundary area, and if the light emitting means such as an LED is not moving when the area having the same attribute is moving, the distance is further increased. A feeling becomes easy to recognize.
[0037]
Thereby, the instruction input person can recognize the position of the light emitting means in the z-axis direction.
[0038]
In the present invention, the outline color of the small window can be arbitrarily set. The execution means takes the exclusive OR of the arbitrarily set outline color of the small window and the color of the border area of the small window in the image displayed on the screen. The outline color can also be determined.
[0039]
As a result, even if the set outline color of the small window is the same as the color of the border area of the small window in the image displayed on the screen, the outline is displayed in a color different from that color. Therefore, it is possible to prevent the instruction input person from losing sight of the small window.
[0040]
Note that the execution means of the present invention is not limited to feedback by a small window, and as the feedback, when the light emitting means passes through each of the boundaries, the execution means appeals to the hearing of the instruction input person using sound. You can also
[0041]
Thereby, like the feedback by the small window, the instruction input person can recognize the position of the light emitting means in the z-axis direction.
[0042]
The execution means of the present invention executes a selection function in a two-dimensional mouse when the light emitting means passes the boundary between the first area and the second area from the second area side. Can do. Further, the execution means performs a selection function in a two-dimensional mouse when the light emitting means passes through the boundary between the first area and the second area in one direction once within a predetermined time. You may make it perform. The execution means executes a selection function in a two-dimensional mouse when the light emission means passes through another boundary defined in the second region in one direction once within a predetermined time. You may do it.
[0043]
The execution means of the present invention, when the light-emitting means passes through the boundary between the first area and the second area once in a predetermined time and then in the opposite direction, A click function in a two-dimensional mouse can be executed.
[0044]
Further, the execution means of the present invention is characterized in that when the light emitting means passes through the boundary between the first area and the second area in the same direction twice within a predetermined time, a double click in a two-dimensional mouse is performed. Can perform the functions.
[0045]
Further, in the present invention, the first area is divided into an area close to the screen and an area close to the second area, and the execution means includes the light emitting means that defines the boundary between the two areas. When passing in one direction, the drag function in the two-dimensional mouse can be executed, and when passing through the boundary in the reverse direction, the drop function in the two-dimensional mouse can be executed.
[0046]
As described above, the two-dimensional mouse functions such as selection, click, double-click, and drag-and-drop are executed according to the way of passing through each boundary of the light emitting means, so that it is not necessary to use a mechanical mechanism. .
[0047]
Further, the execution means of the present invention is such that when the light emitting means passes through a predetermined area close to the first area in the second area, the instruction input person inputs an instruction x The position on the -y plane can be maintained at the position on the xy plane of the light emitting means at the time when the predetermined area is entered.
[0048]
As a result, the position on the xy plane that is designated and input by the instruction input person when passing through a predetermined area close to the first area in the second area is fixed. A two-dimensional mouse function such as selecting an object such as an icon or clicking an object can be executed.
[0049]
In the present invention, the predetermined time may be changeable. Thereby, the usability of the instruction input person can be improved.
[0050]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0051]
FIG. 1 is a block diagram showing a configuration and data flow of a three-dimensional instruction input device according to an embodiment of the present invention. As shown in the figure, the three-dimensional instruction input device is a 3D measurement that measures a three-dimensional position of the LED 10 by receiving a light emitting body (LED) 10 mounted on a user's finger or hand and receiving light from the LED 10. Click, double-click, and drag and drop commands (referred to as mouse commands) are generated from the locus of the z-coordinate value among the coordinate values indicating the three-dimensional position measured by the device 12 and the 3D measuring device 12. And a 3D mouse command generation device 14 for outputting to the personal computer 22 together with the xy coordinate values. Further, the personal computer 22 includes a mouse driver 16 for inputting a mouse command together with an xy coordinate value from the 3D mouse command generation device 14, an image driver 20 for performing image processing, and a command input to the mouse driver 16. Based on this, an OS 18 that performs processing related to instruction input by starting the image driver 20 as necessary is stored.
[0052]
Here, the xy coordinate value is a coordinate value when a plane substantially matching the screen of the personal computer 22 (see FIG. 2) is the xy plane, and the z coordinate value is perpendicular to the xy plane. It is a coordinate value in the z-axis direction.
[0053]
Note that the hardware of the above-described three-dimensional instruction input device is configured using the position detection technique described in Japanese Patent Laid-Open No. 10-9812. Of course, any device capable of measuring a three-dimensional position may be used. However, according to the method described in Japanese Patent Laid-Open No. 10-9812, it can be realized at a relatively low cost.
[0054]
FIG. 2 is a view of the personal computer 22 as viewed from the side. The screen of the personal computer 22 is indicated by M. Of the three-dimensional space extending in front of the screen M of the personal computer 22, the layer closest to the screen M is a fully pressed lower layer, and the next closest layer is a fully pressed upper layer, and is collectively referred to as a fully pressed layer (first layer). The boundary between the two is called boundary B. Further, the two layers on the front side are referred to as a half-pressed layer (second layer), and are called a half-pressed lower layer and a half-pressed upper layer from the side closer to the screen M, respectively. The boundary between the two is called boundary A. The boundary between the half-push layer and the full-push layer is called a boundary K. The layer closest to the user, that is, the layer farthest from the screen M is referred to as a free layer (third layer). The boundary between the free layer and the half-push layer is called a boundary F.
[0055]
The free layer is a layer in which no action is taken only by moving the cursor in response to the movement no matter how the LED 10 is moved. This corresponds to a state in which the mouse is moved without a mouse click in the two-dimensional mouse.
[0056]
When the LED 10 is in the layer before the half-pressed layer (half-pressed upper layer), there is a change in the attribute of the extended cursor described later, and the user is selecting an object. When the LED 10 moves to the screen side of the half-pressed layer (half-pressed lower layer), the xy coordinate value of the cursor is fixed to the value when the boundary A is passed.
[0057]
Mouse commands are generated by the way the LED 10 passes between the half-push layer and the full-push layer.
[0058]
Here, the extended cursor will be described. In the present embodiment, unlike the conventional cursor, the extended cursor surrounds a small area including a point indicated by the conventional cursor on the desktop of the personal computer 22 (hereinafter referred to as a cursor point) with an arc or the like, It is displayed again as a small window. As shown in FIGS. 3A and 3B, a circular interior centered on the cursor point 30 is displayed in a small window (expanded cursor) 32. The conventional cursor 34 is displayed so as to overlap the inside of the extended cursor 32, particularly in the center.
[0059]
Note that FIG. 3A shows a state in which the expansion cursor 32 indicates the center portion of the two columns on the right side, and it can be seen that the desktop icon is somewhat enlarged.
[0060]
FIG. 3B shows the state of the extended cursor 32 on the screen when the LED 10 approaches the boundary K. As shown in the figure, the radius of the expansion cursor 32 is the smallest, and the image in the circle is the same as the background, that is, the enlargement ratio is 1.
[0061]
As described above, the extended cursor 32 changes from an enlarged state so that the camera is focused to the actual size or the radius becomes smaller as the LED 10 approaches the boundary K from the outside in the half-pressed layer. The focus metaphor is reminiscent of the fact that the aperture is reduced.
[0062]
The extended cursor 32 is displayed by the OS using the image driver 20. The conventional cursor 34 is displayed so that the OS overlaps the extended cursor 32 using the mouse driver 16.
[0063]
Next, a process flow of the three-dimensional instruction input device when a mouse command is generated will be described.
[0064]
As shown in FIG. 1, when the LED 10 is in front of the 3D measurement device 12, the position is measured by the 3D measurement device 12. Here, what is necessary for generating the mouse command is the z coordinate value of the LED 10. The z-coordinate value is measured by the distance from the 3D measuring device 12. If the position of the screen M of the personal computer 22 and the position of the 3D measuring device 12 coincide with each other, this distance is the distance from the screen M. As indicated by the arrow in FIG. 1, the direction extending forward is the positive direction of the z coordinate value.
[0065]
As the user moves the hand, the position of the LED 10 attached to the hand changes accordingly, and the three-dimensional coordinate value including the z coordinate value is sent to the 3D mouse command generation device 14 through the 3D measurement device 12.
[0066]
The 3D mouse command generation device 14 checks in what form each boundary region in front of the screen M has passed within a certain time from the time series of the z coordinate values. This time can be arbitrarily set by the user. In the 3D mouse command generation device 14, it is interpreted as three important commands of click, double click, and drag depending on the manner. The signal of the interpreted command is sent to the mouse driver 16 of the personal computer 22. At this point, the mouse driver 16 can receive the same data type whether it is a two-dimensional mouse or a three-dimensional mouse. Accordingly, the processing on the personal computer 22 side thereafter. In the present embodiment, since the extended cursor 32 is represented by a small window, the OS activates the image driver 20 or the like that processes the image as necessary, and is shown in FIGS. 3A and 3B. As described above, the extended cursor 32 is displayed on the screen M of the personal computer 22. The mouse driver 16 is also activated at the same time. The OS causes the mouse driver 16 to overlay the conventional cursor 34 on the image of the extended cursor 32 and display it on the screen M.
[0067]
The image inside the extended cursor 32 is not different from the original image when the LED 10 is in the free layer, and is enlarged when the LED 10 enters the half-pressed layer (the size of the extended cursor itself and the image displayed internally). Further, the magnification of the enlarged image becomes 1 as the LED 10 moves from the half-pressed layer to the fully-pressed layer, that is, toward the boundary K. In particular, the magnification is set to 1 even if the z coordinate value of the LED 10 slightly moves within a predetermined distance immediately before the boundary K. This is a reproduction of the half-pressed state of the shutter of the camera, and it is also expressed that the hardness of the shutter increases immediately before the shutter is lowered and there is no change.
[0068]
Hereinafter, the correspondence between the movement of the LED 10 and the generated mouse command will be described in detail with reference to FIG.
[0069]
FIG. 4 is a view of the personal computer 22 as viewed from above. Moreover, the thick solid line and dotted line which are shown in the figure have shown the locus | trajectory corresponding to the motion of LED10. The thick solid line indicates that the xy coordinate value of the cursor point 30 is fixed (locked), and the xy coordinate value is not fixed (unlocked) in the dotted line portion. Is shown.
[0070]
When the LED 10 enters the half-pressed lower layer from the upper half-pressed layer over the boundary A, the xy coordinate value of the cursor point 30 is locked at the value at the time of passing the boundary A. At this time, if an object is designated, if the LED passes K later, the object is selected regardless of the xy coordinate value of the passing point at the time of passage. That is, after the LED 10 moves and passes through the boundary K and does not reach the boundary B, and again passes through the boundary K in the reverse direction and returns to the upper half-press, the “click” command is sent to the mouse of the personal computer 22. It is sent to the driver 16 (trajectory 40).
[0071]
Further, after the LED 10 enters the lower half-pressed layer from the upper half-pressed layer over the boundary A, the LED 10 does not return to the upper half-pressed layer, passes the boundary K again, but does not reach the boundary B, and reverses the boundary K again. When returning in the direction, a “double click” command is sent to the personal computer 22. (Track 42).
[0072]
Further, when the LED 10 passes through the boundary A and the boundary K from the upper half-pressed layer and passes the next boundary B, it enters an area where dragging is possible (full pressed lower layer). At this time, the lock of the xy coordinate value of the cursor point 30 is released, and the object pointed to by the cursor point 30 in the fully pressed lower layer is dragged according to the movement of the xy coordinate of the LED 10. That is, a “drag” command is sent to the personal computer 22. When the LED 10 again passes through the boundary B in the opposite direction from the fully pressed lower layer, passes further through the boundary K, and returns to the half pressed layer, the xy coordinate value is locked at that time, and the object is dropped. That is, a “drop” command is sent to the personal computer 22 (trajectory 44).
[0073]
The correspondence between the lock state of the xy coordinate values and each boundary will be described in detail. Basically, the xy coordinate value is locked in any case when the LED 10 crosses the boundary A from the half-pressed upper layer to the half-pressed lower layer. In addition, the lock is released when it passes through the boundary A in the reverse direction in the case of a click, and when the boundary K is returned to the reverse direction for the second time in the case of a double click. Further, when the LED 10 enters the fully pressed lower layer beyond the boundary B, the xy coordinate is unlocked. This is because it is in a draggable area. After the object is moved by dragging, when the LED 10 returns to the uppermost full press again, it is locked again. This is a convenience for removing the object from the cursor. When the LED 10 returns to the half-push layer beyond the boundary K, the lock is released again. At this point in time, the LED 10 has not returned to the upper half-pressed layer, so that no object is selected.
[0074]
In FIG. 4, the locus of the LED 10 at the time of clicking and double-clicking appears to move in the x or y coordinate values, but these are exaggerated expressions, and are actually in almost the same position. In the solid line portion, the value for the mouse driver 16 of the personal computer 22 is a constant value.
[0075]
As described above, the space extending in front of the screen is divided into three areas, and the LED 10 worn by the user passes through the boundary of each area by an extended cursor (focal metaphor) that appeals to the user's vision. If the boundary between the half-push layer and the full-push layer is reciprocated once in a certain time while recognizing it, the click operation of the two-dimensional mouse is executed, and if it is reciprocated twice in a certain time, If a double-click operation of the two-dimensional mouse is executed and the half-pressed layer does not return to the half-pressed layer for a certain period of time even after moving from the half-pressed layer, an object such as an icon indicated by the cursor is dragged and When the object returns to the half-pressed layer, the dragged object is dropped, so that the basic mouse operations such as click, double-click, and drag are mechanical. Mechanisms can be performed by LED10 movement in the air without using the (three-dimensional coordinate value).
[0076]
In the above-described embodiment, the example in which the user recognizes that the LED 10 has passed the boundary in the three-dimensional space using the extended cursor 32 has been described. For example, a sound unique to the LED 10 passes through each boundary. It is also possible to generate an operation to help the operation. Furthermore, the maximum enlargement ratio in each layer, color information such as hue, and the size (radius, etc.) of the extended cursor 32 can be freely changed by the user according to his / her preference and usability.
[0077]
FIG. 5 is a diagram illustrating an example of changes in brightness and hue of the extended cursor 32 according to the z coordinate position of the LED 10. As shown in the figure, the hue changes from green to red at the boundary K. That is, when the user moves the LED 10 before and after the boundary K, the color changes, and the user's awareness of where the boundary is is increased. Further, the brightness is changed in the half-pressed layer (left side of the boundary K in FIG. 5) and the fully pressed layer (right side of the boundary K in FIG. 5). In particular, in the half-push layer, the magnification rate of the image captured by the extended cursor toward the boundary K is brought close to 1, and the color is brought closer to the hue of the image captured by the expanded cursor so that the focus is clearly visible. It is also possible to construct a metaphor.
[0078]
In addition, the boundaries and layers in the above-described embodiment are examples, and it is possible to delete the boundaries and combine the two layers. In this case, it is possible to set the same layer-dependent attributes for adjacent layers. Also, the user can freely set the z coordinate value that defines the boundary. Furthermore, by interpreting the meaning of each layer separately, for example, as an extreme example, it is possible to reverse the half-push layer and the full-push layer. In this case as well, in order to activate the mouse command, it is only necessary to determine which boundary the LED 10 has passed and how.
[0079]
Note that at least one of the hue, saturation, and brightness of the image displayed on the extended cursor can be changed according to the z coordinate value of the LED 10. Further, the uneven state of the image displayed on the extended cursor can be changed according to the z coordinate value of the LED 10. Furthermore, you may make it change the magnitude | size and shape of an extended cursor according to the z coordinate value of LED10.
[0080]
In addition, the color of the outline of the extended cursor may be arbitrarily set by the user. Furthermore, the color of the extended cursor outline may be determined by taking the exclusive OR of the arbitrarily set color of the extended cursor outline and the color of the extended cursor boundary in the image displayed on the screen. Good. As a result, when the color of the outline set in advance for the extended cursor is the same as the color occupied by the extended cursor outline in the image displayed on the screen, the color of the extended cursor outline is made different. The extended cursor is clearly visible.
[0081]
The user can also set the maximum allowable time until each operation is completed, but the click interval time required for identifying the click operation by the aerial operation can be longer than the set value of the two-dimensional mouse. preferable. Specifically, the double click exceeds 0.5 seconds that can be set by the basic software OS, and about 1.9 seconds is experimentally appropriate at present.
[0082]
【The invention's effect】
The three-dimensional instruction input device according to the present invention provides each of a plurality of divided regions on the three-dimensional space spreading on the front surface of the screen by feedback (change in focus metaphor, hue, etc.) appealing to at least one of the five senses. Two-dimensional mouse functions such as click, double-click, drag, etc., depending on how the light-emitting means pass through the boundary of each region of the light-emitting means without using a mechanical mechanism, allowing the user to recognize the passing state of the light-emitting means with respect to the boundary There is an effect that can be executed.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration and data flow of a three-dimensional instruction input device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating space division when a personal computer is viewed from the side.
FIG. 3A is a diagram showing a state in which the extended cursor indicates the center part of the two right-side icons toward the right side, and FIG. It is a figure showing the state of the extended cursor on a screen when approaching.
FIG. 4 is a diagram illustrating space division and movement of LEDs in space when a personal computer is viewed from above.
FIG. 5 is a diagram illustrating an example of changes in hue and brightness of an image in an extended cursor displayed on the screen.
[Explanation of symbols]
10 LED
12 3D measuring device
14 3D mouse command generator
16 Mouse driver
18 OS
20 Image driver
22 PC
32 Extended cursor

Claims (16)

A light emitting means that can be worn on the hand or finger of the instruction input person;
Measuring means for receiving light emitted from the light emitting means and measuring a three-dimensional position of the light emitting means from the received light;
A plane that substantially coincides with the screen displaying the predetermined information is defined as an xy plane, and a space on the front surface of the screen is a first area in order from the screen in the z-axis direction perpendicular to the xy plane. The light emission for each boundary that divides the three areas from the z coordinate value of the three-dimensional position of the light emitting means obtained by the measuring means, divided into three areas, the second area and the third area. Select, click, double-click, drag and drop to determine how to pass the means, and depending on the way of passing, and so that feedback is applied to at least one of the five senses of the instruction input person a means for executing the function of the two-dimensional mice containing, as feedback visually appealing of the instruction input person, a small window at a position on the x-y plane of said light emitting means to said screen In the small window, among the images displayed on the screen, an image of a predetermined area including a position on the xy plane of the light emitting means is enlarged and displayed, and z of the light emitting means is displayed. Execution means for changing an enlargement ratio of an image displayed in the small window according to a coordinate value and changing at least one of hue, saturation, and brightness of the image displayed in the small window ;
A three-dimensional instruction input device. A light emitting means that can be worn on the hand or finger of the instruction input person;
Measuring means for receiving light emitted from the light emitting means and measuring a three-dimensional position of the light emitting means from the received light;
A plane that substantially coincides with the screen displaying the predetermined information is defined as an xy plane, and a space on the front surface of the screen is a first area in order from the screen in the z-axis direction perpendicular to the xy plane. The light emission for each boundary that divides the three areas from the z coordinate value of the three-dimensional position of the light emitting means obtained by the measuring means, divided into three areas, the second area and the third area. Select, click, double-click, drag and drop to determine how to pass the means, and depending on the way of passing, and so that feedback is applied to at least one of the five senses of the instruction input person a means for executing the function of the two-dimensional mice containing, as feedback visually appealing of the instruction input person, a small window at a position on the x-y plane of said light emitting means to said screen In the small window, among the images displayed on the screen, an image of a predetermined area including a position on the xy plane of the light emitting means is enlarged and displayed, and z of the light emitting means is displayed. Execution means for changing an enlargement ratio of an image displayed in the small window according to a coordinate value and changing an uneven state of the image displayed in the small window ;
A three-dimensional instruction input device. A light emitting means that can be worn on the hand or finger of the instruction input person;
Measuring means for receiving light emitted from the light emitting means and measuring a three-dimensional position of the light emitting means from the received light;
A plane that substantially coincides with the screen displaying the predetermined information is defined as an xy plane, and a space on the front surface of the screen is a first area in order from the screen in the z-axis direction perpendicular to the xy plane. The light emission for each boundary that divides the three areas from the z coordinate value of the three-dimensional position of the light emitting means obtained by the measuring means, divided into three areas, the second area and the third area. Select, click, double-click, drag and drop to determine how to pass the means, and depending on the way of passing, and so that feedback is applied to at least one of the five senses of the instruction input person a means for executing the function of the two-dimensional mice containing, as feedback visually appealing of the instruction input person, a small window at a position on the x-y plane of said light emitting means to said screen In the small window, among the images displayed on the screen, an image of a predetermined area including a position on the xy plane of the light emitting means is enlarged and displayed, and z of the light emitting means is displayed. Execution means for changing an enlargement ratio of an image to be displayed in the small window according to a coordinate value and changing a size or / and a shape of the small window ;
A three-dimensional instruction input device. When the light emitting means moves from the third area toward the second area, the execution means increases the brightness in the small window continuously and rapidly according to the movement of the light emitting means, The value is kept constant when the brightness reaches a predetermined level due to the movement of the light emitting means, and when the light emitting means moves and exceeds a predetermined z coordinate value, the second region and the first The brightness is reduced again as it approaches the boundary with the area, and the brightness in the small window is displayed on the screen so that the brightness in the small window is the same as the brightness of the corresponding partial image on the screen in the predetermined area including the boundary. After the light emitting means moves to the first area through the boundary between the second area and the first area, the hue in the small window is changed to the first image. What is area 2? The light emitting means abruptly increases brightness as it approaches the screen away from the boundary between the second region and the first region, and the hue in the small window becomes clear. The three-dimensional instruction input device according to claim 1, which appeals to the visual sense of the instruction input person. When the light emitting means approaches the boundary between the first area and the second area from the second area side, the execution means reduces the magnification rate as the boundary approaches the boundary. The three-dimensional instruction input device according to any one of claims 1 to 4, wherein an instruction input person is made to recognize the boundary such that the enlargement ratio is 1 when the boundary passes. The three-dimensional instruction input device according to any one of claims 1 to 5 , wherein an outline color of the small window can be arbitrarily set. The execution means obtains an exclusive OR of the arbitrarily set outline color of the small window and the border color of the small window in the image displayed on the screen. The three-dimensional instruction input device according to claim 6 , wherein the color is defined. The three-dimensional according to any one of claims 1 to 7 , wherein the execution means appeals to the hearing of the instruction input person using sound as the feedback when the light-emitting means passes through the boundaries. Instruction input device. The execution means executes a selection function in a two-dimensional mouse when the light emitting means passes through a boundary between the first area and the second area from the second area side . The three-dimensional instruction input device according to claim 8 . The execution means executes a selection function in a two-dimensional mouse when the light emission means passes through another boundary defined in the second region in one direction once within a predetermined time. The three-dimensional instruction input device according to any one of claims 1 to 8 . The execution means captures the second area by dividing it into an area close to the first area and an area close to the third area, and the light emitting means determines the boundary between the two areas for a predetermined time. The three-dimensional instruction input device according to any one of claims 1 to 8 , wherein a function of selection in a two-dimensional mouse is executed when it passes once in one direction. The execution means is a two-dimensional mouse when the light emitting means passes through the boundary between the first area and the second area once in a predetermined time and then in the opposite direction. The three-dimensional instruction input device according to any one of claims 1 to 8 , wherein a click function is executed. The execution means executes a double-click function in a two-dimensional mouse when the light emitting means passes through the boundary between the first area and the second area in the same direction twice within a predetermined time. The three-dimensional instruction input device according to any one of claims 1 to 8 . The first area is divided into an area close to the screen and an area close to the second area,
The execution means executes a drag function in the two-dimensional mouse when the light-emitting means passes through the boundary between the two regions in one direction, and executes the drag function in the two-dimensional mouse when the light-emitting means passes in the opposite direction. The three-dimensional instruction input device according to any one of claims 9 to 11 , which executes a drop function. When the light emitting means passes through a predetermined area close to the first area of the second area, the execution means is on the xy plane that is input by the instruction input person. The three-dimensional instruction input device according to any one of claims 1 to 14 , wherein the position is maintained at a position on an xy plane of the light emitting means at the time when the predetermined area is entered. The three-dimensional instruction input device according to any one of claims 10 to 13 , wherein the predetermined time can be changed.

Images