JPH07182101A - Graphic input device and method, graphic object operating method, graphic input signal supplying method - Google Patents

Abstract

(57) [Abstract] [Purpose] Detects the user's actions without the user having to attach a special tool, hold the tool, or restrain the user. [Structure] The CCD camera 110 detects a user's finger touching the panel 105 as a plurality of shadows. CCD camera 11
Since 0 can detect a plurality of contact portions caused by contact of a plurality of fingers at the same time, 0 can be used for detecting an input gesture. The panel 105 itself and the contact portion on the panel 105 are determined by the position of the shadow on the video image. The function of the CCD camera 110 is only to detect the presence of a shadow on the panel 105, and only two-dimensional image processing is required. Since the CCD camera 110 can process images on a plurality of panels 105 at the same time, it is possible to generate a multidimensional input signal. Furthermore, since this image has a large contrast, it is only necessary to separate light and dark areas.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a graphic input device in the field of interactive computer graphics. In particular, the present invention relates to a technique for optically detecting multiple objects on the screen of a graphic input device used to control interactive computer graphics, such as multiple fingers of a user.

The description of the present specification is based on the priority of the present application, US Patent Application No. 08 / 141,045 (1993, 1).
(October 26th application), and the content of the description of the US patent application constitutes a part of the present specification by referring to the number of the US patent application. To do.

Reference to Main Patent Application This patent application contains 19
This is a continuation-in-part application of patent application number 07 / 837,372 filed on February 14, 1992.

[0004]

2. Description of the Related Art An interactive computer graphics input device is, in a broad sense, a device for detecting and inputting a user's action. Such a graphics input device is operated by the user. Input devices in this category include a light pen, a mouse, a join stick, a trackball, a keyboard, and the like as those related to a personal computer. However, there is an input device that directly detects a user's action, and a touch screen on a computer screen is an example. Myron W. Kruege
"Artificial Reality II" by r (published in 1991) describes a graphics input device that uses a video camera to detect the movement of the user's hand or body. This Krue
The concept of VIDEO DESK described in the book by ger is
U.S. Pat.No. 4,843,568 (Title of Invention: Real Time Perc
eption of and Response to the Actions of an Unencu
mbered Paticipant / User, granted June 27, 1989). However, this input device has a mechanism in which a user's hand is placed on a screen on the surface of a desk that emits light, and an image of the screen and the hand is detected by a video camera on the top of the desk. Furthermore, when this input device is used, it must be used so as not to obstruct the field of view of the video camera that detects the movement of the hand, which is inconvenient and the input device itself is not simple.

Presented by Richard Greene at the SIGGRAPH Society in San Francisco, 1985
An input device called Drawing Prism uses a large transparent prism on the surface of the input device. This input device is ACM Vol.19, No.3, 1985 pp.103-110 and U.S. Pat. No. 4,561,017 (Title of Invention: Graphic Input Appara
tus, dated December 24, 1985), a video camera detects an image of a contact point between an input device surface and an input tool from a specified angle. But,
This input device utilizes the reflection and refraction of light on the surface of an insulator, requires the use of extremely precise and costly optical components, and the arrangement of these components also requires high precision.

Sensor Frame of Sensor Frame is an input device capable of detecting a plurality of objects which obstruct the view of a plurality of cameras diagonally arranged on the frame. This input device is disclosed in US Pat. No. 4,746,770 (Title of Invention: Method and Apparatus for Isolating and Mani
pulating Graphics Objects on Computer Video Monito
r, granted May 24, 1988). This input device can detect the positions of multiple fingers in the frame at the same time, but it must use multiple cameras.
In addition, this input device also includes the problem of "ghost images" that occur when detecting multiple objects.

A desirable graphics input device for a user is a free input method that is usually called a gesture, and a special tool is attached to the user, or the holding or other restraint of the tool is performed by the user. It is capable of detecting a user's motion and accepting a plurality of input motions at the same time without giving the input to the user. Furthermore, it is considered desirable to have an input device that can perform real-time processing in a normal computer system by using a new method that reduces the load related to image processing in enabling the above input performance. Finally, a control input device for three-dimensional operation that is applicable to three-dimensional computer graphics applications is desirable.

[0008]

A graphics input device according to a preferred embodiment of the present invention comprises a single or a plurality of translucent screens, and a video camera disposed behind the screens. This camera is set to detect a plurality of shadows of an object such as a finger touching the screen. These shadows are called "contact points". This method makes image processing extremely simple, and at the same time, provides a free input operation environment without any constraint on the user. Since the camera only needs to detect the shadow on the screen, the image processing may be two-dimensional. Furthermore, since the image has a very high contrast, the image processing only separates the dark portion and the light portion, and the graphics input environment according to the present invention is extremely natural and unconstrained for the user. . A camera is used to detect where the user touches multiple fingers on the screen surface. This method allows the generation of input commands with multiple gestures. By utilizing this characteristic, it is possible to realize a convenient, practical, and intuitive input device by utilizing the positions and gestures of a plurality of fingers of the user without restraining the user. Finally, a force sensor is attached between the device body and the screen, and the force applied to the screen is sensed to selectively generate the input command in the "Z" direction when operating the graphics display object. It is also possible to add.

The above and other technical features will be illustrated by the following detailed description of the invention and the accompanying drawings.

[0010]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 illustrates a preferred embodiment of the present invention. As shown in FIG. 1, the control device 100 is usually a cube in shape. Of course, this shape can be freely determined according to the application requirements of the user, but when used on Cartesian coordinates, the shape is preferably a cube.

In the control device 100, only the panel 105 is translucent and the other surface is opaque. The translucent panel is flat, but other shapes can be adopted depending on the application requirements. A small CCD camera 110 is installed in the control device 100, and the orientation and focus of the camera are adjusted so that the shadow of an object, for example, a user's finger can be detected on the surface of the panel 105. There is.

The translucent panel 105 allows some light to pass through,
It's not transparent enough to see the objects behind the panel and its background. Therefore, the panel 105 diffuses the light,
The image of the panel obtained by the CCD camera 110 is a uniformly gray image. However, when the user touches the surface of the panel 105 with a finger, the panel image obtained from the CCD camera 110 is blackened at the portion touched by the user's finger, and is illuminated by the touched portion of the finger and ambient light. It is easily distinguished from the rest of the panel. Therefore, the portion touched by the user is identified as the control object 115.

It is also possible to increase the illumination of the panel 105. As shown in FIG. 1, the light source 120 may be mounted outside the control device 100. A mirror 125 attached to the controller 100 is used to direct light onto the panel 105.

Since the CCD camera 110 recognizes the image on the back surface of the panel 105 in a dark environment inside the control device, it is desirable that the outer surface of the panel 105 be uniformly illuminated. That is, the mirror 125 is attached so that the diffused light is uniformly applied to the outer surface of the panel 105.

The CCD camera 110 is connected to the image processing computer 130. This computer 130 is CC
The image data obtained by the D camera is used to generate the graphics input signal of the interactive computer system. These input signals are output to a plurality of control objects 115.
The position of (finger contact point) is supplied at the same time,
It is used as an operation signal for an object displayed on the graphics display 135. Display 1
35 is, for example, the control object 1 on the surface of the panel 105.
An object corresponding to the 15 positions, for example, the display object 140 can be displayed. Further, the object 140 on the display 135 is the panel 105.
It moves according to the movement of the upper control object 115. When the user's finger moves upward on the surface of the panel 105, the display object 140 on the display 135 also moves upward. When the user's finger rotates on the surface of the panel 105, the display object 140 on the display 135 also rotates.

Further, the object 1 on the display
It is also possible to use 40 to manipulate other graphics display objects. For example, the display object 140 shown in FIG. 1 is in contact with another graphics display object 145. Using computer programming techniques, the display object 140 may be rotated and the display object 145 in contact with the display object 140 may be rotated.

Another preferred embodiment of the present invention is shown in FIG. The graphics input control device 200 has a cubic shape and is composed of a plurality of control surfaces. The shape of this device can be freely changed according to its application purpose.

In the control device 200, the panels 202, 204 and 206 are translucent, and the other panels are panels made of an opaque material. Semi-transparent panels 202, 20
4,206 is preferably flat, but other geometries may be used depending on the engineering application. Small size with wide-angle lens
The CCD camera 210 has a corner 212 inside the control device 200.
Located on panels 202, 204 and 206
It is used to detect an image of a shadow produced when an object, for example, a user's finger touches the outer surface of the. CCD
The position of the camera is preferably arranged at the apex 214 inside the device in order to detect the camera images of the panels 202, 204 and 206 uniformly, and when the control device has a three-dimensional shape, the wide-angle lens of the camera is at least 90 ° or more. Need a viewing angle of.

The operation of the preferred embodiment shown in FIG. 2 is similar to the controller shown in FIG. That is, the panel 202,
The material of 204 and 206 has a similar optical property to that of the panel 105, and is a translucent material that allows some light to pass through, but does not allow enough light to be seen as a background object. Therefore,
The panel diffuses the light and the typical image obtained by the CCD camera 210 is uniformly gray. However, when the user touches the outer surface of the panel 202 with a finger, the portion is clearly detected as black by the CCD camera 210, and the surface of the panel 202 illuminated by ambient light, that is, a portion not touched by the finger, is detected. Are recognized as different parts. Since this image is also obtained on the panels 204 and 206 in the same manner, the portion where the user touches these panels is recognized as the control object 215 by the CCD camera. It is also possible to supplement the irradiation light for the panels 202, 204, 206. Light sources 222, 2 are provided outside the control device 200.
FIG. 2 shows an example in which 24 and 226 are attached. Control device 20
The mirrors 232, 234, 236 attached to 0 are such that the light is directed to the semitransparent panel. Since the CCD camera 210 is used to detect a black portion of the panel in a dark environment inside the panel, it is desirable that the light is uniformly emitted from the front surface of the panel. That is, the mirrors 232, 234, 23
6 is used to illuminate the outer surface of these panels with diffused light.

The CCD camera 210 is coupled to an image processing computer 250, and the image from the CCD camera 210 is processed by this computer and used as a graphic input in an interactive computer system.
The input group including the plurality of control objects 215 is
It is used to manipulate display objects on the graphic display 260. For example, the graphic display 260 can display a graphic object corresponding to the control object 215 on the panel 202. The display object 270 is the panel 202
It moves according to the movement of the upper control object 215. When the user's finger moves above the surface of the panel 202, the display object 270 also moves upward according to this movement. When the user's finger rotates on the surface of the panel 202, the display object 270 also rotates on the display 260. Further, the display object 270 can be used to operate another display object. For example, the operation of rotating the display object 270 shown in FIG. 2 and rotating the rectangular graphic object 280 that is in contact with the display object 270 can also be realized by software. The same operation as this operation can be realized on the operation panels 204 and 206.

As shown in FIG. 2, the CCD camera 210 is arranged so as to detect the image of the inner surface of the panels 202, 204 and 206 (the panel 206 is on the rear surface of the cube, and the end portion of the panel 206 is the panel). Share with 202, 204). Figure 3 shows an example of a typical image captured by a CCD camera. As shown in FIG. 3, the panel 20
The images 2, 204, 206 are not rectangular, and the panel shape captured by the CCD camera depends on the shape of the panel itself, the position of the camera, its angle, or the characteristics of the camera lens. However, the image shape of panel 204 obtained in the preferred embodiment shown in FIG. 3 is triangular, and the image shape of panels 202 and 206 is trapezoidal. And the part of the panel surface that the user's finger touches is
In the image, control objects 305, 310, 315
And 320 simultaneously.

The control input system shown in FIG. 2 has a function of generating a gesture as a command, which is an extremely elaborate and intuitive input method. 4 (a)-(c) show the gesture input function. In FIG. 4A, the user is dragging two fingers horizontally on the panel 202.
Control objects 410 and 415 corresponding to two fingers
Is detected by the CCD camera 210, and its position and movement are recognized by the computer 250. The computer 250 displays the user's hands and fingers 420, 425 as graphic objects on the graphic display 260. A horizontally moving finger will be displayed. The movement of the fingers 420 and 425 on the display corresponds to the movement of the user's finger on the panel. The graphic display fingers 420 and 425 are used to operate the slide switch of the display object to provide an extremely intuitive and easy-to-understand interactive user interface.

A three-dimensional rotation command, that is, a "gesture" is shown in FIG. 4 (b). The user touches the surface of the panel 206 with two fingers, and the computer 250 recognizes the position and movement of the control objects 450 and 460 detected by the CCD camera 210. The computer 250 projects a graphic display rotation tool (user's finger) on the graphics display 260, and the graphics rotation tool fingers 465, 4 are displayed.
70 rotates on the yz axis plane. Here, the x-axis is defined as the horizontal direction, the y-axis is defined as the vertical direction, and the z-axis is defined as the direction perpendicular to the graphic display surface. Fingers 465 and 470 of the graphic rotation tool correspond to the movement of the user's finger. This movement is the handle 4 on the graphic display.
75 is operated by a rotating tool, and is another example of an interactive user interface which is extremely intuitive and easy to understand.

A three-dimensional grasping operation, that is, a gesture command is shown in FIG. 4 (c). User panel 204
The computer 250 recognizes the positions and movements of the control objects 480 and 482 detected by the CCD camera 210 by contacting two fingers with the. 25
0 is a tool for grabbing a graphic display object corresponding to the finger of the user.
It is projected to 0 and the finger 48 of the graphics tool
4,486 tries to grab an object on the xz plane. Fingers 484 and 486 of the graphic tool correspond to the movement of the user's finger. This movement is a manipulation of an object 488 on a graphic display with a virtual tool, and is a different example of a highly intuitive and easy-to-understand interactive user interface.

The image processing method obtained by the CCD camera 210 is shown in the flow chart of FIG. First, step 5
At 05, the CCD camera 210 acquires a two-dimensional image (grayscale image quality) of the three panels, for example, the image shown in FIG. In the preferred embodiment, this step is usually performed initially using the controller, with the user's finger not touching the panel during this step. However, it is also possible for this process to be acquired and the data updated while the controller is in use by any suitable method. The image obtained by this step is the computer 25
It is represented by gray scale brightness B _o (i, j) at 0 and is stored as two-dimensional matrix data.

Next, in step 510, the computer 250 starts to capture the input operation image data from the CCD camera 210. This image contains the position information of the control object of each panel due to the touch of the user's finger, and is recognized as the two-dimensional matrix data represented by the gray scale luminance X _k (i, j).

In step 515, the background image brightness data is subtracted from the input operation image brightness data for each corresponding pixel to standardize the input operation image data. Normalized image data Z obtained by this processing
_k (i, j) is used to identify the contact point on the panel.

This normalization process is mathematically expressed as follows.

[0030]

Z _k (i, j) = B _o (i, j) −X _k (i, j) where i = row index of two-dimensional image j = column index of two-dimensional image X _k (i , j) = i of input operation image data in time series k
Luminance of pixel in row j column Z _k (i, j) = i row j of normalized image in time series k
Intensity of Pixels in Row In step 520, the normalized image data is converted to a binary image (black and white image) using a threshold. here,
The grayscale brightness of the image data corresponding to the contact point of the user's finger always includes a larger brightness value than the brightness data corresponding to the non-contact point. Pixels having a brightness exceeding the threshold are set to "1", and pixels having a brightness lower than the threshold are set to "0". Of the pixels corresponding to the contact point of the finger, it is preferable to adopt the lowest brightness value as the threshold value.
The pixels corresponding to the contact points are all treated as "1", and all other parts of the panel are treated as "0" so that they can be clearly separated. In step 530, using the data obtained by identifying the contact point, the center of the portion and the position on the panel are identified, and the obtained data is stored in the computer.

In step 540, the position of the control object is compared with the position of the control object one step before in the image acquisition time series to determine whether the control object has moved or a new control object has appeared. The position of the known control object and the distance of the new control object are calculated to determine if they are in the proper distance range and the data is updated. If the new control object is not within the allowable range, the old control object is deleted and the new control object is entered. At step 550, the position data of the moved control object is updated, the appearance of the new control object is recognized, and the disappearance of the known control object is confirmed. This program is stored in the computer.

In step 540, the position of the control object is compared with the position of the control object one step before in the image acquisition time series to determine whether the control object has moved or a new control object has appeared. The position of the known control object and the distance of the new control object are calculated, it is determined whether it is in the proper distance range, and the data update is executed. If the new control object is not within the allowable range, the old control object is erased and the new control object is entered. At step 550, the position data of the moved control object is updated, the appearance of the new control object is recognized, and the disappearance of the known control object is confirmed. The program then returns to step 510 and continues to run as a loop. In the preferred embodiment, it is preferable to speed up the updating of the old and new control object data, to accommodate rapid changes in the position of the user's finger and to allow that data to be continuously displayed by the computer. , 1/10-1/30 update rate is a proper condition.

Steps 505, 510, 515 and 5
20 can be described in more detail by FIGS. 20 and 21. FIG. 20 (a) shows an image obtained by the CCD camera 210 and corresponds to step 505. FIG. 20B shows a luminance histogram, in which the y-axis is the number of pixels corresponding to each gray scale luminance width, and the x-axis is the gray scale luminance level (0 to 255) of the pixel.

FIG. 20C is an image corresponding to step 510 and shows three contact points of the user. 20 (d) is shown in FIG.
It is a luminance histogram corresponding to (c). FIG. 20E shows an image obtained by subtracting the image 19c from the image 19a and corresponds to step 515. FIG. 20 (f) is a luminance histogram corresponding to FIG. 20 (e). As shown in the figure, there may be no significant difference between the brightness of the background data and the brightness value of the pixel corresponding to the contact point of the user. In such a case, in the preferred embodiment, the process of automatic scaling is executed as the additional image processing step. This processing artificially expands the gray scale luminance histogram of the background pixel and the pixel at the contact point to the maximum scale range (0 to 255 range) by multiplying the pixel by an appropriate numerical value. For example,
If the maximum brightness value of the pixel corresponding to the touch point is 12,
Multiply this value by 20 to get 240. This automatic scaling method is an effective means for clarifying the separation between the touch point by the user's finger and the background data, and facilitates the separation by setting the threshold value in step 520. 21 (g) and 21 (h) show the image and the luminance histogram after the automatic scaling process, respectively. This process is a selective supplemental process of the standardization process in step 515. 21 (i) and
(j) corresponds to the image and the histogram after the processing of step 520 is completed, so that the background brightness and the brightness of the contact point portion with the finger can be clearly separated as discrete values “0” or “1”. A threshold is set.

FIG. 6 shows a functional block diagram of the system in FIG. In FIG. 6, the CCD camera 21
0 transfers the image data with gray scale luminance to the computer 250. The computer 250 is equipped with an analog-to-digital converter that converts an analog signal from the CCD camera into an 8-bit digital signal. The computer 250 also includes image processing software. This image processing software acquires an image for background input operation, normalizes the image, and executes binarization (white / black image) processing of the standardized image using a threshold value. This processing is performed by steps 505, 510, 515 and 520 in FIG.
(The threshold can be set in advance or can be changed appropriately). Further, the image processing software also executes the center position of the control object and the position tracking calculation of the object. . This process is performed by step 530 in FIG.
Corresponding to 540 and 550. The position data of all control objects are transferred to computer graphics application software 630, which draws the graphic objects on graphic display 270.

This graphic application software is CCD
A coordinate conversion process is executed to move the position coordinates of the control object in the image space of the camera to the position coordinates in the graphic screen space on the graphic display 270. The mathematical coordinate transformation formula used at this time depends on the application conditions assumed by the graphics application software. As an example of the coordinate conversion, there is a method of using a numerical table for realizing the conversion from the image space to the control space shown in FIG.
With this method, the position data defined in the table is CCD
It is indexed by the position of the image space obtained by the camera and usually contains three values. The first value specifies whether this location data is from panel 202, 204, or 206. For example, image point 30
The first value of 5 is the signal from panel 206, the first value of image point 310 is the signal from panel 204, and so on. The second and third values include position coordinate data of the touch point of the user on the panel. For example, image point 305
The second and third values of include the "Y" and "Z" coordinates of the touch point on panel 206. Also, the second and third values of image point 310 include the "X" and "Z" coordinates of panel 204, and the second and third values of image point 315 are the "X" and "X" of panel 202. Including the Y "coordinate. In this way, the coordinate data of the converted control object is finally converted into the graphic display 2.
It is used as coordinate data available on 70. This method can be stored as ROM table data in the image processing software, and such software has an advantage that the image space data shown in FIG. 3 can be easily understood by the programmer. In other words, by using this method, a programmer who develops graphic application software does not need to study the internal structure of the control device deeply, and the panel where the contact point exists from the obtained input signal and the position of the contact point on the panel. You can grasp the coordinates directly. However, in addition to this method, there is a method that enables coordinate conversion from the CCD image space to the graphic display space, and which method is used depends on the application content of the graphic application software 630 itself.

Further, the graphic application software 6
30 recognizes the movement of the control object as a command by the gesture and processes it. This function is as shown in FIGS. 4 (a)-(c). Furthermore, the number of contact points of the user's finger can also be used as an input command. For example, when there is one finger contact point, it is used as an input signal for moving the cursor on computer graphics, and the contact point is two.
In one case, it is used as a command to move or rotate the graphic object corresponding to the movement of the finger. When there are three finger contact points, the command is canceled, that is, used as UNDO. If there are four contact points,
For example, a command to delete an object on the graphic screen.

Steps 510 to 550 are shown in FIGS. 7 (a)-(d).
The image format is shown in. FIG. 7A shows an image obtained by the CCD camera 210, which is the camera image space. FIG. 7A shows an image after the execution of step 515. The position data of the contact point is unchanged, but the brightness is a binary image due to the processing using the threshold value.
The black dot shown in FIG. 7 (b) indicates the center position of the control object. FIG. 7 (d) corresponds to steps 540 and 550, and is the position of the current control object and 1 in the time series.
The position before the step is shown, and this data is used to determine which control object has moved to the new position.

FIG. 8 shows the graphic application software 63.
At 0, the direction of direct coordinate conversion from the camera image space to the graphic space of the graphic display 260 is shown. The camera image space 810 is transformed into a point 820 in the graphics space using an appropriate transformation table or mathematical formula. For example, the point 810 is converted into the point 820 by the warping method, which is one of the image processing methods. The distortion deformation method is expressed by the following polynomial.

[0040]

[Formula 2] X '= a ₁ + a ₂ * X + a ₃ * Y + a ₄ * X * Y + a ₅ * X
² + a ₆ * Y ² + ... Y '= b ₁ + b ₂ * X + b ₃ * Y + b ₄ * X * Y + b ₅ * X ² + b ₆ * Y
² + ... where X and Y are old coordinates, X'and Y'represent new coordinates after conversion, and coefficients a ₁ , a ₂ ..., b ₁ , b ₂ ... are desired between two images. It is properly determined according to the data mapping specifications.

Therefore, the control object on the panel 202 is directly connected to the XY of the graphics display 260.
Interpreted as a point on the plane, and the designated point on the panel and the point on the graphic display drawn after the coordinate conversion all have a one-to-one correspondence. Furthermore, panel 2
The contact point of the user's finger on 04 and 206 can also be used as Z-axis position data in the three-dimensional graphic space of the graphic display 260.

The image processing contents in the graphic application software 630 are further shown in FIG. Step 9
At 10, the coordinate exchange formula from the CCD camera image space to the graphic display space is set. In step 920, coordinate conversion of the control object is executed, and in step 930, the number of control objects and their position data are interpreted as a command corresponding to the application of the user.

FIG. 10 shows another preferred embodiment of a controller for detecting the force applied by the touch on the translucent panel and utilizing the data. The control device 1100 has the same operation function as the control device 200, but is different in that it is equipped with a force detection sensor. Force detection sensor 1102
1104, 1106, and 1108 are panel 206
To detect the force applied to. Similarly, the force sensor 111
0, 1112, 1114, and 1116 are panel 2
The force applied to 04 is detected. A force sensor applicable to this purpose is Interlink Electronics (Carpinter
(ia, CA) force detection center or Intelligent
There is a force sensor from Computer Music Systems (Albany, New York).

The signal detected by the force sensor can be effectively used for realizing three-dimensional control, and an example thereof is shown in FIG. In FIG. 11, the user touches the panel 2 with his finger.
The pressure applied to 02 can be used as a signal for moving the position of the tool 1210 of the graphic display in the Z-axis direction. That is, the user rotates the graphic object with a gesture of rotating the object with a finger, and further, the force sensors 1118 and 112 obtained by applying pressure to the panel 202 with the finger.
The signals from 0, 1122, and 1124 can be used to move the graphic object in the Z-axis direction at the same time.

FIG. 12 shows a new input control technique in the control device shown in FIG. One of the performance characteristic evaluation items of the interactive computer graphics input device which is already known in the engineering field is the ratio of control amount to display amount (control-t).
o-display: C / D ratio) is available. This C / D ratio is defined as the ratio of the amount of movement (control amount) of the user's hand or finger to the amount of movement (display amount) of the cursor on the graphics display. A large C / D ratio provides the user with precise cursor movement, but it is difficult to achieve large range or fast cursor movement. Conversely, a small C / D ratio provides large and fast cursor movement, but is unsuitable for precise cursor manipulation. To compensate for these conflicting advantages and disadvantages, the relative coordinate input device has a variable
The C / D ratio is adopted, and a small C / D ratio is set when the moving speed of the input device is fast, and a large C / D ratio is set when the moving speed of the input device is slow. is there. This variable C / D ratio is used in input devices such as mice, and by adopting this technology properly, the user can operate the computer without replacing the position of the hand operating the mouse.
It is possible to manipulate the position of the cursor on the CRT screen within the desired precision.

However, there are drawbacks to operating with this variable C / D ratio. That is, with this technique, it is impossible to position the cursor on all parts of the CRT screen while moving the cursor at a slow speed without replacing the position of the user's hand. In many three-dimensional computer graphics applications, the user is often able to perform manipulations of graphic objects gently and with high precision, without repositioning hands across the computer screen. May want. That is, in the interactive human-computer interface, it is most desirable that the technology satisfying the two requirements of high precision operation and high speed wide range operation.

FIG. 12 shows the controller shown in FIG. The control device 100 is an input device capable of operating a graphic object having six degrees of freedom including a two-dimensional cursor operation. Therefore, as described with reference to FIG. 1, the control device 100 is a device that simultaneously responds to a plurality of control objects (touch points of user's fingers).

As shown in FIG. 12, the area 1205 is a partial area on the panel 105 in the control device 100.
This area 1205 is hereinafter referred to as an object operation area. The object operation area 1205 corresponds to the partial display area 1210 on the graphic display 135. The user operates the object operation area 120 on the panel.
It is possible to move a control object (finger contact point) in 5 to operate any graphic object (including a cursor) on the graphic display area 1210. The object operation area 1205 and the graphic display area 1210 are set to have a perfect one-to-one correspondence in their positions. This correspondence is the object operation area 1
The center of 205 corresponds to the center of the graphic display area 1210, and the point at the upper left corner of the object operation area 1205 corresponds to the point at the upper left corner of the graphic display area 1210. Therefore, when the size of the region 1210 is extremely small, the C / D ratio of this control device becomes extremely large. This characteristic means that it is possible to realize an input function capable of operating a graphic object existing in the area 1210 with extremely high accuracy.

As an additional feature, the graphic display area 1210 can be moved rapidly in position, either with itself or with the area and graphic objects drawn within this area. This function is to move the display area 1210 on the graphic display 135 by using the area 1220 outside the object operation area on the control input device panel. When the user's finger touches the outer area 1220, the display area 1210 on the graphic display 135 moves in the vector direction from the center of the object operation area toward the user contact portion of the outer area. For example, as shown in FIG. 12, the object operation area 120
Touching the outer area 1220 above 5 with the finger causes the display area 1210 to move above the graphic display 135. By moving the display area in this manner, a graphic object that was not included before the movement can be included in the display area, and the graphic object can be operated through the panel of the control device. This function means that the display area 1210 can be moved at high speed and any graphic object on the graphic display 135 can be operated. It is also possible to simultaneously operate the graphic objects while moving the display area 1210 using a plurality of fingers. For example, a graphic object may be slowly rotated and quickly moved to another position.

Such a cursor and graphic object operation technique is based on the control device 200 as shown in FIG.
By using, it can be applied to operations on the xz axis plane and the yz axis plane. The interface technique shown in FIG. 12 is further illustrated in FIGS. 13 (a)-(e). In FIG. 13A, the graphic objects 1305, 1310, 1315, 1320, 132 are displayed on the graphic display 135.
5, and 1330 are depicted. Graphic objects 1320, 1325 and 1330 are located within display area 1210. Therefore, the graphic objects 1320, 1325, 1330 are the same as the control device 1
In 00, these objects can be operated on the graphic screen by touching the object operation area 1205 with a finger. As shown in Fig. 13 (a),
Since the user's finger touches the left outer area of the object operation area 1205, the display area 1210 is moving to the left. In FIG. 13B, the user has a panel 105.
In the upper object operation area 1205, a gesture for rotating the object is performed, and the graphic object 1320 rotates in response to this command. In FIG. 13C, the user's finger touches the upper outside area of the object operation area 1205 on the panel 105, and the display area 1210 correspondingly moves upward. In FIG. 13D, the graphic object 1305 is located within the display area 1210 and is rotating in response to the rotation gesture command in the object operation area 1205 on the panel 105. FIG.
(e) shows an interface function that executes two operations at the same time. In FIG. 13E, the user is touching the lower right outer area of the object operation area 1205 on the panel 105 at the same time as the gesture of grasping the graphic object 1305. In response to this command, the graphic object 1305 moves to the lower right of the graphic display 135 together with the display area 1210.

14 (a)-(c) show a cylindrical control device. As shown, the controller 1500 includes an opaque base 1505, an opaque cylindrical outer frame 1510, and a translucent cylinder 1515. The cylinder 1515 is a semi-transparent disc 1
516 and is located at the top of the cylinder. Therefore, the control device 1500 is an opaque material except for a semitransparent cylinder.

The semi-transparent cylinder is designed to fit in the outer frame, as shown in the sectional view of FIG. 14 (b). A CCD camera 1520 having a wide-angle lens is arranged in the base 1505, and its mounting angle is set so that an image can be detected in the cylinder 1515 when a user touches the cylinder 1515.

In the command operation, the control device 1500
Has the same operability as the control device 100 shown in FIG. 1 or the input device 200 shown in FIG. That is, some light will pass through the semi-transparent cylinder, but not enough light to reveal the object itself outside it.
Therefore, light is diffused on the inner surface of the cylinder, and the image obtained by the CCD camera 1520 is usually a gray image with uniform brightness. If the user touches the outer surface of this cylinder with a finger, etc.,
The contact part such as a finger is shown in black in the image obtained by the CCD camera 1520, and the difference from the non-contact part of the cylinder is remarkable. Further, the force applied in the radial direction and the axial direction by the contact is detected by the force sensors 1525 and 1530, and the position where the force is applied in the radial direction can be detected from the position of the force sensor 1525.

The control input device 1500 is equipped with two force sensors for detecting the force applied to the cylinder 1515.
The force sensor 1525 is a sensor that is mounted between the cylinder 1515 and the outer frame 1510 and detects the magnitude of the force applied in the radial direction and the position where the force is applied. On the other hand, force sensor 1530
Is a sensor that detects the magnitude of the force applied in the radial direction of the cylinder 1515. FIG. 15 shows the CCD shown in FIG.
A typical image obtained from a camera is shown. Area 1610
Corresponds to the side of the cylinder. The area 1620 corresponds to the upper end of the cylinder, and the black points A and B and the points E and F become clear by referring to FIG. That is, referring to FIG. 16 showing the operation of the control device, the black dots A and B are the contact parts of the user's finger, and the two points at the upper end of the cylinder correspond to E and F. In input operation, Z
The operation input command of the axial position can be generated by sliding a finger on the side surface of the cylinder 1515 or by pushing the cylinder upper disc 1516 with the finger. The operation input command in the radial direction can be generated by pushing the cylinder 1515 with a finger or sliding the finger on the cylinder upper circular surface. The rotation input command can be generated by making a finger contact gesture that rotates the cylinder about the Z axis, or by making a twisting gesture on the cylinder upper disk 1516.

As shown in FIG. 17A, the cylinder 1515 can be changed to a hemisphere 1805, and even if the shape is changed to a hemisphere, the operability of the control device does not change.

The control device employing the hemisphere is useful for generating a rotation input command for an arbitrary axis in the three-dimensional space. For example, in virtual reality system or CAD / CAM system applications, the user needs to rotate the graphic object by any angle about any axis. In such a case, in the control device having a cubic shape, the three axes (x,
It has an intuitive and easy-to-understand shape when rotating a graphic object with respect to the (y, z axes), but it is not always appropriate for inputting rotation about an arbitrary axis. The hemisphere is natural for this required specification, and has a proper shape for intuitively generating a rotation input command. That is, the user can easily generate a rotation input command by sliding one or more fingers on the outer surface of the hemisphere.

FIG. 17B shows an image obtained by the CCD camera when the user touches the hemisphere 1805. CCD
Since the contact point 1810 obtained in the camera image is detected in the two-dimensional image space (xy space), the position coordinate of this contact point center is converted into the polar coordinate (r, θ) shown by the following equation.

[0058]

## EQU00003 ## r = SQRT (x ² + y ² ) θ = tan ^-1 (y / x) With this conversion, the contact point 1810 can recognize the locus of its position as a function of the radius r and the angle θ. . 18 (c)-(h) show a rotation input command generated by a finger touch on the hemisphere 1805.

In FIG. 18 (c), the user uses one finger with respect to the y-axis of the hemisphere and drags the finger to draw a circle on the surface of the hemisphere. The locus of this contact point is CC
It is recognized by the D camera image as shown in FIG. 18 (d), and the trajectory is such that the angle changes while keeping a constant radius r. An input operation by a finger contact that satisfies the characteristic of the locus, that is, the angle change and the constant radius is interpreted as a rotation input command for the y-axis.

FIGS. 18E and 18F show the rotation input command for the x axis. 18 (g) and 18 (h) show a rotation input command for the z axis. In these cases, the locus has a feature that the radius r changes and the angle θ is constant. A control input operation satisfying this condition is interpreted as a rotation input command for the x axis or the z axis.

FIGS. 19 (i) and 19 (j) show an operation for generating a rotation input command about an arbitrary axis. FIG. 19
In (i), the user generates a rotation input command about the z'axis. Here, the z ′ axis is an axis rotated and moved by an angle φ from the z axis. The locus shown in FIG. 19 (j) is shown in FIG.
Similar to 8 (f) and (h), but the angle θ changes by 90 + φ.

In FIG. 19 (k), the user draws an arbitrary locus with a finger, and a CCD image corresponding to this is shown in FIG. 19 (l). In FIG. 19 (k), the user has issued a simultaneous rotation input command about the x, y, and z axes, and the z variable (r, θ) of this locus continuously changes.

The light source and the semi-transparent panel used in the preferred embodiment can be replaced with an infrared light source and a filter that transmits only infrared rays. The infrared transmission filter used here is a special filter that reflects all light in the visible light region and transmits only light in the infrared wavelength region.

FIG. 22 shows a control device for generating a control input command having six degrees of freedom. This device has an opaque body 20
10, a small infrared camera 2020, an infrared light emitting diode group 2030, and three infrared transmission filters 204
5, 2050, 2060. The force detection sensor 2060 has infrared transmission filters 2040, 20.
50, 2060 and the main body 2010 are selectively mounted, and detect the force applied to the infrared transmission filter when the user touches. The visible light is filtered by the filter 2040,
The infrared light reflected from the surfaces of 2050 and 2060 and emitted from the infrared light emitting diode group 2030 is filtered by the filter 2
040, 2050, 2060 surface is transmitted. The infrared light emitting diode group is arranged so as to uniformly irradiate infrared light from around the filter. When the user touches the infrared transmission filter, infrared light does not pass through the touched portion, so the infrared camera 2020 detects this touched portion as a different portion. The input operation function of this control device is basically the same as that of a control device using visible light as a light source, but it processes a binary image due to a disturbance generated due to a change in the visible light irradiation distribution of the environment used by the user. It is possible to eliminate potential errors in doing so.

The transmittance of the infrared transmitting filter is extremely low for light having a short wavelength such as visible light region and extremely high for light having a long wavelength such as infrared light, and is already commercially available. As such an infrared transmission filter, for example, a glass material called “black glass” manufactured by Rolyn Opics and having a product number of 65.1398 (RG850) is suitable. This filter has a light transmittance of less than 10 ^-3 % in the visible light region (wavelength 390 nm to 770 nm), a transmittance of 50% at a wavelength of 850 nm, and a transmittance of 900 nm at a wavelength of 900 nm.
It will rise to 97.5%. Infrared light emitting diode has a wavelength of 800 nm
Since it emits infrared light of ˜1000 nm (peak wavelength 900 nm), it can be said that this filter has desirable characteristics as a preferred embodiment. The semi-transmission characteristic of the infrared transmission filter is that the filter surface (only one side) is finely polished,
Alternatively, it can be realized by attaching a gray translucent light diffusion sheet that diffuses infrared rays radiated to the filter surface. As an example of the light diffusion sheet, Edmu
A suitable sheet is LENSCREEN from nd Scientific (Barrington, NJ).

Although the preferred embodiments of the present invention have been described above, those skilled in the art can make various modifications and changes without departing from the spirit and scope of the present invention. For example, the input technology shown in FIGS. 11 and 12 is a control input device (for example, an input device such as a mouse) other than an input device in which a video camera is arranged behind a screen.
It can also be applied to. Alternatively, the force detection sensor shown in FIG. 10 can be applied to the control input device shown in FIG. In this application, the panel 105 of FIG. 1 itself does not need a force sensing property, but the force applied to the panel 105 can be detected by another sensor. Also, the control input device of the present invention is realized by using a CMOS area sensor, an infrared ray collection area sensor, or another equivalent detector as an alternative device of the small CCD camera shown in the preferred embodiment of the present invention. It is also possible to do so. That is, all such modifications and changes are considered to belong to the technical scope of the present invention.

[Brief description of drawings]

FIG. 1 is a perspective view showing a configuration of an interactive computer graphics system and a mechanical device for graphics according to a preferred embodiment of the present invention.

FIG. 2 is a perspective view showing the configuration of an interactive computer graphics system and its graphic input device according to another preferred embodiment.

FIG. 3 is a diagram showing an image space, that is, an image obtained by the CCD camera shown in FIG.

4 is a diagram showing a control operation gesture in the input device shown in FIG. 2 and a movement of a graphic object on the graphic display of FIG.

5 is a flow chart of image processing steps obtained with the CCD camera of FIG.

FIG. 6 is a functional diagram of FIG.

7 is a diagram showing an image format in the processing of FIG.

FIG. 8 is a diagram showing how user touch point coordinates in an image of a camera are transformed into a graphic display space in a graphic application example.

FIG. 9 is a more detailed diagram relating to image processing.

FIG. 10 is a view showing a preferred embodiment in which a force sensor for detecting a force applied to the panel is mounted.

11 is a diagram showing an operation of the control device shown in FIG. 10 as an interactive computer graphics interface.

FIG. 12 is an explanatory diagram for explaining a new input control method applicable to the control device 100 and the like shown in FIG. 1.

FIG. 13 is a diagram showing an operation of the interface method of FIG.

14A and 14B show a cylindrical control device according to a preferred embodiment of the present invention, in which FIG. 14A is a perspective view, FIG. 14B is a sectional view, and FIG.
Is a top view.

15 is a diagram showing a typical image obtained by a CCD camera in the device arrangement of FIG.

16 is a diagram showing a correspondence relationship between a plurality of locations in the image of FIG. 15 and a device when the device of FIG. 14 is operated.

FIG. 17 (a) shows a hemispherical control device according to a preferred embodiment of the present invention, and FIG. 17 (b) is an explanatory view for explaining generation of a rotation input command using the control device shown in FIG. 17 (a). Is.

18 (c)-(h) are explanatory views for explaining generation of a rotation input command using the control device shown in FIG. 17 (a).

19 (i)-(l) are explanatory views for explaining generation of a rotation input command using the control device shown in FIG. 17 (a).

20 is a diagram showing a data image and a histogram cited to describe steps 505 to 515 shown in FIG. 5. FIG.

21 is a diagram showing a data image and a histogram that are cited to describe steps 515 to 520 shown in FIG. 5. FIG.

FIG. 22 is a perspective view showing a control device with six degrees of freedom based on the use of infrared rays according to a preferred embodiment of the present invention.

[Explanation of symbols]

 100 control device 105 panel 110 CCD camera 120 light source 125 mirror 130 computer 135 graphics display

Claims (41)

[Claims] 1. A graphic input device for an interactive computer system, comprising: an opaque housing including a translucent light diffusing panel; and force sensing for providing a signal in response to a force applied to the panel. A graphic input device comprising: means, and an image detection device which is mounted in the housing and supplies an image of the panel and an image signal according to a change of the image. 2. The graphic input device according to claim 1, further comprising a light source for illuminating an outer surface of the panel, the light source being attached to the outside of the housing. 3. The reflector according to claim 2, which is for reflecting the light from the light source toward the outer surface of the panel,
The graphic input device further comprising a reflector attached to an outer surface of the housing. 4. A graphic input method for supplying a graphic input signal to an interactive computer system using an opaque housing having a semi-transparent light-diffusing panel for sensing force, the step of contacting an outer surface of the panel. A step of acquiring an image inside the panel using an image detection device mounted in the housing; a step of generating an image signal in response to a change in the image; and a step depending on the magnitude of the force applied to the panel. And a step of generating a generated signal. 5. A graphic input device for an interactive computer system, the first translucent light diffusing panel and a second translucent light diffusing panel mounted at an angle different from the first light diffusing panel. An image detection device for generating an image signal according to a change in the images of the first and second panels, the image detection device being mounted in the housing. And a graphic input device. 6. The method according to claim 5, wherein the force applied to the first panel is detected, a first signal corresponding to the detected force is generated, and the force applied to the second panel is detected and detected. The graphic input device further comprising force detection means for generating a second signal according to the applied force. 7. The housing according to claim 5, further comprising a third translucent light diffusing panel, wherein each of the first to third panels is attached to a surface of the housing, and The graphic input device is arranged in a vertical position, and the image detection device is installed in the housing so as to capture an image of an inner surface of the first to third panels. 8. The method according to claim 7, wherein the force applied to the first panel is detected, a first signal is generated according to the detected force, and the force applied to the second panel is detected. It further comprises force detection means for generating a second signal according to the detected force, detecting the force applied to the third panel, and generating a third signal according to the detected force. And a graphic input device. 9. A graphic input method for supplying a graphic input signal to an interactive computer system using an opaque housing having a plurality of semi-transparent light diffusion panels for sensing force, wherein an outer surface of the panel is contacted. A step of acquiring an image inside the panel using an image detection device mounted in the housing; a step of generating an image signal according to a change of the acquired image; and a panel contact portion in the image. Determining a position coordinate of the graphic inputting method, and supplying a signal corresponding to the position coordinate to the interactive computer system. 10. The method according to claim 9, wherein a background image of the panel before the user touches the panel is acquired,
The graphic input method further comprising the step of normalizing a video screen signal according to the acquired background image. 11. The graphic input according to claim 9, further comprising a step of converting an image signal into a binary signal before determining a position coordinate of a portion corresponding to a contact portion in the panel image. Method. 12. The graphic input method according to claim 9, further comprising the step of tracking the position of the object in the image. 13. A graphic input method for supplying a graphic input signal to an interactive computer system using an opaque housing having a plurality of force-sensitive translucent light diffusing panels attached thereto, wherein an outer surface of the panel is contacted. A step of acquiring an image of the inside of the panel by an image detection device mounted in the housing; a step of generating an image signal according to a change of the acquired image; A graphic input characterized by including the steps of determining position coordinates, converting the determined position coordinates into three-dimensional space coordinates, and generating a signal corresponding to the position coordinates obtained by the conversion. Method. 14. The method according to claim 13, further comprising the step of acquiring a background image of the panel before the user touches the panel, and normalizing the video screen signal according to the acquired background image. A graphic input method characterized in that. 15. The graphic according to claim 13, further comprising a step of converting an image signal into a binary signal before determining a position coordinate of a portion corresponding to a contact portion in the panel image. input method. 16. The method according to claim 13, wherein the step of generating the image signal, the step of determining the position coordinates, and the step of converting the coordinates are repeated, and the step of tracking the position of the object in the image. A graphic input method comprising: 17. A method of operating a graphic object on an interactive computer graphic display according to an operation of a panel touch control input device, the method comprising: displaying the graphic object on the graphic display; A first area on the graphic display device in response to a touch by a user in the first area on the panel touch control input device, and the first signal is generated by the graphic input device. Manipulating an object contained within the area in response to the first signal; and at the same time generating a second signal in response to user contact on the second area on the panel touch control input device. Graphic display in response to the second signal And a step of moving the designated area above. 18. The graphic object operating method according to claim 17, wherein the second area captures the periphery of the first area. 19. A method for providing a graphic input signal from an input device to an interactive computer system using an opaque housing having a translucent single light diffusing panel, the method comprising: contacting an outer surface of the panel. A step of acquiring an image inside the panel by an image detection device mounted in the housing; a step of generating an image signal according to a change in the acquired image; and a position coordinate of a panel contact portion in the image. Graphic input comprising: determining, identifying a touch position on the panel by the determined position coordinates, and providing a signal corresponding to the position coordinates to the interactive computer system. Signal supply method. 20. The method according to claim 19, further comprising the step of acquiring a background image of the panel before the user touches the panel, and normalizing the video screen signal according to the background image. Characteristic graphic input signal supply method. 21. The graphic input according to claim 19, further comprising a step of converting an image signal into a binary signal before determining a position coordinate of a portion corresponding to a contact portion in the panel image. Signal supply method. 22. The method according to claim 19, further comprising the step of tracking the position of the object in the image. 23. A method of providing a graphic input signal from an input device to an interactive computer system using an opaque housing having a translucent single light diffusing panel, the method comprising: contacting an outer surface of the panel. Acquiring an image inside the panel with an image device mounted in the housing; generating an image signal according to a change in the acquired image; and determining a position coordinate of a panel contact portion in the image. A method of supplying a graphic input signal, comprising: a step of converting the determined position coordinates into three-dimensional space coordinates; and a step of generating a signal according to the converted position coordinates. 24. The method of claim 23, further comprising acquiring a background image of the panel before a user touches the panel and normalizing the video screen signal in response to the acquired background image. A method for supplying a graphic input signal, characterized by being provided. 25. The graphic according to claim 23, further comprising a step of converting the image signal into a binary signal before determining a position coordinate of a portion corresponding to a contact portion in the panel image. Input signal supply method. 26. The method according to claim 23, wherein the step of generating an image signal, the step of determining the position coordinates, and the step of converting the coordinates are repeated, and the steps of tracking the position of the object in the image are repeated. A method for supplying a graphic input signal, which is characterized in that: 27. A graphic input device for an interactive computer system, comprising: an opaque housing including a translucent light diffusing panel; an image detection device mounted in the housing; An image detection device that acquires an image of a portion in contact with the panel and supplies an image signal according to a change in the panel and the panel contact portion, and means for generating a signal indicating a position of the contact portion A graphic input device characterized by. 28. The graphic input device according to claim 27, wherein the panel has a cylindrical shape, and a translucent light diffusion disk is attached to an upper portion of the panel. 29. The graphic input device of claim 28, further comprising a translucent light diffusing disc mounted on the cylindrical panel. 30. The graphic input device according to claim 29, further comprising a force sensor for detecting a force applied to the disc and the cylindrical panel. 31. The graphic input device according to claim 27, wherein the panel has a hemispherical shape. 32. The graphic input device according to claim 31, further comprising a force sensor for detecting a force applied to the panel. 33. The light source according to claim 2, wherein the light source includes an infrared light source, the panel is a translucent long-wavelength light transmission filter, and the image detection device is an image detection device that senses infrared light. Graphic input device. 34. The infrared light source for irradiating an outer surface of a panel according to claim 5, wherein the panel is a semi-transparent long-wavelength light transmission filter, and the imaging device senses infrared rays. Graphic input device. 35. The infrared light source for irradiating an outer surface of a panel according to claim 6, further comprising an infrared light source attached to the outside of the housing, wherein the panel includes a semitransparent long-wavelength light transmission filter. The graphic input device is characterized in that the image detecting device is an image detecting device that detects infrared rays. 36. The light source according to claim 27, wherein the light source includes an infrared light source, the panel is a semi-transparent long-wavelength light transmission filter, and the image detection device is an image detection device that senses infrared light. Graphic input device. 37. The method according to claim 4, wherein the panel is a long-wavelength light transmission filter, the image detection device is an image detection device that senses infrared light, and the panel further comprises the step of illuminating the panel with the infrared light source. A graphic input method characterized by. 38. The panel according to claim 9, wherein the panel is a long-wavelength light transmission filter, the image detection device is an image detection device that senses infrared rays, and the method further comprises the step of illuminating the panel with the infrared light source. A graphic input method characterized by. 39. The method according to claim 13, wherein the panel is a long-wavelength light transmission filter, the image detection device is an image detection device that senses infrared light, and the method further comprises the step of illuminating the panel with the infrared light source. A graphic input method characterized by. 40. The method according to claim 19, wherein the panel is a long-wavelength light transmission filter, the image detection device is an image detection device that senses infrared light, and the method further comprises the step of illuminating the panel with the infrared light source. A method for supplying a graphic input signal, characterized by: 41. The method according to claim 23, wherein the panel is a long-wavelength light transmitting filter, the image detecting device is an image detecting device that senses infrared rays, and the panel further includes the step of irradiating the panel with the infrared light source. A method for supplying a graphic input signal, characterized by: