JPH11212691A - Method and device for pen input - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow various kinds of operation corresponding to pen operation on paper document to be reflected on an electronic document corresponding to the paper document in real time. SOLUTION: At an arithmetic processing part 40, an arithmetic part 42 detects the position of a stylus pen 1 (1a and 1b) and a display control part 45 displays the plotting track or the like of the stylus pen 1 on a display part 50 corresponding to that detected position. Based on electronic document preserved in this system, a paper document 10 is printed out. At the arithmetic processing part 40, a paper input control part 44 detects a specified operation of the stylus pen 1 on that paper document 10 and manages this paper document 10 in one-to-one correspondence to the electronic document, and when an operation of the stylus pen 1 is performed on this paper document 10 based on the managed result, the operation of the stylus pen 1 on this paper document 10 is reflected on the corresponding electronic document.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pen input method and apparatus having a position detecting mechanism for an input pen and performing input processing in accordance with a designated position of the input pen and a pen operation. One-to-one between paper documents
The present invention relates to a pen input method and apparatus having an input user interface capable of reflecting a pen operation on a paper document on a corresponding electronic document by associating the pen operation with the electronic document.

[0002]

2. Description of the Related Art Conventionally, input devices in a computer system or the like having a handwriting input function include: 1) an infrared type; 2) a pressure-sensitive type; 3) a capacitive type; 4) an electromagnetic type; An ultrasonic type is known.

[0003] In the infrared method of 1), a spot position detected by a light pen that emits infrared light is detected using a semiconductor position detecting element, and a plurality of infrared light emitting elements and a light receiving element are arranged on an input surface. There is known a device that detects an indicated position by determining a position where infrared light emitted from an infrared light emitting element is blocked by a finger, an input pen, or the like from an output of the light receiving element.

[0004] In the pressure sensing method of 2), an electrode sheet provided with a plurality of X-axis electrodes and a plurality of Y-electrodes are arranged.
A pressure-sensitive panel in which a support on which an axis electrode is disposed is overlapped with a dot spacer interposed therebetween,
Sheets are superimposed via a three-dimensional spring,
2. Description of the Related Art There is known a device that detects a designated position by detecting a change in current at a position touched by a finger, an input pen, or the like.

[0005] In the case of the method of 3), the conductive films formed by vapor deposition on two sheets of glass are used as X-axis and Y-axis electrode patterns, respectively. A device using a capacitive coupling type panel in which surfaces are bonded to each other is known.

Further, according to the electromagnetic method of 4), a plurality of sensor antenna coils are arranged on an input surface, and a resonance circuit including a coil and a capacitor is built in an input pen. The current is passed to the resonance circuit of the input pen using electromagnetic induction by flowing an alternating current to the coil, and then the alternating current passed to the plurality of sensor antenna coils is turned off. In this state, the resonance circuit of the input pen is turned off. There is known a device that detects a pointing position of an input pen by detecting a magnetic field generated by a current flowing through the sensor antenna with the plurality of sensor antenna coils.

According to the ultrasonic method of 5), the ultrasonic transmitting means is mounted on the stylus pen, and the ultrasonic receiving means is arranged at a predetermined position in the input area of the stylus pen, and the stylus pen is mounted on the stylus pen. While measuring the distance between the ultrasonic transmitting means and the ultrasonic receiving means from the time when the ultrasonic wave is transmitted from the mounted ultrasonic transmitting means and the time when the ultrasonic wave is received by the ultrasonic receiving means, 2D or 3D stylus pen using measured distance
2. Description of the Related Art There is known an ultrasonic position detecting mechanism that calculates a dimensional position coordinate and inputs a position indicated by a stylus pen in a predetermined input area based on the calculated position coordinate.

[0008] In general, when a person discusses complicated contents, a document is often used. Generally, documents are roughly divided into electronic documents and paper documents.
Correction, storage search, and the like are performed, and operations such as browsing and storage are performed using paper documents. Improving the quality of discussions by displaying electronic documents on a large screen using a projector or the like in the above-mentioned discussions, and performing on-the-fly editing and correction using various pen input devices as described above. Can be. In addition, the electronic document that has been added or modified can be reused as minutes.

On the other hand, there are actually many discussions based on paper documents. In most cases, such as general meetings, OHP (Over Head Projecto)
r) In an environment where discussions using paper documents are performed while using the device. At this time, it is naturally possible to perform correction work on the content printed on the paper document, but at present, reflecting the content of the correction work performed on this paper document in an electronic document in real time is, This was not possible with any of the various pen input systems described above.

More specifically, for example, in a pen input system using a tablet, a predetermined area on the tablet is associated with a display area of the computer system, and when a position is designated in the area by a pen, the area corresponds to the position. The cursor moves to the displayed display area. Also,
The buttons provided on the pen correspond to the buttons of a mouse, and the buttons are used to perform an input operation to the computer system. However, in this system, the input area on the tablet corresponds to the display area of the computer system, and it is not possible to perform pen input from any position (on a paper document) other than the input area on the tablet.

Another conventional system disclosed in Japanese Patent Application Laid-Open No. 5-54178 divides a form into a form for character recognition and a form for correction.
3 discloses a technique in which character recognition is performed and the recognition result is stored in a result storage unit 104, while the correction form is read by a correction form reading unit 106 and corrected by a correction unit 107. According to the publication, when a document (form) is modified, the form can be modified without a modification terminal having a display, a keyboard, or a mouse. However, a fixed form is used to modify the document. Or OCR (Opti
The degree of freedom when modifying a document was very small, such as processing through a character recognition device such as cal Character Reader).

Further, Japanese Patent Application Laid-Open No. 9-44383 discloses that a computer scans a paper document created by printing out a hypertext document having link information, and after importing, reads the link information. By locating the above link information on a hard copy for the purpose of eliminating the inconvenience that could not be reused,
A technique for making link information reusable in an electronic document after scanning the hard copy is disclosed.
The publication is the same as that of the previous publication in that a paper document is read by OCR or the like and the result is reflected in an electronic document, and operations such as corrections performed on the content printed on the paper document are the same. It was not intended to reflect the contents in an electronic document in real time.

[0013]

Among the above conventional systems, in a pen input system using a tablet, an input area on the tablet corresponds to a display area of the computer system, and any input area other than the input area on the tablet. Pen input could not be performed from the position. Most of the conventional systems represented by Japanese Patent Application Laid-Open Nos. 5-54178 and 9-44383, when performing a correction operation on an electronic document using a paper document, etc. Form) by OCR, etc.
The electronic document is modified based on the result of the reading, and there is a problem that the input operation to the computer cannot be directly performed on the printed content using the paper document as an input medium. there were.

Accordingly, the present invention solves the above-mentioned problems, and uses a paper document and interactively interacts with a computer to convert an electronic document corresponding to the paper document into various types corresponding to a pen operation on the paper document. It is an object of the present invention to provide a pen input method and device capable of reflecting an operation in real time.

Another object of the present invention is to provide a pen input method that allows one or more operators to operate a corresponding electronic document from a plurality of paper documents without being restricted by an input area. And a device.

Another object of the present invention is to provide a method for operating a corresponding electronic document using a paper document such that the content of the operation performed on the paper document remains as writing information on the paper document. A pen input method and apparatus are provided.

Still another object of the present invention is to provide a pen input method and a pen input method capable of performing various operations on an external electronic object which does not originally exist in the system main body through a pen operation on a corresponding paper document. Is to provide.

[0018]

According to a first aspect of the present invention, a pointing position of an input pen having an operation content indicating function is detected, and an operation of the input pen is performed based on the pointing position. A pen input device for performing input processing based on the electronic document, a storage unit for storing at least an electronic document, a printing unit for printing and outputting the electronic document as a paper document, and an operation of recognizing the operation content of the input pen on the paper document Recognizing means; inter-document correspondence managing means for managing an electronic document corresponding to the paper document in a one-to-one correspondence with the paper document in accordance with a specific operation of the recognized operation contents; Before the operation of the input pen on the paper document according to the operation content recognized by the recognition means; Characterized by comprising an operation reflecting unit for reflecting the electronic document.

According to a second aspect of the present invention, in the first aspect of the present invention, the specific operation is an operation of instructing a predetermined information pattern in a predetermined marking area on the paper document by the input pen. It is characterized by.

According to a third aspect of the present invention, in the first aspect of the invention, the specific operation is a drawing operation of predetermined graphic information in a predetermined marking area on the paper document by the input pen. It is characterized by.

According to a fourth aspect of the present invention, in the first to third aspects, the specific operation is a drawing operation of information corresponding to a predetermined password in a predetermined area on the paper document by the input pen. It is characterized by including.

According to a fifth aspect of the present invention, in the first aspect of the present invention, a position indicating operation of the input pen at an arbitrary position on the paper document is recognized, and the input pen is formed from the pointing position of the input pen at the time of the recognition. Positional relationship setting means for setting a positional relationship between an area on the paper document and an area formed by the electronic document, and an area corresponding to the setting area of the electronic document based on the setting area by the positional relationship setting means Control means for displaying the contents of the input pen on the display means, and input area regulating means for controlling the processing for reflecting the operation by the input pen on the electronic document only in the setting area. And

According to a sixth aspect of the present invention, in the fifth aspect of the present invention, the display control means displays the operation content of the input pen and an operation reflection result on the electronic document by the operation reflection means based on the operation content. The information is displayed on the display means.

According to a seventh aspect of the present invention, in the fifth aspect of the present invention, the operation content indicating function of the input pen is selected from among selection, duplication, movement, deletion, editing, attribute change, and execution of an object on the paper document. And the operation reflecting means corresponds to a corresponding object on the electronic document based on a recognition result of each instruction operation by the operation instruction function by the operation recognizing means. It is characterized by performing an instruction operation.

According to an eighth aspect of the present invention, in the seventh aspect of the present invention, the object on the paper document is link information, and the operation reflecting means transmits the link information by the input pen on the paper document. Select, duplicate,
The present invention is characterized in that a corresponding instruction operation on the link information is performed for each instruction operation of moving, deleting, and executing.

According to a ninth aspect of the present invention, in the seventh aspect of the present invention, the object on the paper document is a program, and the operation reflecting means executes the program based on an operation instructing execution of the object by the input pen. It is characterized by executing.

In a tenth aspect based on the seventh aspect, the paper document includes an object serving as transmission destination information and an object functioning as a transmission instruction command, and the operation reflecting means includes the former using the input pen. The transmission destination information is displayed on the display means based on the object selection instruction operation, and the corresponding electronic document is transmitted according to the transmission destination information based on the latter object execution instruction operation.

According to an eleventh aspect, in the tenth aspect, the object on the paper document includes a fixed text and an illustration, and the operation reflecting means selects, duplicates, and moves the object using the input pen. The above-mentioned fixed-form sentences and illustrations are respectively selected based on each of the instruction operations described above, and are copied or moved on the corresponding electronic document.

According to a twelfth aspect of the present invention, in the seventh aspect, the object on the paper document is file management information, and the operation reflecting means selects, duplicates, moves, and selects the object with the input pen. The present invention is characterized in that a corresponding instruction operation on the file management information at the indicated position is performed based on each instruction operation of deletion, editing, attribute change, and execution.

According to a thirteenth aspect, in the eighth aspect, the link information is another paper document,
The operation reflecting means may cause an electronic document corresponding to the another paper document to be developed under the control of the inter-document correspondence management means based on a selection instruction operation of the link information by the input pen.

According to a fourteenth aspect of the present invention, in the fifth aspect of the present invention, a movable real object corresponding to a print object of the paper document is arranged on the paper document, and the print object is input to the print object by the input pen. Each operation of duplication, movement, and deletion is instructed in association with the position of the real object, and the operation reflecting means responds to the instruction operation of duplication, movement, and deletion by the input pen on the paper document. And copying, moving, and deleting the object corresponding to the print object on the electronic document to be executed.

According to a fifteenth aspect of the present invention, in the invention of the seventh aspect, the input pen has an operation function of designating a copy or move destination area when the copy or move instruction operation is performed. Has a function of enlarging or reducing the information related to the duplication or movement so as to fit in the area.

According to a sixteenth aspect of the present invention, in the seventh aspect, the input pen has a writing implement function, and an operation trajectory on the paper document can be written on the paper document. .

According to a seventeenth aspect of the present invention, there is provided a pen input device for detecting an indicated position of an input pen having an operation content indicating function and performing input processing based on the operation of the input pen based on the indicated position. Storage means for storing, operation recognizing means for recognizing the operation content of the input pen on the paper document, recognizing drawing information based on a specific drawing operation among the recognized operation contents, and corresponding to the drawing information An external information fetching means for fetching an external electronic document from the outside and storing it in the storage means; and a document for managing the external electronic document in a one-to-one correspondence with the paper document on which the specific drawing operation has been performed. Correspondence management means, and the input page on the paper document according to the operation content recognized by the operation recognition means. Characterized by the operation comprising an operation reflecting unit for reflecting the corresponding said external electronic document.

According to an eighteenth aspect, in the seventeenth aspect, the paper document is a magazine, the drawing information is a magazine name and the number of pages of the magazine, and the external information capturing means is a network or A means for acquiring an external document corresponding to the contents of the page of the magazine through a portable storage medium.

The invention according to claim 19 is a pen input method for detecting an indicated position of an input pen having an operation content indicating function and performing an input process based on the operation of the input pen based on the indicated position. The operation content of the input pen on the obtained paper document is recognized, and the electronic document corresponding to the paper document is one-to-one with respect to the paper document in accordance with a specific operation among the recognized operation contents. And the operation of the input pen on the paper document is reflected on the corresponding electronic document in accordance with the recognized operation content.

According to a twentieth aspect, in the nineteenth aspect, a position indicating operation of the input pen at an arbitrary position on the paper document is recognized, and the input pointing pen is formed from the pointing position of the input pen at the time of the recognition. Setting the positional relationship between the area on the paper document and the area formed by the electronic document, and displaying the contents of the area corresponding to the setting area of the electronic document on the display means based on the setting area; The control is performed so that the process of reflecting the operation by the input pen on the electronic document only within the setting area is enabled.

According to a twenty-first aspect, in the twentieth aspect, the input pen operates at least one of selection, duplication, movement, deletion, editing, attribute change, and execution of an object on the paper document. The process of reflecting the operation of the input pen on the electronic document is a process of performing an instruction operation corresponding to a corresponding object on the electronic document based on a recognition result of each of the instruction operations. It is characterized by the following.

[0039]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A pen input device according to the present invention comprises:
The transmission timing of the ultrasonic waves transmitted from the ultrasonic transmission means provided on the input pen is synchronized with the reception timing of the ultrasonic reception means for receiving the ultrasonic waves transmitted from the ultrasonic transmission means. Above, the distance between the ultrasonic wave transmitting unit and the ultrasonic wave receiving unit is calculated from the time difference between the timing at which the ultrasonic wave is transmitted from the ultrasonic wave transmitting unit and the timing at which the ultrasonic wave is received by the ultrasonic wave receiving unit, and further, The position coordinates of the input pen are calculated using the distance between the ultrasonic transmission unit and the reception unit, and the position indicated by the input pen in a predetermined input area such as a display of a personal computer is input using a cursor or the like. It presupposes an ultrasonic position detection mechanism that realizes input processing such as. In the present invention, for example, the ultrasonic wave receiving means for receiving the ultrasonic wave transmitted from the input pen is 3
A plurality of ultrasonic receiving elements arranged in a dimension are used. In this case, the position of the input pen based on the reception outputs of the plurality of ultrasonic receiving elements can be detected three-dimensionally.

Further, in the pen input device of the present invention, not only the input operation from the above-mentioned predetermined input area is performed, but also
Using the position detection mechanism, a specific operation of an input pen on a paper document is detected, and this paper document is associated with an electronic document managed by a computer in the system on a one-to-one basis. In response to a pen operation performed, a similar operation can be performed on a corresponding electronic document.

The main differences between the pen input device of the present invention and the conventional system are as follows. For example, in a conventional input system using a tablet, a predetermined area on the tablet is assigned to a display area of the computer system, but in the present invention, a predetermined input area on a paper document is It does not correspond to the display area of the computer system. That is, the present invention does not provide a general-purpose input system on a computer like a tablet-type input system.

In the present invention, there are a plurality of paper documents having an input area, and each of the input areas can exist independently. That is,
The present invention is an input system in which a single computer system has a plurality of different input areas, and the same pen operation in each input area causes a computer to execute completely different processing.

In order to realize the above processing, in the present invention,
A one-to-one correspondence between an electronic document created and stored on a computer and a paper document actually output from the printer by the printer is realized by a pen input operation. Based on this technology, by operating the content printed on a paper document with an input pen,
The contents of the electronic document (or information resource on the computer) corresponding to the paper document can be operated in the same manner as the pen operation on the paper document.

In the present invention, the electronic document created and stored on the computer is basically associated one-to-one with the paper document actually output from the printer by the printer. Conventionally, there has been a method according to the present invention in which a form corresponding to a paper document is read by a scanner, the content written on the form is read and recognized, and then used. In addition, for example, when an n-page document (electronic document) is created by a word processing software on a computer and output as a paper document by a printer, when the m-th document is associated between paper and electronic, a word processing software Such a function (for example, a function of jumping to a designated page) can be easily realized.

However, the present invention is a pen input device using a paper document. The electronic document and the paper document are associated with each other by the pen operation of the input pen on the paper document. When the pen operation is reflected on the corresponding electronic document, it provides an environment in which the operation can be performed seamlessly as if we are handling paper.

Hereinafter, embodiments of the pen input device according to the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a schematic configuration diagram for explaining the principle of position detection of the pen input device according to the present invention. In this pen input device, three ultrasonic receiving elements 31a and 31b that receive ultrasonic waves transmitted from an ultrasonic transmitting element built in the stylus pen 1,
31c is arranged at a predetermined position on a screen panel (corresponding to a display unit 50 described later), and the ultrasonic receiving elements 31a, 31b, 31 of the ultrasonic waves transmitted from the ultrasonic transmitting elements are arranged.
The three-dimensional position coordinates of the stylus pen 1 are calculated according to the arrival time at the position c, and an input process is performed based on the position coordinates.

FIG. 2 is a diagram showing the overall configuration of an information processing system including the pen input device. This system 10
At 0, the operator performs an operation in the input area (display area of the display device used as the display unit 50) using the stylus pen (ultrasonic transmitter) 1 synchronized by the pen holder. At this time, the distance between the stylus pen 1 and the ultrasonic sensors (ultrasonic receiving elements) 31a, 31b, 31c is calculated from the transmission / reception timing of the ultrasonic wave transmitted from the stylus pen 1 in the ultrasonic receiving unit 30, The data is input to a computer as an arithmetic processing unit 40. The Computer calculates the three-dimensional position of the stylus pen 1 in the display area of the display device from the distance value received from the ultrasonic receiving unit 30, and performs a process on the application based on the data.

FIG. 3 is a block diagram showing a schematic configuration of one embodiment of the information processing system in FIG. As shown in FIG. 1, the information processing system includes a stylus pen (hereinafter, referred to as a stylus pen) provided with an ultrasonic wave transmitting unit that outputs ultrasonic waves.
.., And a plurality (three in the embodiment) of ultrasonic receiving elements 31 a, 3 a for detecting the ultrasonic waves transmitted from the input pen 1.
An ultrasonic receiving unit 30 having a pair of ultrasonic receiving units 1b and 31c, and a position indicated by the input pen 1 is detected from an ultrasonic receiving state of the ultrasonic receiving unit 30, and the detected position is corresponded to the detection result. And an arithmetic processing unit 40 that controls display on the display unit 50. In the present invention, a paper input control unit 44 that controls input processing for the operation of the input pen 1 on the paper document 10 is provided in the arithmetic processing unit 40. The operation of the paper input control unit 44 will be described later, and here, a part related to a position detection mechanism of the input pen 1 as a premise thereof will be described.

FIG. 4 shows a specific configuration of the reception processing section 32 in the ultrasonic receiving section 30 shown in FIG.
In this embodiment, a pen holder 35 for accommodating the input pen 1 when the input pen 1 is not used is provided. The pen holder 35 has an output from the ultrasonic transmission means of the input pen 1 as described later in detail. A synchronization circuit 35 is provided for synchronizing the output timing of the ultrasonic wave to be received with the reception processing in the reception processing unit 32.

The synchronization circuit 35 counts the basic clock output from the basic clock generation circuit 324 provided in the reception processing section 32, and every time the counted value becomes a constant value, the ultrasonic wave of the input pen 1 is transmitted. It is configured to output a synchronization signal for controlling the output timing of the ultrasonic wave output from the means.

The input pen 1 has a basic clock generating circuit (an oscillator 1 to be described later) for generating a basic clock having the same frequency as the basic clock generating circuit 324.
3) is provided, and the basic clock generation circuit includes:
When the input pen 1 is stored in the pen holder 34, a synchronization signal output from the synchronization circuit 35 is input, and output to the synchronization circuits 35 in the pen holder 34 and the counters (321 to 323) from the synchronization circuit 35. After the synchronization with the synchronization signal is established and the input pen 1 is taken out of the pen holder 34, it operates in synchronization with the synchronization circuit 35, that is, the basic clock in the input pen 1 as shown in FIG. The output timing of the ultrasonic wave output from the ultrasonic wave transmitting means of the input pen 1 is controlled every time the count value becomes a constant value.

That is, when the input pen 1 is stored in the pen holder 35, the output timing of the ultrasonic wave output from the ultrasonic wave transmitting means of the input pen 1 and the reception processing in the reception processing unit 32 are synchronized, and the input pen 1 is synchronized. The output timing of the ultrasonic wave output from the ultrasonic transmitting means
From the timing of the synchronization signal applied to the reception processing unit 32.

The synchronization signal applied from the synchronization circuit 35 to the reception processing unit 32 is output from the counters 321 and 3 of the reception processing unit 32.
22 and 323 are applied to the reset terminals,
1, 322 and 323 are cleared in synchronization with this synchronization signal.

The basic clock output from the basic clock generation circuit 324 is input to the count input terminals of the counters 321, 322, 323.
To the control input terminals for controlling the stop of counting of 1, 322, 323, receiving clocks respectively output from the ultrasonic receiving elements 31a, 31b, 31c when receiving the ultrasonic wave output from the tip of the input pen 1 are input. Have been.

That is, the counters 321, 322, 32
3 are simultaneously reset at the timing when the ultrasonic wave is output from the tip of the input pen 1, and the ultrasonic receiving elements 3 are respectively reset.
The basic clock output from the basic clock generation circuit 324 is counted until the reception clock is input from 1a, 31b, 31c, and as a result, the counters 321, 322, 3
The count value of 23 corresponds to the time Ta, Tb, Tc until the ultrasonic wave output from the tip of the input pen 1 is detected by the ultrasonic receiving elements 31a, 31b, 31c.

FIG. 6 shows the counters 321, 322, 3
23 is a timing chart showing the detection operation of the times Ta, Tb, and Tc according to the embodiment 23. That is, the input pen 1
6A is a diagram for generating a basic clock shown in FIG. 6A and outputting an ultrasonic wave from the tip of the input pen 1 in synchronization with the rise of the synchronization signal shown in FIG. 6B. A transmission clock shown in FIG. 6C is generated, and an ultrasonic drive signal shown in FIG. 6D for driving the ultrasonic transmission means of the input pen 1 is generated in synchronization with the transmission clock. The ultrasonic transmission signal of the input pen 1 is driven by the ultrasonic driving signal, and the ultrasonic wave is output from the tip of the input pen 1 only while the transmission clock is generated.

The ultrasonic waves output from the tip of the input pen 1 are received by the ultrasonic receiving elements 31a, 31b, and 31c, and the ultrasonic receiving elements 31a, 31b, and 31c receive the ultrasonic waves corresponding to the reception of the ultrasonic waves. A reception clock as shown in FIG. 6 (e) is output. Therefore, the counters 321, 32
2 and 323, the time T from the rising timing of the synchronization signal shown in FIG. 6B to the rising timing of the reception clock shown in FIG.
Tc can be obtained by counting the basic clocks shown in FIG.

Times Ta, Tb, and Tc until the ultrasonic waves output from the tip of the input pen 1 counted by the counters 321, 322, and 323 are detected by the ultrasonic wave receiving elements 31a, 31b, and 31c are reception times. Holding circuits 325, 32
6 and 327, and transmitted to the arithmetic processing unit 40 shown in FIG. Arithmetic processing unit 4
0 calculates the designated position coordinates of the input pen 1 using the notified times Ta, Tb, and Tc, the details of which will be described later (see FIG. 24).

Next, a specific detailed configuration of the input pen 1, the ultrasonic receiving unit 30, and the arithmetic processing unit 40 shown in FIG. 3 will be described. FIG. 7 shows an internal circuit configuration of the input pen 1. In FIG. 7, this input pen 1
A transmission timing control unit 11 for controlling the transmission timing of the ultrasonic wave from the input pen 1; an operational amplifier 16; an amplifier unit 15 for amplifying the ultrasonic driving clock output from the transmission timing control unit 11; Transmitter 18
And a transmitting unit 17 for transmitting an ultrasonic wave based on an ultrasonic transmitter driving signal output from the amplifier unit 15.

In particular, in this embodiment, a multi-pen function which can simultaneously use a plurality of pens and a multi-mode function which can switch a plurality of modes, which will be described in detail later, are enabled. According to the output of the buttons of the pen 1 (for example, the first button 14a and the second button 14b shown in FIG. 14) and the basic clock from the oscillator (basic clock generating unit) 13 that generates the basic clock, the input pen 1 Has a sequencer 12 for controlling the state transition of the ultrasonic transmission timing control. The details of the state transition control of the ultrasonic transmission timing control by the sequencer 12 will also be described later (see FIG. 16).

FIG. 8 shows a specific configuration of the ultrasonic receiving section 30 and the arithmetic processing section 40 shown in FIG.
8, the ultrasonic receiving unit 30 includes a receiving sensor unit 3
1, a reception processing unit 32 and a communication unit 33. Here, the receiving sensor unit 31 is the ultrasonic receiving element 3 shown in FIG.
1a, 31b and 31c. The reception processing unit 32 includes an amplifier unit 60 and a reception timing control unit 70. The amplifier unit 60 includes an operational amplifier 61. The reception timing control unit 70 includes a counter, a register 71,
A sequencer 72, an oscillator 73, and an SCC (serial communication control unit) initialization unit 74 are provided. Here, the counter and the register 71 correspond to the counter 32 shown in FIG.
1, 322, 323 and reception time holding circuits 325, 3
26 and 327, and the oscillator 73 corresponds to the basic clock generation circuit 324 shown in FIG.

The sequencer 72 includes an oscillator 73
The transition state of the reception timing control of the reception timing control unit 70 is controlled based on the basic clock output from. The sequencer 72 is reset by a synchronization signal (reset signal) from the synchronization circuit 35 of the pen holder 34 shown in FIG.
The transition state of the reception timing control is controlled. The details of the control of the transition state of the reception timing control by the sequencer 72 will be described later (see FIG. 17). The communication unit 33 includes an SCC (serial communication control unit) 331 and a 232C driver 332 for driving an RS232C interface.

In FIG. 8, the arithmetic processing unit 40
A window processing unit (Wi-Fi) on which a predetermined application is mounted and controls display corresponding to the pointing positions of the pens 1 and 2
nfor Pen), mouse driver, etc.
It comprises a personal computer connected to the communication unit 33 of the ultrasonic receiving unit 30 via the 32C interface. The paper input control unit 44 shown in FIG. 3 is also provided in the personal computer.

Here, points to be noted in designing a position detecting mechanism using ultrasonic waves will be described. FIG. 9 shows FIG.
5 is a diagram showing an example of a timing chart of the most basic ultrasonic wave transmission operation from the input pen 1 shown in FIG. In the figure, FHz is the frequency of the ultrasonic wave to be handled. Also, Tse
nd is the time for transmitting the ultrasonic wave, and Twait is the time corresponding to the interval from the transmission of the ultrasonic wave to the next transmission. The interval of Twait can be variably set as shown in FIG. 10 by a dip switch provided on the input pen 1, for example. Thus, the number of position coordinate data that can be input in one second (point / sec in FIG. 10) is determined.
The value indicated by the reaching distance in FIG. 10 indicates the distance over which the ultrasonic wave propagates within the Twait time, and the setting # must be determined in consideration of this value and the influence of the output intensity and reflection of the ultrasonic wave. No.

Since this point is extremely important in designing this type of ultrasonic position detecting mechanism, FIG.
This will be described in more detail with reference to FIGS. In this system, the ultrasonic wave is transmitted from the input pen 1
a, 31b, and 31c are transmitted directly to the ultrasonic receiving elements 31a, 31b, and 31c.
There is a case where there is one that is reflected by one or a plurality of obstacles and transmitted directly to the ultrasonic receiving elements 31a, 31b, and 31c (path P2). FIG. 11 shows a first path P1 in which ultrasonic waves transmitted from the input pen 1 are directly transmitted to the ultrasonic receiving element 31a, and four obstacles 36a to 36d.
And a second path P2 which is reflected by and transmitted to the ultrasonic receiving element 31a. In this case, since the second path P2 is longer than the first path P1, the ultrasonic wave transmitted from the input pen 1 in response to one ultrasonic drive signal 37 as shown in FIG. In the ultrasonic receiving element 31a, the reception signal 37 of the ultrasonic wave passing through the first path P1
a of the ultrasonic reception signal 37b passing through the second path P2
One arises.

In this embodiment, as shown in FIG. 13, ultrasonic waves are intermittently transmitted from the input pen 1 at predetermined time intervals Twwait. Here, as shown in FIG. 13, there are two paths P1 and P2 as shown in FIG. 11 as a transmission path of the ultrasonic wave transmitted by a certain ultrasonic drive signal 37, and this ultrasonic drive signal 37 The ultrasonic reception signal 37a of the first path P1 and the second path P as the ultrasonic reception signals of the corresponding ultrasonic reception element 31a.
Two ultrasonic reception signals 37b are generated, and the ultrasonic wave transmitted by the ultrasonic driving signal 38 generated at the next transmission timing of the ultrasonic driving signal 37 is received by the ultrasonic receiving element 31a. Assuming that the signal is received as the signal 38a, the ultrasonic reception signal 37b of the second path P2
Is received after the ultrasonic drive signal 38.

In this case, the reception signal 37b of the ultrasonic wave on the second path P2 is erroneously recognized as the leading wave of the ultrasonic wave transmitted by the ultrasonic drive signal 38, and the reception signal 38a of the ultrasonic wave is the ultrasonic wave. The leading signal of the ultrasonic wave transmitted by the drive signal 38 is not recognized, and a malfunction occurs. Such malfunctions correspond to the size of the display screen of the display unit 50 of the apparatus, the intensity of the ultrasonic wave generated from the input pen 1, the reception threshold level in the ultrasonic receiving element, and the transmission from the input pen 1. It can be eliminated by appropriately setting the time interval Twait of the ultrasonic wave. That is, by appropriately setting the intensity of the ultrasonic wave generated from the input pen 1 and the reception threshold level in the ultrasonic receiving element, the ultrasonic receiving element arrives via a plurality of obstacles as shown in FIG. The ultrasonic wave transmitted from the input pen 1 can be prevented from being received, and the time interval Twwait of the ultrasonic wave transmitted from the input pen 1 can be lengthened to reduce the influence of the reflected wave due to the obstacle.

Next, a description will be given of a multi-pen function that can use a plurality of input pens 1 and a method of recognizing an on / off operation of a button mounted on the input pen 1 in this information processing system. Here, to simplify the description, it is assumed that the input pen 1 as shown in FIG. 14 is used. The input pen 1 shown in FIG.
4a and a second button 14b. Here, the first button 14a is a drawing button, and drawing processing is performed by pressing this button. Also, the second button 14
b is a mode switching button, and by pressing this button, it is possible to switch the processing mode on the application such as drawing, erasing, and gesture. It is possible to prepare a plurality of such input pens 1 (see FIG. 3) and simultaneously perform processing at a position on a screen (display unit 50) or the like.

FIG. 15 is a timing chart of the ultrasonic transmission timing control of the pen of PenNO = 0 and the pen of PenNO = 1 when the multi-pen function is selected in the present system. here,
The pen with PenNO = 0 and the pen with PenNO = 1 each transmit ultrasonic waves alternately with four ultrasonic transmission timings as one unit.

That is, as shown in FIG.
The pen with O = 0, that is, pen Pen # 0, transmits ultrasonic waves with four ultrasonic transmission timings as one block. Here, the ultrasonic transmission period at each ultrasonic transmission timing is Tsend, and the ultrasonic transmission interval is Tw.
ait. At the first ultrasonic transmission timing of the four ultrasonic transmission timings, the cursor displayed on the display unit 50 shown in FIG. 3 corresponding to the position of the pen Pen # 0 is controlled to follow. Always transmit ultrasonic waves. Also, at the second ultrasonic transmission timing, an ultrasonic wave is transmitted only when the first button 14a is pressed with the pen Pen # 0. Also, at the third ultrasonic transmission timing, ultrasonic waves are transmitted only when the second button 14b is pressed with the pen Pen # 0. Further, at the fourth ultrasonic transmission timing, similarly to the second ultrasonic transmission timing, the ultrasonic waves are transmitted only when the first button 14a is pressed with the pen Pen # 0.

Also, the pen with PenNO = 1, that is, pen Pen # 1 transmits ultrasonic waves with four ultrasonic transmission timings as one block, similarly to pen Pen # 0.
That is, at the first ultrasonic transmission timing of the four ultrasonic transmission timings, the cursor displayed on the display unit 50 shown in FIG. 3 is controlled to follow the position of the pen Pen # 1. Always transmit ultrasonic waves. Also, at the second ultrasonic transmission timing,
Ultrasonic waves are transmitted only when the first button 14a is pressed by the pen Pen # 1. In addition, at the third ultrasonic transmission timing, the pen Pen # 1 uses the second button 14
The ultrasonic wave is transmitted only when b is pressed. At the fourth ultrasonic transmission timing, the first button 1 is pressed with this pen Pen # 1 similarly to the second ultrasonic transmission timing.
The ultrasonic wave is transmitted only when 4a is pressed.

Here, the pens Pen # 0 and Pen # 1
The reason for configuring to generate the ultrasonic wave indicating that the first button 14a is pressed at both the second ultrasonic transmission timing and the fourth ultrasonic transmission timing is as follows.
In this embodiment, the drawing is performed while pressing the first button 14a when drawing with the input pen 1, so that the detection density (sampling cycle) of the designated position related to the drawing is increased. The ultrasonic drive signal (ultrasonic drive clock) generated at each ultrasonic transmission timing is, for example, an FHz signal, and the ultrasonic drive clock is transmitted through the amplifier 15 shown in FIG.
7, and the transmission unit 17 transmits the FHz ultrasonic waves.

FIG. 16 is a state transition diagram showing the state transition of the ultrasonic wave transmission timing control by the input pen (the sequencer 12 provided in the pen shown in FIG. 7). As described above, this embodiment has a multi-pen function that allows simultaneous use of a plurality of input pens. Whether or not to select this multi-pen function and the number of input pens to be used simultaneously are switched by a dip switch (not shown). Can be
Now, the multi-pen function is selected by a dip switch (not shown) (MPen = 1), and the number of pens to be used at the same time is PenNO = 0 (for example, corresponding to pen 1) and PenNO = 1 (for example, corresponding to pen 2). It is assumed that the two are set. In the state transition diagram shown in FIG. 17, the transition from one transition state to another transition state is P
This is performed every time a predetermined number of basic clocks output from the oscillator 13 shown in FIG. 7 provided in the pen with enNO = 0 or PenNO = 1 are counted.

In FIG. 16, state 102 (Init
0000 (0)) is a so-called initial state of the input pen.
When this input pen is housed in the pen holder 34 shown in FIG. 4 and receives a synchronization signal from the synchronization circuit 35 of the pen holder 34 (ResetPen = 1), any state 101
This state 102 (Init) is also obtained from (AnyState).
0000 (0)). This state 102 (Ini
At t 0000 (0)), nothing is performed. by the way,
In this case, MPen = 1 when the multi-pen function is selected
Then, the pen with PenNO = 0 transits to the state 103 (Trace 0001 (1)), and the pen with PenNO = 1 changes to the state 114 (Wa).
It transitions to itMPen0 1100 (C)).

State 103 (Trace 0001)
(1)) is a state in which the ultrasonic receiving unit 30 shown in FIG. 8 recognizes the position of the pen with PenNO = 0. That is, this is a state in which the cursor displayed on the display unit 50 shown in FIG. 3 is moved in accordance with the position of the pen. Therefore, in this state, this Pe
Ultrasonic waves are output from the pen with nNO = 0. State 1
03 (Trace 0001 (1))
The second button 14b of the pen with enNO = 0 shown in FIG.
Is not pressed (ModeSync = 0) and this P
First button 14a of pen with enNO = 0 shown in FIG.
Is pressed (PenDownSync = 1)
In the case of, the state 105 (Data0 0110 (6))
To the state 104 (Dummy) otherwise.
0 0010 (2)).

State 105 (Data 0 0110)
In (6)), an ultrasonic wave is output from the pen of PenNO = 0 so that the ultrasonic receiving unit 30 recognizes that the first button 14a of the pen of PenNO = 0 is pressed. This state 105 (Data0 0110)
From (6)), the state 107 (Dummy10) is unconditionally set.
101 (5)).

The state 104 (Dummy0 001)
0 (2)), nothing is performed, and this state 104 (Dum
If the second button 14b is pressed at my0 0010 (2)) (ModeSync = 1), the state 106 (Mo
de0011 (3)), and if the second button 14b is not pressed (ModeSync = 0), the state transits to the state 107 (Dummy1 0101 (5)).

State 106 (Mode 0011 (3))
Then, in order to make the ultrasonic receiving unit 30 recognize that the second button 14b of the pen with PenNO = 0 is pressed, an ultrasonic wave is output from the pen with PenNO = 0.
From this state 106 (Mode 0011 (3)), a transition is made unconditionally to state 109 (Dummy 20100 (4)).

The state 107 (Dummy1 010)
1 (5)), nothing is performed, and in this state 107 (Dum
my1 0101 (5)), the PenNO =
When the first button 14a of the pen No. 0 is pressed (P
In the case of enDownSync = 1, the state 108 (D
trans.data1 0111 (7)) and this PenNO
If the first button 14a of the pen with = 0 is not pressed (PenDownSync = 0), the state 109 is set.
(Dummy20100 (4)).

State 108 (Data1 0111)
From (7)), when the multi-pen function is selected (MPen = 1), the state 114 (WaitMPen0) is selected.
1100 (C)) and if the multi-pen function is not selected (MPen = 0), the state 103 (Tra
ce 0001 (1)).

The state 109 (Dummy2 010)
0 (4)), if the multi-pen function is selected (MPen = 1), the state 114 (WaitMPen)
01100 (C)), the multi-pen function is not selected (MPen = 0), and the second button 1
4b is not pressed and state 103 (Trace
0001 (1)) when the second button 14b is not pressed (ModeSync = 0 & PreModeSync)
= 0) is the state 103 (Trace 0001 (1))
Transitions to.

The state 109 (Dummy2 010)
0 (4)), the multi-pen function is not selected (MPen = 0), the second button 14b is currently pressed, and the state 103 (Trace 0001)
(1) When the second button 14b is pressed (Mo)
deSync = 1 & PreModeSync = 1)
State 110 (WaitModeUp0 1000
Transition to (8)).

This state 110 (WaitModeUp0)
1000 (8)), while the second button 14b pressed in the state 103 is kept pressed, the state 111 (W
aitModeUp1 1001 (9)), state 112
The state transits to the state 113 (WaitModeUp3 1011 (B)) via (WaitModeUp21010 (A)). Here, four states 110 to 113 are provided to synchronize with the other pen of PenNO = 1.

State 113 (WaitModeUp31)
011 (B)), when the multi-pen function is selected (MPen = 1), the state 114 (WaitMPen)
01100 (C)). In addition, state 113 (W
aitModeUp3 1011 (B)), when the multi-pen function is not selected (MPen = 0) and the second button 14b is not pressed at the present time (PreMo)
deSync = 0) is the state 103 (Trace 00)
01 (1)). Also, state 113 (Wait
In ModeUp3 1011 (B)), when the multi-pen function is not selected (MPen = 0) and the second button 14b is currently pressed (PreModeSy)
nc = 1) is the state 110 (WaitModeUp0)
1000 (8)).

Now, it is assumed that the multi-pen function is currently selected, that is, (MPen = 1). Therefore, here, the state 113 (WaitModeUp3) is selected.
1011 (B)), state 109 (Dummy201)
00 (4)), state 108 (Data1 0111)
(7)) to state 114 (WaitMPen0 110)
0 (C)).

State 114 (WaitMPen0 110)
0 (C)), the state 115 (WaitMPen) as long as the multi-pen function is selected (MPen = 1).
11101 (D)), state 116 (WaitMPen2)
1110 (E)) and state 117 (WaitMPe).
n3 1111 (F)). These states 114
117 are provided so as to wait for the time corresponding to the transmission of the ultrasonic wave by the other pen of PenNO = 1.

The state 102 (Init 0000)
(0)) to state 114 (WaitMPen0 110)
0 (C)), the pen of PenNO = 1 that has transited to
Unconditionally, state 115 (WaitMPen1 1101
(D)), state 116 (WaitMPen2 1110)
(E)) and the state 117 (WaitMPen111)
11 (E)).

State 117 (WaitMPen1 111)
1 (E)), the state 103 (Trace 0001)
(1)) when the second button 14b is pressed and the second button 14b is currently pressed (ModeSync)
= 1 & PreModeSync = 1) only state 114
(WaitMPen0 1100 (C)), otherwise (ModeSync = 0 & PreMode)
(Sync = 0) is the state 103 (Trace 0001)
Transition is made to (1)).

Then, a pen of PenNO = 0 and Pe
In the case of a pen with nNO = 1, the above-described state transition is performed by these pens.
Is repeated until a synchronization signal is received from the synchronization circuit 35 of the pen holder 34.

FIG. 17 is a state transition diagram showing the state transition of the reception timing control by the ultrasonic receiving unit (the sequencer 72 shown in FIG. 8). The state transition shown in FIG. 17 transits in synchronization with the state transition of the ultrasonic transmission timing control in the pen shown in FIG. 16, and when the multi-pen function is selected in the pen (MPen =
1), state 201 (Trace0 001 (1)), state 202 (Data0010 (2)), state 203 (M
mode0 011 (3)), state 204 (Data1
The ultrasonic wave received at the timing of 100 (4) is Pen
Processing is performed as ultrasonic waves transmitted from the pen with NO = 0, and state 205 (Trace 1 101 (5)) and state 206
(Data2 110 (6)), state 207 (Mode
1 111 (7)), state 208 (Data 3000)
The ultrasonic wave received at the timing of (0)) is PenNO =
Processing is performed as ultrasonic waves transmitted from the first pen.

If the multi-pen function is not selected in the pen (MPen = 0), the state 201 (T
race0 001 (1)), state 202 (Data0
010 (2)), state 203 (Mode 0 011
(3)), state 204 (Data1 100 (4)),
State 205 (Trace1 101 (5)), State 20
6 (Data2 110 (6)), state 207 (Mod
e1 111 (7)), state 208 (Data300
All the ultrasonic waves received at the timing of 0 (0)) are Pen
Processing is performed as ultrasonic waves transmitted from the pen with NO = 0.

In FIG. 17, state 208 (Da
ta3000 (0)) is an initial state of the transition state of the reception timing control in the ultrasonic receiving unit 30, and a hardware reset (Reset) is performed on the ultrasonic receiving unit 30.
If Pen = 1) occurs, any state 209
From (AnyState), this state 208 (Dat
a3000 (0)). This state 208 (D
The designated position based on the ultrasonic wave received at data 3000 (0) is recognized as drawing data on the display unit 50 shown in FIG. State 208 (Data 3000)
(0)), the state 201 (Trace 0
001 (1)). State 201 (Trace
The designated position based on the ultrasonic wave received in (001 (1)) is recognized as position data in the display unit 50 shown in FIG.

State 201 (Trace0 001)
From (1)), unconditionally, the state 202 (Data0)
010 (2)). This state 202 (Data
The designated position based on the ultrasonic wave received in (010 (2)) is recognized as drawing data on the display unit 50 shown in FIG. From the state 202 (Data0 010 (2)), the state 203 (Mode0 011
Transition to (3)). This state 203 (Mode0 0
The ultrasonic wave received in 11 (3) is recognized as mode switching data by the pen.

State 203 (Mode 0 011 (3))
From state 204 (Data1 100
Transition to (4)). This state 204 (Data 11 1)
The designated position based on the ultrasonic wave received in 00 (4)) is recognized as drawing data on the display unit 50 shown in FIG. From the state 204 (Data1 100 (4)),
Unconditionally, state 205 (Trace1 101 (5))
Transitions to. This state 205 (Trace1 101
The pointing position based on the ultrasonic wave received in (5)) is recognized as position data for controlling the follow-up movement of the cursor on the display unit 50 shown in FIG.

State 205 (Trace 1 101)
From (5)), the state 206 (Data2) is unconditionally set.
110 (2)). This state 206 (Data
The designated position based on the ultrasonic wave received in 2110 (6)) is recognized as drawing data on the display unit 50 shown in FIG. From the state 206 (Data2 110 (6)), the state 207 (Mode1 111
Transit to (7)). This state 207 (Mode 1 1
The ultrasonic wave received in 11 (7) is recognized as mode switching data. State 207 (Mode 1 111
From (7)), unconditionally, the state 208 (Data3)
000 (0)). The state transition is repeated until a hardware reset occurs in the ultrasonic receiving unit 30.

Next, a method of mounting the ultrasonic transmitter on the input pen 1 will be described. FIG. 18 shows a configuration example of the input pen 1 when the input pen 1 also has a function as a writing instrument. That is, the pen 1 shown in FIG.
An ultrasonic transmission element 6 for detecting the position of the input pen 1 is mounted, and a writable pen core 3 capable of actually writing an image on a paper document described later is mounted at the tip thereof. ing. When the input pen 1 shown in FIG. 18 is used, for example, in an input operation on a paper document, which will be described later, the content of the operation is actually drawn as a writing history on the paper document, and an actual image is written on the paper document. Can be displayed on the display screen (50) of the system main body.

In the input pen 1 shown in FIG. 18, since the position of the writable pen core 3 of the input pen 1 and the mounting position of the ultrasonic wave transmitting element 6 are separated, the direction of the input pen 1 or In accordance with the inclination, the mounting position of the ultrasonic transmitting element 6 and the position of the pen core 3 are shifted, so that the graphic drawn on the paper document and the graphic displayed on the display screen (50) of the system main body are changed. It may not match exactly. In order to solve this problem, the internal structure of the input pen 1 may be, for example, as shown in FIG. That is, in FIG. 19, the input pen 1 has the ultrasonic transmitting element 6 built therein. The ultrasonic transmission element 6 is driven by an ultrasonic drive signal from a control circuit 5 that operates by power supply from a battery 4 as a power supply, and generates ultrasonic waves. The ultrasonic wave generated from the ultrasonic wave transmitting element 6 is reflected by an ultrasonic reflecting section 8 provided at the tip of the pen 1, and the input pen 1
Is transmitted to the outside through an ultrasonic wave passing window 7 provided at the tip of the device. Also, a writable pen core 3 capable of actually writing an image on a paper document is attached to the tip of the input pen 1.

FIG. 20 is an enlarged view of a portion surrounded by a dotted line with the input pen 1 in FIG. As is apparent from the figure, the ultrasonic reflecting section 8 provided at the tip of the input pen 1
Has a conical shape, is generated from the ultrasonic transmitting element 6,
The ultrasonic wave propagating in the axial direction of the input pen 1 is reflected by the ultrasonic reflection unit 8 so that the tip of the input pen 1 spreads in a range of 360 degrees in a direction perpendicular to the axial direction of the input pen 1. Ultrasonic waves which are converted into ultrasonic waves and spread in the range of 360 degrees are generated from the tip of the input pen 1. In addition,
At the tip of the input pen 1, as shown in FIG. 21, a plurality of ultrasonic passage windows 7 are formed around the input pen 1, and the ultrasonic waves reflected by the ultrasonic reflecting section 8 are reflected by the plural ultrasonic waves. The light is sent out through the passage window 7.

According to the configuration shown in FIGS. 18 to 21, the writable pen core 3 provided at the tip of the input pen 1 is provided.
And the position where the ultrasonic wave transmitted from the input pen 1 is transmitted, that is, the position where the ultrasonic wave passing window 7 is disposed, can be brought close to each other. The graphic drawn on the paper document and the graphic displayed on the display screen (50) of the system main body can be substantially matched. By using the input pen 1 having the structure shown in FIG. 21, the position of the ultrasonic transmission element 6 and the transmission position of the ultrasonic wave transmitted from the input pen 1, that is, the arrangement position of the ultrasonic passage window 7 Between
As shown in FIG. 19, since the input pen 1 is separated by a distance L, it is needless to say that when the input pen 1 is used, the tip position of the input pen 1 needs to be corrected by the distance L. .

Next, a method of calculating the three-dimensional coordinates of the input pen 1 in the present system will be described with reference to FIGS. To correspond to the three-dimensional coordinate calculation, for example,
When the system main body 100 is in the form of a desktop information processing apparatus as shown in FIG. 22, three ultrasonic receiving elements 31a and 31 are provided near the display (input surface).
b and 31c are implemented. At this time, the ultrasonic transmitter is mounted inside the input pen 1 as described above. In this arrangement, as shown in FIG. 23, the ultrasonic waves output from the ultrasonic transmitter (input pen 1) propagate in the air and are received by the three ultrasonic receiving elements 31a, 31b, and 31c. Is done.

In this state, the three ultrasonic receiving elements 31a, 31b, 3
Based on 1c, three-dimensional coordinate axes X, Y, and Z as shown in FIG. 24 are generated. In the position detecting mechanism, the output time of the ultrasonic wave from the ultrasonic transmitter and each ultrasonic receiving element 31
a, the distance between the ultrasonic transmitter and each ultrasonic receiving element is calculated from the ultrasonic receiving time of 31a, 31c, and 31c.
In the three-dimensional coordinate system, three-dimensional position coordinates of the input pen 1 moving on or around the display are detected.

Hereinafter, the three ultrasonic receiving elements 31a,
The principle of three-dimensional detection of the position of the tip of the input pen 1 using 31b and 31c will be described. Now, consider a three-dimensional coordinate system XYZ fixed to three ultrasonic receiving elements 31a, 31b, 31c as shown in FIG. Here, the ultrasonic receiving element 31a is disposed at the origin of the three-dimensional coordinate system XYZ, and the ultrasonic receiving element 31b is located on the Y axis of the three-dimensional coordinate system XYZ. Point (0, Ly, 0)
And the ultrasonic receiving element 31c is provided with a three-dimensional coordinate system X
Suppose that it is arranged at a point (0, 0, Lz) on the Z axis of −YZ.

It is assumed that the position pointed by the input pen 1 is a point (x, y, z) on the three-dimensional coordinate system XYZ, and a point (x, y, z) from the ultrasonic receiving element 31a. z), the distance from the ultrasonic receiving element 31b to the point (x, y,
Assuming that the distance to z) is m and the distance from the ultrasonic receiving element 31c to the point (x, y, z) is l, l ² = (Lz−z) ² + n ² −z ² (1) ^{) n 2 = x 2 + y} 2 + z 2 ... (2) relationship of ^{m 2 = x 2 + (Ly} -y) 2 + z 2 ... (3) is satisfied. From Expressions (9) to (11), x,
When solving for y and z, x = ± SQRT [n ² − ｛(Ly ² + n ² −m ² ) / 2Ly｝ ² − ｛(Lz ² + n ² −l ² ) / 2Lz｝ ² ] (4) y = (Ly ² + n ² −m ² ) / 2Ly (5) z = (Lz ² + n ² −l ² ) / 2Lz (6) where SQRT (A) indicates the square root of A.

Is obtained. Here, Ly and Lz are known values determined by the arrangement positions of the ultrasonic receiving elements 31a, 31b, and 31c. Therefore, the distances n, from the ultrasonic receiving element 31a to the point (x, y, z) are Ultrasonic receiving element 31
If the distance m from b to the point (x, y, z) and the distance l from the ultrasonic receiving element 31c to the point (x, y, z) are known,
Three ultrasonic receiving elements 31a, 3 arranged three-dimensionally
The position (x, y, z) indicated by the input pen 1 on the three-dimensional coordinate system XYZ fixed to 1b and 31c can be obtained.

Here, the distance n from the ultrasonic receiving element 31a to the position (x, y, z) indicated by the input pen 1, the position (x, y) indicated by the input pen 1 from the ultrasonic receiving element 31b.
The detection of the distance m from the ultrasonic receiving element 31c to the position (x, y, z) indicated by the input pen 1 from the ultrasonic receiving element 31c is performed when the ultrasonic wave output from the tip of the input pen 1 is an ultrasonic wave. It can be obtained by detecting the time Ta, Tb, Tc until the detection by the receiving elements 31a, 31b, 31c.

That is, assuming that the spatial transmission speed of the ultrasonic wave output from the tip of the input pen 1 is C, the time Ta,
By detecting Tb and Tc, the distance n, the distance m,
The distance 1 can be obtained by n = C × Ta (7) m = C × Tb (8) 1 = C × Tc (9)

The processing for detecting the time Ta, Tb, Tc until the ultrasonic wave output from the tip of the input pen 1 is detected by the ultrasonic wave receiving elements 31a, 31b, 31c is shown in FIG.
As described above, the processing is performed by the reception processing unit 32 in the ultrasonic receiving unit 30. The input pen 1 counted by the counters 321, 322, 323 in the reception processing unit 32
The ultrasonic wave output from the tip of the ultrasonic receiving element 31a,
Times Ta, Tb, T until detected at 31b, 31c
c is held by the reception time holding circuits 325, 326, and 327, respectively, and transmitted to the arithmetic processing unit 40 shown in FIG.

When the arithmetic processing section 404 receives the times Ta, Tb, and Tc via the communication section 41, the arithmetic section 42 firstly inputs the input pen 1 based on the above-described equations (7) to (9). Calculate the distances n, m, l to the ultrasonic receiving elements 31a, 31b, and then, for the distances n, m, l,
By performing the calculations according to the above equations (4) to (6), the position indicated by the input pen 1 on the three-dimensional coordinate system XYZ fixed to the ultrasonic receiving elements 31a, 31b, 31c (x, y, z) is calculated.

In the calculating section 42, the discriminating section 43 discriminates the operation contents of the input pen 1 based on the information on the operation state of the input pen 1 received from the ultrasonic reception processing section 32 via the communication section 41. I do. Further, the display control unit 46
Is the designated position coordinate detected by the calculation unit 42 and the discrimination unit 43
The display control for the display unit 46 is performed based on the operation content of the input pen 1 determined in the above. Note that the information regarding the operation state of the input pen 1 is, for example, the reception processing unit 32
However, based on information received at the first or second ultrasonic transmission timing of the four ultrasonic transmission timings of the input pen 1 in the sequence shown in FIG. It can be realized by determining whether the operation has been performed and passing data indicating the result to the arithmetic processing unit 40.

In this way, the ultrasonic receiving elements 31a,
The coordinates of the input pen 1 in a three-dimensional coordinate system based on the installation positions of the input pens 31b and 31c are obtained.
Input processing can be performed.

By the way, as will be described below, the present invention also enables input processing with the input pen 1 on a paper document obtained by printing out an electronic document stored in, for example, the arithmetic processing unit 40. Is what you do. In order to realize input processing on the paper document by the input pen 1, it is necessary to use the input pen 1 to recognize the installation location of the paper document and an arbitrary position or area inside the paper document. This recognition method will be described below.

FIG. 25 shows an operation image in the case of performing input processing according to the operation of the input pen 1 on a paper document in an information processing system to which the pen input device according to the present invention is applied. As shown in the figure, in this operation, a paper document 10 is placed on a table or the like arranged at an arbitrary position away from the system main body 100.
And the paper document 10 is used as an input surface.

That is, in this embodiment, by pointing an arbitrary position on the display screen 50 of the system main body 100 with the input pen 1, not only the input of the specified position on the display screen 50 of the system main body 100 can be performed, but also By pointing the paper document 10 at an arbitrary position separated from the system main body 100 with the input pen 1, the pointing position for the paper document 10 can be input, and a display based on the pointing position for the paper document 10 is displayed on the display screen. 50 can be displayed. Here, the pen that points to an arbitrary position on the display screen 50 and the pen that points to an arbitrary position on the paper document 10 may be the same pen, or by using the multi-pen function of the present invention. The pen for designating an arbitrary position on the display screen 50 and the pen for designating an arbitrary position on the paper document 10 may be composed of different pens.

However, unlike the one shown in FIG. 15, the input pen 1 used in this embodiment is assumed to have n buttons as shown in FIG. Each of these buttons
It is associated to indicate each function at the time of an input operation on the paper document 10 described later.
In this embodiment, the number of input pens 1 that can be used simultaneously is three.

Now, in the operating environment shown in FIG.
The paper document 10 is disposed at an arbitrary position away from the system main body 100. In this case, the system body 1
At 00, in order to detect the position pointed on the paper document 10 by the input pen 1, the system body 1
00, the ultrasonic receiving elements 31a, 31b attached to
It is necessary to detect the relative position of the paper document 10 with respect to the arrangement position of the paper document 31c. In the present invention, in order to realize the relative position detection, prior to the position input using the paper document 10, an operation of designating a plurality of positioning areas set in advance on the paper document 10 with the input pen 1 is performed.

FIG. 27 is a diagram showing an example of setting the positioning area of the paper document 10. As shown in FIG. This paper document 10
In the system body 100, the ultrasonic receiving element 31a,
In order to notify the relative position of the paper document 10 with respect to the arrangement positions of 31b and 31c, four positioning areas P1 to P4 are provided at the four corners. When attempting to notify the relative position of the paper document 10 to the system main body 100, the operator sequentially designates the four positioning areas P1 to P4 set in advance on the paper document 10 using the input pen 1. Thereby, the system body 1
In 00 (the arithmetic processing unit 40), a process of collecting initial setting data based on the flowchart shown in FIG. 28 is performed.

As shown in the figure, when this initial setting data collection is started (step 201), first, n = 1 is set (step 202), and the display, that is, the display screen 50 of the system main body 100 is displayed. A message "Point to positioning area n with pen and press drawing button" is displayed (step 203).

Next, it is checked whether or not the drawing button has been pressed (step 204), and if it has not been pressed (step 20).
When the button is pressed (yes in step 204), the coordinates of the pen at the time the button is pressed are detected as (xn, yn, zn) (step 4). 205).

Next, n = n + 1 is set (step 20).
6) It is checked whether n = 5 (step 207). here,
If n is not 5 (no in step 207), 4
Since the detection of the individual positioning areas P1 to P4 has not been completed, the process returns to step 203, and the processing from step 203 to step 207 is repeated.

If n = 5 is satisfied in step 207 (yes in step 207), the detection of all four positioning areas P1 to P4 has been completed, and this initial setting data collection ends (step 208). .

Thus, the four positioning areas P1
To P4 (xn, yn, zn) (n = 1 to 4) can be obtained.

Actually, four corners of the paper document 10 are pointed by the pen tip of the input pen 1 and, for example, the button 1 as a drawing button is pressed. In each operation, the arithmetic processing unit 4
The arithmetic unit 42 in 0 detects the position coordinates of the pen tip of the input pen 1 and stores it. The stored positioning areas P1 to P1
As shown in FIG. 28, the coordinates of P4 are (x1, y1, z
1), (x2, y2, z2), (x3, y3, z3), and (x4, y4, z4).

After the location of the paper document 10 is specified by the above processing, a method of specifying the operation position of the input pen 1 at an arbitrary position on the paper document 10 based on the coordinates related to the installation location will be described below. . In this case, as shown in FIG. 29, the operator sets the paper document 1 whose installation location, that is, the position coordinates of the four corners, is specified.
The user operates the input pen 1 on an arbitrary internal area of “0”. At this time, if the calculated pen coordinates are (xa, ya, za), the position of the input pen 1 in the area inside the paper document 10 can be obtained by the following calculation formula.

That is, the relationship between the pen coordinates (xa, ya, za) and the positioning areas P1 to P4 in the pen instruction operation in FIG. 29 can be grasped as shown in FIG. In the figure, the horizontal distance from the positioning area P1 to the pen coordinates (xa, ya, za) is x, the distance from the positioning area P1 to the positioning area P4 is 1, and the positioning area P1
, The distance between the positioning area P1 and the pen coordinates (xa, ya, za) is m, the distance between the positioning area P4 and the pen coordinates (xa, ya, za) is n, The point at which the vertical extension of the coordinates (xa, ya, za) intersects the top side of the paper document 10 and the pen coordinates (xa, ya, za)
, The coordinates of the positioning area P1 are (x1, y1, z1), and the coordinates of the positioning area P2 are (x2, y2, z).
2) The coordinates of the positioning area P3 are (z3, y3, z3), the coordinates of the positioning area P4 are (x4, y4, z4), and the pen coordinates (x
a, ya, za) and this paper document 10
The coordinates of the intersection of the upper and lower sides of (x5, y5, z5)
(X6, y6, z6), assuming that the paper document 10 is not bent and exists on a certain plane,
The coordinates (xa, ya, za) can be calculated by the following equation.

X ² + y ² = m ² (10) (1−x) ² + y ² = n ² (11) Therefore, x = (l ² −n ² + m ² ) / 2l (12) y = SQR (m ² -((l ² −n ² + m ² ) / 2l) ² ) (13) where SQR (n) is a positive square root of n.

Therefore, (x5, y5, z5) and (x6, y6, z6)
Is determined by the following equation.

X5 = x1 + (x4-x1) × x / l; (14) y5 = y1 + (y4-y1) × x / l; (15) z5 = z1 + (z4-z1) × x / l; (16) x6 = x2 + (x6-x2) × x / l; (17) y6 = y2 + (y6-y2) × x / l; (18) z6 = z2 + (z6-z2) × x / l (19) Accordingly, the position coordinates (xa, ya, za) of the stylus pen are obtained by the following equation.

Xa = x5 + (x6-x5) × y / p; (20) ya = y5 + (y6-y5) × y / p; (21) za = z5 + (z6-z5) × y / p; (22) The above process is a process of causing the system body 100 to recognize the relative position of the paper document 10 and then detecting the coordinates of the position indicated by the input pen 1 in the area inside the paper document 10. However, it is needless to say that the area specified by the input pen 1 can be specified inside the paper document 10 by using this detection processing.

Next, an input operation by the input pen 1 on the paper document 10 in the information processing system using the pen input device of the present invention will be described. In this system, the same operation is performed by operating the input pen 1 performed on the paper document 10.
It can be performed on the electronic document corresponding to 0. In order to realize this operation, it is necessary to associate the paper document 10 with the electronic document 10 on a one-to-one basis. This system uses this paper document 1
The association between 0 and the electronic document is also performed by operating the input pen 1 on the paper document 10.
Then, by operating the content printed on the paper document 10 with the input pen 1 based on the correspondence between the documents, the electronic document (or information resource on the computer) corresponding to the paper document 10 is operated. The contents can be operated in the same manner as the pen operation on the paper document 10. The method of initializing the position where the paper document 10 is placed and the method of recognizing the position of the input pen 1 inside the paper document area, which are prerequisites for realizing this operation, are as described above.

Hereinafter, various input processes by a pen operation on a paper document in the system of the present invention will be sequentially described with specific examples. First, FIG. 31 shows a detailed configuration of the arithmetic processing unit 40 of the present system. In the arithmetic processing unit 40, a paper input control unit 44
Detects a specific operation of the input pen 1 on a paper document, and stores the paper document in a storage unit 40 in the system.
By making a one-to-one correspondence with the electronic document stored in the electronic document 1, control is performed to perform the same operation on the corresponding electronic document in response to the pen operation performed on the paper document. This paper input control unit 4
Reference numeral 4 includes a specific operation recognition unit 441, an inter-document correspondence management unit 442, and an operation reflection unit 443. In the arithmetic processing unit 40, the storage unit 401, the operation unit 402, the document processing unit 403, the printer unit 404, the external interface 40
5, various operation function units 406, a CPU 407, and the like. The arithmetic processing unit 40 is actually realized by a computer (40) having a form as shown in FIG. Therefore, it should be understood that the arithmetic processing unit 40 has a function that can cope with, for example, various processes described below, in addition to the elements illustrated in FIG.

In this system, it is necessary to prepare the paper document first before the operation with the input pen 1 on the paper document. This can be realized, for example, by printing out an electronic document created by the document processing unit 403 and stored in the storage unit 401 as a paper document by the printer unit 404 in accordance with a print instruction from the operation unit 402.

At the time of printing the electronic document, the document processing unit 403 controls so as to automatically add, for example, a marking having the content shown in FIG. 32 to the upper part of the paper document. In this case, the printing position of the marking on the paper document may be any position on the paper surface. However, this marking is 1 on the paper document.
Must be one-to-one correspondence. In the example shown in FIG.
Black circles are printed inside four of the squares. In each of the rectangles, a black circle corresponds to bit information “1”, and a black circle corresponds to bit information “0”. In the same figure, the left end of the marking is the first bit, and the right end is the last bit (LSB).
It is. Under these conditions, the paper document shown in the figure is assigned the number “0101011”. According to this example, it can be seen that a total of 128 types of paper documents can be assigned. Information on these markings is stored and managed in the storage unit 401, for example.

Next, in order to associate the paper document with the electronic document, the input pen 1 (the one having n buttons as shown in FIG. ) And pressing a button assigned in advance for the process. In order to indicate that the specific operation on the black circle has been completed, for example, the input pen 1 is moved to the upper right portion on the paper document, and the button is pressed. On the other hand, in the paper input control unit 44, the inter-document correspondence management unit 442 can previously know in which position of the paper document a square and a black circle are printed based on the correspondence management information stored in the storage unit 401, for example. ing. Further, the specific operation recognition unit 441 recognizes where on the paper document the specific button has been pressed. Accordingly, the inter-document correspondence management unit 442 performs the paper operation on which the specific operation has been performed based on the correspondence management information and the operation recognition result of the specific operation recognition unit 441, and the electronic document stored in the storage unit 401. Is associated with.

As another method for associating a paper document with an electronic document, there is a method of recognizing a figure (hereinafter, referred to as a paper print) printed on the paper document by using a specific operation of the input pen 1. FIG. 33 shows an example of a paper document used in this method. In this example, the paper pattern is arranged in the upper right rectangle on the paper document.
There is no special designation for the placement position. The user touches the paper crest accurately with the input pen 1 while pressing a pre-assigned button. Here, as a matter of course, the paper pattern must be recognizable at the input resolution in pen input. When the input of the paper pattern by the input pen 1 is completed, the input pen 1 is moved to, for example, the upper right portion of the paper document to indicate that the specific operation has been completed, and the button is pressed. In response to this operation, in the paper input control unit 44, the specific operation recognition unit 441 recognizes the button operation of the input pen 1 and a drawing pattern (paper pattern). The inter-document correspondence management unit 442 compares the recognized paper print pattern with a pre-registered pattern (correspondence management information), and stores the electronic document corresponding to the paper print pattern in the storage unit 401 with the pen operation target. Is performed in a one-to-one correspondence with the paper document.

In associating the paper document with the electronic document, there may be a case where the operator needs to input a password. As a case where this function is necessary, for example, there is a case where a password is set for an important electronic document in the storage unit 401, or a case where an individual who operates the document is used for specified use. In this system, this kind of password function is also realized by a pen input operation on a paper document. Hereinafter, two specific examples will be described using the personal identification function as an example.
Note that both the password setting function and the personal identification function can be basically handled by the method described below.

In the first example, as shown in FIG. 34, a password input area (an area written as “Tanaka”) is provided in advance on a paper document. In this example, a square area is printed to make it easy for the operator to understand. On this paper document, the operator uses the input pen 1 and presses a button previously assigned to the function, while signing an individual operator (in this embodiment, the personal name "Tananaka"). )) In the signature input area. In this operation, the paper input control unit 4
In 4, the specific operation recognizing unit 441 presses the button of the input pen 1 and the locus of the signature (personal name pattern).
Is recognized. The inter-document correspondence management unit 442 compares the recognized personal name pattern with a pre-registered personal name pattern (correspondence management information), specifies the operator based on the result of the comparison, and then associates the paper document with the electronic document. Make correspondence with the document.

As another method for specifying the operator, there is a method using an on / off operation of a button provided on the input pen 1. In this case, first, the input pen 1 is moved to the signature input area on the paper document. next,
A plurality of buttons mounted on the input pen 1 are pressed in the order according to the button input pattern assigned to each individual in advance. In response to this operation, the specific operation recognition unit 4
Reference numeral 41 recognizes the button pressing pattern, and the inter-document correspondence managing unit 474 compares the pressing pattern with a previously registered pressing pattern (corresponding management information), and specifies the operator based on the matching result. As the simplest method of realizing this example, a method in which three buttons are mounted on the input pen 1 and individuals are specified in the order in which these three buttons are pressed can be considered.

As described above, after the process of associating the paper document with the electronic document, position registration is performed to make the arithmetic processing unit 40 recognize the relative position of the paper document. In this case, as shown in FIG. 35, the operator performs an operation of sequentially moving the input pen 1 to the four corners of the paper document and pressing a specific button at each of the four corners. This button operation is the same as the operation described in FIG. In response to each of these button operations, the arithmetic unit 42 collects initial setting data through the processing shown in FIG. 28, and calculates three-dimensional position coordinates of the pen tip of the input pen 1 (four corners of the paper document). Storage unit 401
To memorize. Thereafter, when an arbitrary position is designated on the paper document by the input pen 1, the calculation unit 42 calculates the above formulas (20) to (22) based on the already calculated position coordinates of the four corners of the paper document. As described above, any designated position in the paper document can be detected by using. Then, in the paper input control unit 44, the storage unit 4
Based on the position coordinates stored in No. 01 and the arbitrary designated position coordinates notified from the arithmetic unit 42, it is possible to recognize which position of the paper document has been designated, and perform various operation reflection processes described below.

In this example, a case has been described where the pen pointing positions at the time of position registration are set at the four corners of the paper document in order to enable input processing in the entire area of the paper document. Can be set arbitrarily. For example, in the case where the contents are discussed using the input pen 1 on a paper document while displaying the electronic document on the display unit 50 of this system, the display unit 50 does not display the entire page of the document,
There are cases where only a partial area is desired to be displayed. Further, there are cases where the operator wants to perform an operation on a paper document only in a partial area (for example, in order to avoid erroneous input, the input in an area other than a necessary part on the paper document is ignored. If you want to).

In such a case, only the necessary area of the paper document may be registered, and only the relevant area may be displayed on the display unit 50. FIG. 36 (a)
Shows an example of a position registration method of a partial area of a paper document. In this case, the entire area of one paper document is divided into a plurality of areas in advance, and
Numbers as shown in FIG. Thus, for example, a shadowed portion in the drawing can be designated by a section number of 2 horizontally and 2 vertically. The operator moves the input pen 1 to the four corners of each of the two horizontal and vertical section numbers, and at each of the four corners, a button () assigned to the area position registration function in advance. Button A). In this case, the position designation order for the four corners is performed in a predetermined order. Next, in order for the arithmetic processing unit 40 to recognize the designated area, the buttons corresponding to the respective section numbers are pressed in the horizontal and vertical order. By performing the above processing, a specific area in the paper document can be designated. Thereafter, the arithmetic processing unit 40 controls display on the display unit 50 and sets an input operation for a specific area inside the paper document based on the area designation result.

In the example of FIG. 36A, the area is specified by
Although the process is performed for only one region, a plurality of regions may be performed. Accordingly, it is possible to display the information on the display unit 50 and to perform an input operation on a plurality of areas. FIG. 36B shows an example of a position registration method in a case where a plurality of areas are specified and an input operation is performed on the plurality of areas. In this case, the position registration can be easily realized by using the button B for adding an area designation instead of the button A described above.

After the paper document and the electronic document are associated with each other and the positions of these paper documents are registered as described above, the operation result of the input pen 1 on the paper document is reflected on the electronic document. It is possible to do. This processing function is performed by the operation reflecting unit 442 in the paper input control unit 44. In this case, if the electronic document is displayed on the display unit 50, the operation can be performed while confirming the pen operation on the paper document on the display unit 50.

On the other hand, it is desirable that various operations performed by the operator can be visually confirmed even on a paper document. To achieve this, the system of the present invention uses the input pen 1 with a writing instrument (see FIGS. 18 and 19). By using this function, the operator can perform the pen operation performed on the paper document by using the display unit 50.
It is possible to check the written contents on the paper document without looking at the electronic document displayed by the, and to improve the operability. For example, in FIG.
By using the function of writing on a paper document provided on the input pen 1 when inputting a signature for specifying an individual operator, the operator can easily input the signature. Also, in various pen operations on a paper document described below, it is very useful to be able to record the trajectory of the input pen 1 on the paper document. The first reason is that the operator can contribute to the prevention of erroneous input by performing pen input while recognizing the contents of the pen locus, similarly to the above-described sign input. The second reason is that, by recording the operation contents for each object on the paper document described later on the paper document, it becomes possible to determine at a glance what kind of operation has been performed. At this time, the writing color may be changed for each pen operation content, and in this case, the determination is further facilitated.

Hereinafter, a specific example of the processing of reflecting the pen operation result on the paper document in the electronic document in the system of the present invention will be sequentially described. FIG.
Shows an example of an operation method for selecting information (hereinafter, referred to as an object) such as characters / graphics printed on a paper document. In this case, the operator moves the input pen 1 so as to surround the area of the object to be selected while pressing a button (selection button) assigned in advance for object selection. At this time,
A pen locus may be drawn by a writing implement mechanism provided on the input pen 1. FIG. 37 shows an operation example when a certain area is selected as an object.
To select a single object, as shown in FIG. 38, the input button 1 may be moved over the relevant object, and then the selection button may be pressed. When selecting an object by this method, it is not necessary to draw a pen track on a paper document.

In response to the object selection operation, in the paper input control unit 44, the specific operation recognition unit 441 recognizes what position (or area) and what kind of object selection has been made. The operation reflecting unit 443 specifies an object (single or area) corresponding to the above selection operation on the electronic document managed by the document correspondence management unit 442 based on the recognition result,
A process is performed on the specified object to reflect an operation corresponding to a pen operation performed on the paper document thereafter.

The operation reflection processing for the selected object includes, for example, processing such as duplication, movement, deletion, editing, attribute change, and execution of the object. When copying or moving the selected object, as shown in FIG. 39, for example, the operator copies the input pen 1 while pressing the copy / move button corresponding to the copy or movement command after the object selection operation. Or move to the place you want to move. At this time, it is desirable that the movement trajectory is drawn on the paper document in the writing color indicating the duplication or movement. Assuming such a use, the writing implement mechanism of the input pen may have a structure in which a pen tip of a corresponding writing color is selected in accordance with pressing of each button such as a copy / move button.

When an object is to be deleted, for example, as shown in FIG. 40, after the object is selected, the operator presses the delete button using the input pen 1 and presses the delete button on the object. Perform an operation such as writing a mark. At this time, it is desirable that the drawing color of the X mark also be a color corresponding to the deletion. As another example of the deletion operation, for example, the object can be configured to be traced with the input pen 1 while pressing the eraser button, and the traced portion of the object is deleted. It is also possible to actually attach an eraser to the pen.

To edit an object, select an object by the above selection operation, and then
Is operated to perform writing corresponding to the edited content on the object. The written content is recognized by the specific operation recognition unit 441, and the operation reflection unit 443 performs an additional correction process on the electronic document based on the recognition result. As an example of changing the attribute of an object, for example, if the object is to be enlarged, the input pen 1 is shaken above the paper document after selecting the target object, and if the object is to be reduced, the input pen 1 is moved downward. Perform a shaking operation. Also,
If it is desired to change the color of the selected object (area), for example, an operation of writing a character indicating the color on the paper is performed. FIG. 41 shows an example of a pen operation when the information in the central area of the paper document is enlarged and changed to red, and the input pen 1 is swung upward and corresponds to the red to be changed ( RED).

In response to the above pen operations of move, copy, delete and edit (FIGS. 39 to 41), the paper input control unit 44
Based on the drawing trajectory information of the input pen 1 input from the arithmetic unit 42, the specific operation recognizing unit 441 recognizes that the commands are movement, duplication, deletion, and editing, and the operation reflecting unit 442 recognizes the command. Based on the result, the corresponding object on the electronic document corresponding to the paper document is moved, copied, deleted, and edited.

The above-described paper document is a program or data associated with a local or network computer or a URL (Universa) on the Internet as an object to be selected by the input pen 1.
l Resource Locator) may be printed on link information. In the system of the present invention, selection for such an object on a paper document,
Processing such as duplication, movement, deletion, and execution can be performed.

FIGS. 42 and 43 show an example of a paper document applicable to this type of processing. Especially,
The paper document in FIG. 42 is obtained by printing a character string indicating two programs (Game0, Game1) and three sound data (Song0, Song1, Song2) as the above objects. Is a print of two URLs on the Internet as the above objects. On the paper document shown in FIG. 42, after the input pen 1 is placed on one of the above-mentioned program or sound data object and the execution button is pressed, the paper input control unit 44
The specific operation recognition unit 441 recognizes that the execution instruction has been issued and the position where the instruction has been issued. At this time, the inter-document correspondence management unit 442 manages the electronic document corresponding to the paper document. Operation reflection unit 443
Is the specific operation recognition unit 44 on the electronic document.
1 to identify the link information at the position recognized by the user and execute the program corresponding to the object whose execution is instructed, or execute the sound data (Song Song) corresponding to the object whose execution is instructed based on the link information.
0, Song1 or Song2). Similarly, on the paper document shown in FIG. 43, after the input pen 1 is placed on one of the objects consisting of the above two URLs and then an operation of pressing the execution button is performed, the operation reflecting unit 443 causes In the paper document on the electronic document, link information at a position corresponding to the position indicated by the input pen 1 is specified, and based on the link information, control is performed to display a corresponding URL home page on the display unit 50.

In the paper documents shown in FIGS. 42 and 43, the link destination is a program or a URL, but the link destination may be another paper document.
The operation method in this case will be described with reference to FIGS. First, FIG. 44 shows that, in a material composed of a plurality of paper documents (10-1, 10-2, 10-3,..., 10-n), each paper document (10-1, 10-2, 1
0-3,..., 10-n), a paper pattern for uniquely recognizing each document is printed on the upper right. Here, each paper document (10-1, 10-2, 10
-3, ..., 10-n) upper right letters "A1, A2, ..., An"
Indicates a paper pattern, in which the number following “A” indicates the number of pages of each paper document. Paper input control unit 4
As described above, in 4, the paper document to be operated can be individually recognized by using the paper print.

Further, according to the present invention, link information indicating a link to another paper document is printed for each of these paper documents (10-1, 10-2, 10-3,..., 10-n). I do. FIG. 45 shows the contents of the first page of the paper document (10-1) in the plurality of documents shown in FIG. 44. This is link information to 10-2). here,
“A2” is a paper print uniquely associated with the second paper document (10-2) (see FIG. 44). On the first page of the paper document (10-1), by using the input pen 1 and pressing the execution button on the link information "A2" at the lower right of the page, the management target of the document correspondence management unit 442 is set. The electronic document can be switched to an electronic document corresponding to the paper document (10-2) having the paper pattern "A2".

That is, in response to the pen operation, in the paper input control unit 44, the specific operation recognizing unit 441 recognizes that the instruction for execution has been issued and the position of the instruction. At this time, the inter-document correspondence management unit 442 manages an electronic document corresponding to the paper document (10-1). The operation reflecting unit 443 specifies link information of the position on the electronic document recognized by the specific operation recognizing unit 441, and, based on the link information, the second page of the paper document corresponding to the execution-instructed object. A process for expanding the electronic document corresponding to (10-2) is executed.

After the pen operation, if the operator superimposes the paper document (10-2) on the second page directly above the paper document (10-1) on the first page, the development is performed. Since the position of the electronic document corresponds to the position of the second page paper document (10-2), the position of the second page paper document (10-2) can be continuously set without performing the position setting operation of the second page paper document (10-2). A pen input operation can be performed for the paper document (10-2). The second page paper document (10-2)
Is used at a position distant from the first page of the paper document (10-1), the operator uses the method described with reference to FIG.
The location registration operation of 0-2) may be performed.

FIG. 46 is a diagram showing an operation example of the input pen 1 when the user wants to use the paper document (10-2) of the second page at a position away from the paper document (10-1) of the first page. . In this case, when the position registration of the second page of the paper document (10-2) is completed, the registration of the first and second pages of the paper document is recognized by the arithmetic processing unit 40. A pen input operation can be performed using a sheet of paper document.
When this function is used, a plurality of input pens 1a,
1b,... Can be used, a plurality of operators input a plurality of paper documents (10-1, 10-2, 1).
0-3,..., 10-n) can be operated simultaneously. Further, the plurality of paper documents (10-1, 10-2, 10-3,.
-n) may correspond to the same paper document.

Next, another paper document handled by the present system and a pen input operation on the paper document will be described. FIG. 47 shows an example of a paper document on which the configuration of a data file handled by the arithmetic processing unit 40 of the present system is printed. In order to perform input processing using this paper document, first, a paper pattern printed on the upper right of the paper document is traced with the input pen 1 in the manner described with reference to FIG. The management unit 442 is made to recognize it. Next, the positions of the four corners of the paper document are designated by the input pen 1 in the order shown in FIG. 47, and the "execute" button is pressed at each designated position to register the position in the arithmetic processing unit 40. I do.

The operation of selecting an object on the paper document after the completion of the position registration is performed by the method described with reference to FIG. As a result, on the paper object shown in FIG.
When the input pen 1 is moved to an object indicating “” and the selection button is pressed, the specific operation recognizing unit 444 of the paper input control unit 44 determines, based on the pointing position of the input pen 1, the electronic document corresponding to this paper document. It is recognized that the operation corresponds to the file “picture0”.

Next, the selected file "picture0"
Is to be copied or moved to the tmp directory, the operator pushes the input pen 1 while pressing the copy or move button corresponding to the copy or move operation, respectively, as shown in FIG.
Move to the mp directory. At this time, in order to make the operation performed by the operator easy to understand, it is also possible to draw the movement trajectory on the paper document in a writing color corresponding to the operation as shown in FIG.

The operation of deleting a file on a paper object is performed by a pen operation as shown in FIG. FIG. 11 is an operation example particularly when tmp_file0 in the tmp directory is deleted. In this case, the operator may select the object to be deleted, tmp_file0, with the input pen 1 on the paper document, and then draw an X (cross) mark on the object as illustrated.

Editing and execution operations on an object on a paper document are performed by moving the input pen 1 on the target object and double-clicking a specific button. By this pen operation, for example, in the case of a data file such as an image file, an editing program is started, and the target file becomes editable. In the case of a program file, the program is started by the pen operation.

Further, with respect to the operation of changing the attribute of the file on the paper document, a drawing operation corresponding to the changed content is performed by the input pen 1 on the object to be changed. FIG. 51 shows an example of a pen operation related to a file writing attribute (whether writing is permitted or not). As shown in FIG.
If you want to allow writing, first, the file pictur
After selecting e0, "W
○ ”may be drawn. On the other hand, in the case where the target file picture2 is write-protected, it is sufficient to first select the file picture2 and then draw “W ×” in the vicinity of the file picture2 to indicate write-protection.

Next, mail transmission control using a paper object will be described. In this case, for example, a paper document on which a mail transmission form as shown in FIG. 52 is printed is used. As shown in the figure, in this paper document, a paper pattern "M1" for uniquely recognizing the paper document is printed in the upper right portion. In this paper print, the letter M indicates a mail transmission form, and the number following the letter indicates the number of a mail sheet stored in the storage unit 401 of the arithmetic processing unit 40. The area a on the paper document is an area for writing a mail text. In the area b, a mail address is printed. Further, the area in which the letters "send" is drawn is used as a command for instructing the transmission when the created mail document is actually transmitted.

Also in the case of performing input processing using this paper document, the paper print printed on the upper right thereof is traced by the input pen 1 and the paper document is recognized by the inter-document correspondence management unit 442 of the paper input control unit 47. At the same time, the position of the four corners of the paper document is designated with the input pen, and the “execute” button is pressed at each designated position, thereby performing position registration with respect to the arithmetic processing unit 40.

After the position registration is completed, an operation of drawing a message to be transmitted or the like with the input pen 1 is performed in the area a on the paper document. As a result of this drawing operation, in the paper input control unit 44, the input pen 1 is used to draw and input the mail transmission form corresponding to the paper document managed by the inter-document correspondence management unit 442 to the area a corresponding to the area a. An email document with the embedded content is generated.

FIG. 53 shows the generated mail document (actually, the print result of the mail document). Among these, the content described in the area a is the content drawn and input by the input pen 1. This content is used as the mail text, for example, in the storage unit 4 of the arithmetic processing unit 40.
01 is registered. The contents can be converted into character information by a character recognition system, and the character information can be transmitted as a mail text. In this case, the transmission address is specified on the paper document in FIG. The mail address book is printed in the area b of the paper document. The operator specifies a transmission address from the mail address book. As an actual operation, the operator moves the input pen 1 to a position where a desired address is printed, and
Press the execute button. After the designation of the transmission address, a transmission instruction is performed. As the actual operation of this transmission instruction,
The input pen 1 is moved to the position where "Send" is printed on the paper document, and the execution button is pressed. After this operation, the arithmetic processing unit 40 automatically transmits the mail including the contents of the above-described mail document to the transmission address designated by the pen operation in the area b via the external interface 405.

In the examples shown in FIGS. 52 and 53, a paper document on which a mail address book is printed is used. As a modified example, a paper document on which a mail sheet dedicated to a specific destination is printed is used. May be. In this case, the destination of the mail is one-to-one with the paper crest on the paper document.
In order for the operator to recognize to whom the mail sheet is to be sent, it is desirable to print the address on the mail sheet, for example, in a mail body area. Figure 5 shows this type of e-mail transmission form.
Second, as compared with the mail transmission form shown in FIG. 53, an advantage is that the corresponding content of a specific destination can be previously stored in a mail sheet.

FIG. 54 shows an example of a paper document according to this modification. The paper document of this example forms an e-mail transmission form dedicated to a specific transmission destination in which the transmission destination of the e-mail is fixed to “Tanaka-san”. In the area a of the e-mail transmission form, the contents (“Sama Tanaka, goodbye.”) Necessary for the e-mail document to be transmitted to “Mr. Tanaka” are printed in advance.
On the paper document having the mail transmission form, the operator writes the mail text by the drawing operation of the input pen 1 in the manner described above. FIG. 55 shows a state after the operator has drawn (written) the mail text on the paper document shown in FIG. 54 using the input pen 1.
By this drawing operation, the contents input and drawn by the input pen 1 are embedded in the corresponding portion of the mail transmission form corresponding to the paper document managed by the inter-document correspondence management unit 442 in the paper input control unit 44. The generated mail document is generated. Thereafter, when the operator moves the input pen 1 to the area on the paper document where “Send” is printed and presses the execution button, the content of the mail document is transmitted to the arithmetic processing unit 40 by the specific transmission. It is automatically transmitted via the external interface 405 to the destination (addressed to Mr. Tanaka) associated with the destination-specific mail transmission form.

In the examples of FIGS. 54 and 55, the contents such as “Hello Tanaka, goodbye.” Etc. on the mail document can be essentially added to the mail document to be transmitted. Considering the operability of the operator when creating the mail text, it is desirable that the contents be printed in advance on a paper document.
Using a mail transmission form specialized for this transmission, a mail document can be created with relatively little trouble.

Next, still another modified example of the paper document for transmitting the mail will be described. In the paper document of this modified example, objects corresponding to already created sentences, pictures, files, etc. are printed, and these objects are selected by a pen operation to embed the corresponding sentences, pictures, files, etc. in the mail document. By doing
It is intended to make mail transmission work more efficient. FIG.
Reference numeral 6 denotes an example of a paper document according to this modification, which includes a paper print, an area for instructing transmission, an area a for writing a mail text, an area on which a transmission address is printed, A mail transmission form having an area in which objects corresponding to texts, pictures, files, and the like are printed.

Mail transmission using a paper document having this mail transmission form is performed by a pen operation as shown in FIG. First, the operator uses the input pen 1 to select an object to be used on the paper document, and then performs an operation of copying the object to a desired arrangement position in the area where the mail text is to be entered. The figure particularly shows an example of pen operation when a fixed text 0 and an illustration 1 are arranged at desired positions in the text entry area.

In response to this pen operation, the operation reflecting unit 443 of the paper input control unit 44 sends the
Then, a mail document is generated in which a fixed text 0 and an illustration 1 which are objects duplicated by the input pen 1 are embedded in an electronic document corresponding to the paper document being managed in step (1). In the document generation operation using the paper document in this example, the operation of embedding frequently used texts and pictures in the body of the mail document can be greatly improved. After that, in order to transmit this mail document, the transmission operation described in FIG. 53 and FIG. 55 is performed using the fact that the paper document is provided with a transmission instruction area and a transmission address printed area. Can be dealt with.

Further, in this system, a real object such as a piece of paper or a doll is placed on a paper document, and the input pen 1 is moved in accordance with the movement of the real object. By recognizing the location of the real object on the electronic document to be displayed on the display unit 50, it is possible to enjoy a game by pen input from a paper document. FIG. 58 shows an example of a paper document used in this case. Specifically, a shogi board for shogi is printed. On this shogi board,
Each piece of Shogi (real object)
Are lined up. Here, since the arrangement of each piece in the initial state is always constant, it is not necessary to cause the arithmetic processing unit 40 to recognize the arrangement state of these pieces. That is, in the storage unit 401 of the arithmetic processing unit 40, an electronic document corresponding to the paper document on which the shogi board is printed is stored in advance, and when starting shogi, the inter-document correspondence management unit 442 stores the electronic document. In response to a pen operation (for example, an operation shown in FIGS. 32 and 33) on a paper document (shogi board), a corresponding electronic document need only be specified and displayed on the display unit 50.

In order to enjoy shogi on this paper document, the following operation is performed. That is, when it is his / her turn, the player selects a piece to be moved with the input pen 1 and further moves the piece to a desired position with the input pen 1. During this moving operation, the moving button of the input pen is kept pressed. FIG. 59 shows an operation example of the input pen 1 at the time of moving the piece. The operation of the input pen 1 allows the arithmetic processing unit 40 to recognize the movement of the piece, and displays the same situation as the arrangement of the piece on the paper object on the display unit 50 based on the recognition result. . This allows non-opponents to be on paper objects,
The battle status can be confirmed on the display unit 50. Needless to say, the game record of the battle at this time can be stored in the storage unit 401. In addition, it is also possible to restart shogi from a certain game record state by using the saved result.

Further, when recognizing the initial position of a piece or the like, a measure is taken, for example, by a pen operation as shown in FIG. First, an object corresponding to each piece is printed in a piece area on the left side of the paper on a paper document serving as a shogi board. When arranging pieces, the operator must first
A piece to be arranged is specified using the input pen 1 in the piece area. This can be executed by moving the input pen 1 to a desired frame in the frame area and pressing the selection button, as in the processing described above. This allows
The arithmetic processing unit 4 can recognize the type of frame desired by the operator. Next, the copy button of the input pen is pressed, the input pen 1 is moved to a position where the selected frame is to be moved, and the pressing of the copy button is released. A real object corresponding to the selected piece is arranged at the copy destination of this piece. By performing such a pen operation for each frame, the initial stage arrangement of the frames as shown in FIG. 60 can be realized while maintaining the correspondence between the paper document and the electronic document. In the example shown in FIG. 60, the arrangement of the pieces at the initial stage is recognized. However, an arbitrary piece may be arranged at an arbitrary position to recognize the positional relationship between them. Thereby, for example, it is possible to cope with stuffed shogi. In each of the above examples, only the movement or duplication of an object (piece) has been described. However, it is apparent that the object can be dealt with by referring to the above description.

By the way, each of the above-mentioned examples of paper documents has information such as a document created by an application such as word processing software in the arithmetic processing unit 40 and printed out by the printer unit 404. That is, in each of the above-described examples, it is assumed that an electronic document corresponding to a paper document always exists in the system body. However, the present invention can also handle paper documents such as magazines that do not originally exist in the system body. For example, as shown in FIG. 61, a certain magazine A is printed with the paper print of magazine A printed on each page and the number of pages, such as P30, following the same crimper operation as in the past. Can be recognized by the arithmetic processing unit 40. Using this function, an object (content) printed on a certain page is selected with the input pen 1 on the paper document (magazine) by the same pen operation as described above. The contents are determined using the function), and can be recognized. On the other hand, information corresponding to the magazine A is prepared as a database on a CD-ROM or a network.
Then, according to the recognition result, the external interface 4
05, the corresponding contents may be copied from the CD-ROM, the network, or the like. in this case,
Unlike the current operation using a keyboard and a mouse, desired information can be extracted by performing a pen operation on the magazine itself. The object (information) selected by the input pen 1 and subjected to the duplication operation is automatically pasted on a scrap document or the like set in the arithmetic processing unit 40 and can be reused.

In addition, it goes without saying that the present invention can be variously applied or modified without departing from the above-mentioned gist. For example, copying paper documents,
Regarding the movement, a configuration may be adopted in which the area to be moved is specified together with the specification of its size. In the area specification in this case, the destination position is specified by the input pen 1 using the area specification button of the input pen 1. In addition, the change in size can be designated by setting a destination area. Further, in the present invention, the multi-pen function is supported. In connection with this function, the operation on a paper document is set by setting different colors for the writing colors of a plurality of input pens used simultaneously. It is possible to easily identify the operator who has performed. In addition, each input pen may be dedicated to a specific operation. In this case, it is extremely easy to recognize each operation on the paper document.

[0178]

As described above, according to the present invention, the paper document and the electronic document are associated one-to-one according to the specific operation of the input pen on the paper document. In addition, since the operation of the input pen on the paper document is reflected on the corresponding electronic document, the electronic document corresponding to the paper document is interactively interacted with the computer using the paper document. Various operations corresponding to pen operations on a document can be performed in real time.

Here, the link information to another paper document is provided on the paper document, and the electronic document corresponding to the another paper document is managed by the inter-document management based on the operation of selecting the link information by the input pen. With this configuration, one or more operators can perform the operation from a plurality of paper documents from arbitrary positions without being restricted by the input area.

If a writing instrument is provided on the input pen and writing corresponding to the operation content can be performed in accordance with the operation content for the paper document, the corresponding electronic document can be operated using the paper document. In this case, the contents of the operation performed on the paper document remain as writing information on the paper document, and the correction history can be easily confirmed.

According to the seventeenth aspect of the present invention, an external electronic document corresponding to the drawing information is fetched from the outside based on the specific drawing information by the input pen on the paper document, and 1 Since the operation of the input pen on the paper document is reflected in the corresponding external electronic document after being managed in a one-to-one correspondence, electronic objects such as magazines which do not originally exist in the system main body can be used. Various operations can be performed through pen operations on corresponding paper documents.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram for explaining the principle of position detection of a pen input device according to the present invention.

FIG. 2 is a schematic configuration diagram of an information processing system to which the pen input device according to the present invention is applied.

FIG. 3 is a block diagram showing an embodiment of the information processing system in FIG. 2;

FIG. 4 is a block diagram showing a functional configuration of a reception processing unit of the ultrasonic receiving unit shown in FIG.

FIG. 5 is a timing chart showing a relationship between a basic clock and an ultrasonic wave transmission timing in the stylus pen shown in FIG. 4;

FIG. 6 is a timing chart showing an operation of detecting an ultrasonic reception time by the counter shown in FIG. 4;

FIG. 7 is a block diagram showing an internal circuit configuration of a stylus pen used in the information processing system shown in FIG. 3;

FIG. 8 is a block diagram illustrating a functional configuration of an ultrasonic receiving unit and an arithmetic processing unit illustrated in FIG. 3;

FIG. 9 is a timing chart showing a basic operation of ultrasonic transmission of the stylus pen shown in FIG. 7;

FIG. 10 is a table showing variable setting conditions of an ultrasonic transmission interval Twait in FIG. 9;

FIG. 11 is a diagram showing a state of a path through which ultrasonic waves transmitted from a stylus pen are received by an ultrasonic receiving element.

12 is a view for explaining an ultrasonic reception signal in the ultrasonic reception element in the case shown in FIG. 11;

FIG. 13 is a view for explaining inconvenience in the case shown in FIG. 12;

FIG. 14 is a diagram showing an external configuration of a stylus pen used in the information processing system shown in FIG. 3;

FIG. 15 is a diagram showing an ultrasonic transmission timing of the stylus pen when the multi-pen function is selected.

FIG. 16 is a state transition diagram of ultrasonic transmission timing control by a stylus pen.

FIG. 17 is a state transition diagram of reception timing control by the ultrasonic receiving unit.

FIG. 18 is a diagram illustrating a configuration example of a stylus pen having a function as a writing instrument.

FIG. 19 is a view showing another configuration example of a stylus pen having a function as a writing instrument.

FIG. 20 is an enlarged configuration diagram showing a portion surrounded by a dotted line in FIG. 19;

FIG. 21 is an enlarged configuration diagram showing a portion of the ultrasonic wave passing window shown in FIG. 19;

FIG. 22 is a diagram showing a mounting form of an ultrasonic receiving element in the information processing system of FIG. 3;

FIG. 23 is a view showing a state of propagation of ultrasonic waves in the mounting mode shown in FIG. 22;

FIG. 24 is a view for explaining the principle of detecting stylus pen position coordinates in the mounting mode shown in FIG. 22;

FIG. 25 is a view showing an operation image when a pen input operation is performed on a paper document in the information processing system shown in FIG. 3;

FIG. 26 is a diagram showing an external configuration of a stylus pen used in the operation mode shown in FIG. 25.

FIG. 27 is a diagram showing an example of setting a positioning area relating to position registration of a paper document.

FIG. 28 is a diagram showing a process of collecting initial setting data related to position registration of a paper document.

FIG. 29 is a diagram illustrating an operation of instructing a stylus pen when detecting coordinates of an internal area position of a paper document.

FIG. 30 is a view showing how to capture the coordinates of the pointing position of the stylus pen and the respective positioning areas related to the detection of the coordinates of the internal area of the paper document.

FIG. 31 is a diagram showing a detailed configuration of an arithmetic processing unit in FIG. 3;

FIG. 32 is a diagram illustrating an example of a paper document that can be associated with an electronic document.

FIG. 33 is a view showing another example of a paper document that can be associated with an electronic document.

FIG. 34 is a view showing a paper document that enables password setting.

FIG. 35 is a diagram showing a position registration operation of a paper document.

FIG. 36 is a diagram showing a position registration operation of a partial area of a paper document.

FIG. 37 is a diagram showing an example of an object selecting operation on a paper document.

FIG. 38 is a view showing another example of an object selecting operation on a paper document.

FIG. 39 is a diagram showing a copy or move operation of a selected object.

FIG. 40 is a diagram showing a deletion operation of a selected object.

FIG. 41 is a view showing an attribute changing operation of a selected object.

FIG. 42 is a view showing an example of a paper document having link information as an object.

FIG. 43 is a view showing another example of a paper document having link information as an object.

FIG. 44 is a view showing a paper pattern printing mode of a material composed of a plurality of paper documents.

FIG. 45 is a diagram showing a paper document having another paper document as a link destination.

FIG. 46 is a diagram showing a position registration operation in a linked paper document.

FIG. 47 is a view showing a paper document formed by printing file management information.

FIG. 48 is a view showing an object selecting operation on the paper document in FIG. 47.

FIG. 49 is a view showing the copy or move operation of the selected object in FIG. 47.

50 is a view showing an object deletion operation on a paper document in FIG. 47.

FIG. 51 is a diagram showing an object attribute changing operation on the paper document in FIG. 47.

FIG. 52 is a view showing an example of a paper document formed by printing a mail transmission form.

FIG. 53 is a view showing the mail generation / transmission operation on the paper document in FIG. 52;

FIG. 54 is a view showing another example of a paper document formed by printing a mail transmission form.

FIG. 55 is a view showing a state of the paper document in FIG. 54 after the mail text is entered.

FIG. 56 is a view showing still another example of a paper document formed by printing a mail transmission form.

FIG. 57 is a view showing the mail generation operation on the paper document in FIG. 56;

FIG. 58 is a view showing an example of a paper document formed by printing a shogi board.

FIG. 59 is a view showing a real object moving operation on the paper document in FIG. 58;

FIG. 60 is a diagram showing an initial position registration operation of a piece on a paper document formed by printing a shogi board.

FIG. 61 is a view showing a paper pattern printing form on a magazine used as a paper document.

[Explanation of symbols]

1,1a, 1b ... Stylus pen, 10,10-1,10-
2, 10-3, ..., 10-n: paper document, 11: transmission timing control unit, 12: sequencer, 13: oscillator,
14 button, 14a first button, 14b second button, 15 amplifier section, 16 operational amplifier, 17 transmitting section, 18 ultrasonic transmitter, 30 ultrasonic receiving section, 31 ...
Receiving sensor unit, 31a, 31b, 31c: ultrasonic receiving element, 32: receiving processing unit, 321, 322, 323: counter, 324: basic clock generating circuit, 325, 32
6,327: reception time holding circuit, 60: amplifier section, 61
.., Operational amplifier, 70, reception timing control unit, 71, counter, register, 72, sequencer, 73, oscillator, 74, SCC (serial communication control unit) initialization unit, 3
3 communication unit, 331 SCC (serial communication control unit),
332 ... 232C driver, 34 ... pen holder, 35 ...
Synchronous circuit, 40 arithmetic processing unit, 41 communication unit, 42 arithmetic unit, 43 determination unit, 44 paper input control unit, 441 specific operation recognition unit, 442 inter-document correspondence management unit, 4
43: operation reflection unit, 401: storage unit, 402: operation unit,
403: Document processing unit, 404: Printer unit, 405: External interface, 406: Various operation function units, 407:
CPU (central processing unit), 45: display control unit, 50: display unit

Claims (21)

[Claims] 1. A pen input device for detecting an indicated position of an input pen having an operation content indicating function and performing an input process based on an operation of the input pen based on the indicated position, wherein at least a storage means for storing an electronic document; A printing unit that prints out the electronic document as a paper document; an operation recognition unit that recognizes an operation content of the input pen on the paper document; and a paper unit according to a specific operation of the recognized operation content. An inter-document correspondence management unit that manages an electronic document corresponding to a document in a one-to-one correspondence with the paper document; and, based on the operation content recognized by the operation recognition unit, the document on the paper document. Operation reflecting means for reflecting the operation of the input pen on the corresponding electronic document. Pen input device according to claim. 2. The pen input device according to claim 1, wherein the specific operation is an operation of instructing a predetermined information pattern in a predetermined marking area on the paper document by the input pen. . 3. The pen input device according to claim 1, wherein the specific operation is a drawing operation of predetermined graphic information in a predetermined marking area on the paper document by the input pen. . 4. The method according to claim 1, wherein the specific operation includes an operation of drawing information corresponding to a predetermined password in a predetermined area on the paper document by the input pen. Pen input device. 5. Recognizing a position indicating operation of the input pen at an arbitrary position on the paper document, a region on the paper document formed from the pointing position of the input pen at the time of the recognition and the electronic document are recognized. Position relation setting means for setting a position relation with an area to be formed; display control means for displaying the contents of an area corresponding to the setting area of the electronic document on a display means based on the setting area by the position relation setting means; 2. The pen input device according to claim 1, further comprising an input area control unit that controls so as to enable a process of reflecting an operation by the input pen on the electronic document only in the setting area. 6. The display control unit according to claim 1, wherein the display control unit displays, on the display unit, an operation content of the input pen and an operation reflection result on the electronic document by the operation reflection unit based on the operation content. 6. The pen input device according to 5. 7. The operation content instructing function of the input pen is configured to be capable of instructing at least one of selection, duplication, movement, deletion, editing, attribute change, and execution of an object on the paper document, 6. The method according to claim 5, wherein the operation reflecting unit performs an instruction operation corresponding to a corresponding object on the electronic document based on a recognition result of each of the instruction operations by the operation instruction function by the operation recognition unit. A pen input device as described. 8. The object on the paper document is link information, and the operation reflecting unit performs each of selection, duplication, movement, deletion, and execution of the link information by the input pen on the paper document. 8. The pen input device according to claim 7, wherein a corresponding instruction operation for the link information is performed for the instruction operation. 9. The object according to claim 7, wherein the object on the paper document is a program, and the operation reflecting unit executes the program based on an operation instruction operation of the object by the input pen. Pen input device. 10. The paper document includes an object that is transmission destination information and an object that functions as a transmission instruction command, and the operation reflecting unit is configured to execute the transmission destination information based on an operation of selecting the former object using the input pen. 8. The pen input device according to claim 7, wherein the information is displayed on the display unit, and the corresponding electronic document is transmitted in accordance with the transmission destination information based on the latter instruction to execute the object. 11. The object on the paper document includes a fixed sentence and an illustration, and the operation reflecting means is configured to select, copy, and move the object with the input pen to perform the fixed sentence and the illustration, respectively. 11. The pen input device according to claim 10, wherein is selected and copied or moved onto the corresponding electronic document. 12. The object on the paper document is file management information, and the operation reflecting means performs each instruction operation of selecting, duplicating, moving, deleting, editing, changing attributes, and executing the object with the input pen. 8. The pen input device according to claim 7, wherein a corresponding instruction operation is performed on the file management information at the instruction position based on the instruction. 13. The link information is another paper document, and the operation reflecting means transmits an electronic document corresponding to the another paper document based on an operation of selecting the link information by the input pen between the documents. The pen input device according to claim 8, wherein the pen input device is developed under the management of the correspondence management unit. 14. A real object corresponding to a print object of the paper document and movable on the paper document, and each operation of duplicating, moving, and deleting the print object by the input pen is performed by the real object. In response to the position of the object, the operation reflecting means, in response to the duplication, movement, deletion instruction operation by the input pen on the paper document, the object corresponding to the print object on the corresponding electronic document 6. The pen input device according to claim 5, wherein a process of duplicating, moving, and deleting is performed. 15. The input pen has an operation function of designating a copy or move destination area when the copy or move instruction operation is performed, and the operation reflecting means transmits information relating to the copy or move to the area. 8. The pen input device according to claim 7, wherein the pen input device has a function of performing enlargement or reduction processing so as to fit in the range. 16. The pen input device according to claim 7, wherein the input pen has a writing instrument function, and is capable of writing an operation trajectory on the paper document on the paper document. 17. A pen input device for detecting an indicated position of an input pen having an operation content indicating function and performing an input process based on an operation of the input pen based on the indicated position, wherein at least a storage unit for storing an electronic document; An operation recognizing means for recognizing an operation content of the input pen on a paper document; recognizing drawing information based on a specific drawing operation among the recognized operation contents, and externally recognizing an external electronic document corresponding to the drawing information. External information capturing means for capturing and storing in the storage means; and inter-document correspondence managing means for managing the external electronic document in one-to-one correspondence with the paper document on which the specific drawing operation has been performed; Corresponds to the operation of the input pen on the paper document according to the operation content recognized by the operation recognition means Pen input apparatus characterized by comprising an operation reflecting unit for reflecting the external electronic document that. 18. The paper document is a magazine, the drawing information is a magazine name and the number of pages of the magazine, and the external information taking-in means is a content of the page of the magazine through a network or a portable storage medium. 18. The pen input device according to claim 17, wherein the pen input device is means for obtaining an external document corresponding to. 19. A pen input method for detecting an indicated position of an input pen having an operation content indicating function and performing an input process based on an operation of the input pen based on the indicated position, wherein a paper document obtained based on an electronic document is provided. Recognizing the operation content of the input pen above, and managing the electronic document corresponding to the paper document in one-to-one correspondence with the paper document in accordance with a specific operation of the recognized operation content A pen input method for reflecting the operation of the input pen on the paper document in the corresponding electronic document according to the recognized operation content. 20. Recognizing a position indicating operation of the input pen at an arbitrary position on the paper document, an area on the paper document formed from the pointing position of the input pen at the time of the recognition and the electronic document are recognized. In addition to setting the positional relationship with the area to be formed, the content of the area corresponding to the setting area of the electronic document is displayed on the display means based on the setting area, and the operation with the input pen is performed only in the setting area. 20. The pen input method according to claim 19, wherein control is performed so as to enable the processing to be reflected on the electronic document. 21. The input pen is capable of instructing at least one of selection, duplication, movement, deletion, editing, attribute change, and execution of an object on the paper document. 21. The pen input according to claim 20, wherein the process of reflecting on the electronic document is a process of performing an instruction operation corresponding to a corresponding object on the electronic document based on a recognition result of each of the instruction operations. Method.