JPH06131108A - Information input device - Google Patents

Abstract

PURPOSE:To provide an information input device as a human interface excellent in handleability by integrating pen input and speech input. CONSTITUTION:This information input device is equipped with an input pen 1 which has a microphone 1a and a function for sending a speech signal gathered by the microphone 1a to a computer body through a cord or by radio, a display and tablet 2 which detects a touch by the input pen 1, means 5, 6, and 7 which performs a speech recognizing process and a natural language process for the speech signal from the microphone 1a, and means 8, 9, 10, and 11 which performs a character recognizing process, an image recognizing process, and a position recognizing process for the input signal from the table 2. Further, this device is equipped with a means 15 which combines the input from the microphone 1a and the input from the tablet 2 together and sends data or a command to an operating system 18 or application program 19.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information input device for a user to input information such as data and commands in personal computers, word processors and the like.

[0002]

2. Description of the Related Art As an information input device of this kind, a keyboard has been used for a long time, but recently, a type of tracing characters on a tablet with a pen and inputting characters and figures has started to be used. In addition, improvements in voice recognition technology have made it possible to realize voice input.

[0003]

As described above, various types of information input modes have appeared, but conventionally, they have been used individually. For example, even in a device having a pen input function and a voice input function, the pen input and the voice input are independent, and the item input by the pen and the item input by the voice are distinguished. Also, the inputs of each other were not directly related.

On the other hand, since a microphone for inputting voice is attached to the computer main body or connected via a cord, the wiring is an obstacle and it can be used only near the computer main body. In addition, the voice input from the microphone cannot be used unless the power of the computer main body is turned on and the voice can be accepted.

The present invention has been proposed in view of the above points, and an object thereof is to provide an information input device as a human interface which is easy to use by combining a pen input and a voice input. is there.

[0006]

In order to achieve the above object, the present invention provides a microphone, an input pen having a function of transmitting a voice signal sampled by the microphone to a computer main body by wire or wirelessly, and the input pen. A tablet which also serves as a display for detecting the contact of the user, means for performing voice recognition processing and natural language processing on the voice signal from the microphone, and character recognition processing, image recognition processing and position on the input signal from the tablet. A means for performing a recognition process and a means for providing data or a command to an operating system or an application program by combining the input from the microphone and the input from the tablet are provided.

For example, when the input from the microphone and the input from the tablet are inputs related to the same object, one having a higher degree of recognition is adopted as the data or command given to the operating system or the application program. In addition, data is input from a microphone, and an edit command such as insertion / deletion is input by a gesture with an input pen.

On the other hand, the input pen may be provided with a memory for accumulating the voice signal collected by the microphone.

Furthermore, it is also possible to transmit a voice signal from the computer main body to the input pen and reproduce sound with a microphone.

[0010]

In the information input device of the present invention, when the user speaks into the input pen held in his / her hand, the voice is collected by the microphone built in the input pen and transmitted to the computer main body by wire or wirelessly. To be done. If you trace the tablet that also serves as the display with the input pen, the information is input.

Inside the computer main body, voice recognition processing and natural language processing are performed on the voice signal from the microphone of the input pen, and character recognition processing, image recognition processing and position recognition are performed on the input signal from the tablet. Processing is performed.

Then, the input from the microphone and the input from the tablet are appropriately combined and given as data or a command to the operating system or application program. For example, when the input from the microphone and the input from the tablet are related to the same target, the one with a higher degree of recognition is adopted and given as data or a command. Further, the user can input an editing command such as insertion / deletion with the input pen with respect to the data input from the microphone.

On the other hand, when the input pen is provided with a memory for accumulating the voice signal picked up by the microphone, the voice is output even when the computer main body is not in an operating state or when working away from the computer main body. It is possible to input and to send audio signals to the computer body at a time later.

Furthermore, it is also possible to transmit a voice signal from the computer main body to the input pen and reproduce sound with the microphone.

[0015]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of the information input device of the present invention.

In FIG. 1, the present embodiment has an input pen 1 which a user holds and uses, and a tablet 2 which also serves as a display for drawing characters and figures by pressing the input pen 1.
And a plurality of circuit units (including both hardware and software) that process signals from the input pen 1 and the tablet 2 and give or control signals to them.
It is composed of an operating system 18 that controls the entire computer body and an application program 19 that performs a predetermined task.

In addition to the basic function of tracing the tablet 2 on the input pen 1 and giving the locus as information, a microphone 1a for collecting voice and a voice signal collected by the microphone 1a are also provided. A memory 1b for storing
A transmission / reception unit 1c is provided for transmitting an audio signal collected by the microphone 1a or accumulated in the memory 1b to the computer main body and receiving an audio signal when the microphone 1 works as a speaker from the computer main body.
The memory 1b may be stored in either a digital or analog system. Further, the input pen 1 and the receiving unit 3 and the transmitting unit 4 can be connected not only wirelessly but also by wire.

On the other hand, a plurality of circuit units for processing signals from the input pen 1 and the tablet 2 and for giving or controlling signals to them are roughly classified into a portion for communicating with the input pen 1 (reception unit 3, 3). A transmission unit 4), a portion (speech recognition unit 5, natural language processing unit 6, character / command input unit 7) for extracting characters / commands from the input voice, and a locus of contact of the input pen 1 on the tablet 2. A portion for extracting characters, images, positions, etc. from the image (image processing unit 8, character recognition unit 9, image recognition unit 10, gesture input unit 11, position recognition unit 1
2, position input section 13, image input section 14) and input pen 1
For generating a voice signal to be given to the voice (voice generator 16)
And a part that controls the display of the tablet 2 (display control unit 1
7) and a part (total processing unit 15) for controlling the whole. The natural language processing unit 6 and the character / command input unit 7 are also used for processing handwritten characters input from the tablet 2.

Next, the operation of the above embodiment will be described.

First, it is assumed that the user is in a state of being able to interact with the operating system 18 or activating an appropriate application program 19 to carry out a desired task. The application program 19 is, for example, a word processor or the like.

The user follows the function of the operating system 18 or the application program 19,
Data or commands are input. At this time, voice input and pen input can be used as a method of inputting data or commands. When a keyboard is connected to the computer main body, it is naturally possible to input from the keyboard.

When the user speaks into the input pen 1, this voice is sampled by the microphone 1a and wirelessly transmitted to the computer body side by the transmitting / receiving unit 1c. By setting the recording mode, it is possible to temporarily store the audio signal in the memory 1b and transmit it to the computer main body side later. In this case, if the input pen 1 is provided, it is not necessary to be near the computer main body at the time of utterance, and the computer main body may not be in the operating state.

On the computer main body side, the receiving unit 3 receives a voice signal from the input pen 1, demodulates it, and passes it to the voice recognizing unit 5.

The speech recognition unit 5 performs well-known analysis processing (speech segment detection, acoustic analysis, feature extraction, segmentation, phoneme recognition, etc.) on the received speech signal, and outputs data indicating a sequence of phonemes. .

The natural language processing unit 6 analyzes the sequence of phonemes well known in the field of machine translation (word matching, syntactic analysis,
Semantic analysis, etc.) is performed and the extracted character string is output.

The character / command input unit 7 prepares the character string given from the natural language processing unit 6 as a data or command in a predetermined format and gives it to the general processing unit 15.

In the general processing section 15, the data or command given from the character / command input section 7 is given to the operating system 18, and the operating system 18 accepts the command for itself,
Those for the application program 19 are delivered to the application program 19.

On the other hand, when the user draws a character, a figure, etc. on the tablet 2 with the input pen 1, the state of contact with the tablet 2 is detected by the image processing unit 8.

The character recognition unit 9 determines whether or not the drawn object is a character, and if there is a recognizable character, the recognized character is passed to the natural language processing unit 6. Natural language processing unit 6
From then on, the operation is the same as in the case of voice input.

Further, the image recognition unit 10 determines whether or not what is drawn on the tablet 2 is a gesture (movement of a pen defined corresponding to a predetermined command such as insertion or deletion). If there is a recognizable gesture, it is notified to the gesture input unit 11.

The gesture input section 11 gives data or commands corresponding to the gesture to the general processing section 15.

In the general processing section 15, the data or command given from the gesture input section 7 is given to the operating system 18, and the operating system 18
Is a data or command for the application program 19 and is passed to the application program 19.

Further, the position recognition section 12 recognizes the contact position of the input pen on the tablet 2 and supplies the information to the position input section 13.

The position input section 13 arranges the data indicating the position in a predetermined format and sends it to the general processing section 15.

In the general processing section 15, the data given from the position input section 13 is given to the operating system 18, and the operating system 18 passes the data for the application program 19 to the application program 19.

Further, the image input unit 14 uses the locus itself drawn on the tablet 2 as image data, and the integrated processing unit 1
5, the general processing section 15 gives it to the operating system 18, and delivers the data for the application program 19 to the application program 19.

On the other hand, when the microphone 1a of the input pen 1 is operated as a speaker, the general processing section 15 gives the audio data to the audio generation section 16.

The voice generation unit 16 performs voice synthesis or the like to generate a voice signal and supplies it to the transmission unit 4. The transmission unit 4 wirelessly transmits it to the input pen 1.

On the input pen 1 side, when the transmission / reception unit 1c receives this, it is demodulated into an audio signal and given to the microphone 1a to be converted into sound.

Further, the display control unit 17 includes the general processing unit 15
Under the control of, the content displayed on the tablet 2 which is also used as the display is controlled.

Although the basic operations of voice input and pen input have been described so far, both inputs are actually performed in close relation to each other. That is, the general processing section 15 appropriately combines the input from the microphone 1a of the input pen 1 and the input from the tablet 2 and gives them to the operating system 18 and the application program 19 as data or commands.

For example, when the input from the microphone 1a and the input from the tablet 2 are related to the same target (as a specific example, when the application program 19 requests data input of a certain item, etc.), comprehensive processing is performed. The unit 15 employs one having a higher degree of recognition as the data or command given to the operating system 18 or the application program 19. In addition,
The degree of recognition is a value that can be collected during the process of recognition in the voice recognition unit 5, the natural language processing unit 6, the character recognition unit 9, the image recognition unit 10, and the like. By doing so, the accuracy of recognition can be significantly improved.

By inputting an edit command such as insertion / deletion with the input pen 1 and the tablet 2 by a gesture while inputting data from the microphone 1a, a text or the like can be smoothly input.

For example, as shown in FIG. 2, the voice "BTA" is input from the microphone 1a (), but originally "DTA" is input.
When it is desired to input "ATA", a deletion gesture of drawing "x" on the incorrect portion "B" is performed on the tablet 2 by the input pen 1 ().

As a result, "B" is erased and "TA" is deleted.
Is packed one character before ().

Then, the insertion gesture "∨" is performed before "TA" (), and when "DA" is input by voice from the microphone 1a, "DATA" is obtained ().

Note that in FIG. 2, the character at the back is clogged at the front when deleted, but the deleted character is left blank and the character at the back is left unchanged, and a new character is inserted. In this case, the letters behind may be shifted as necessary.

[0049]

As described above, the information input device of the present invention has the following effects.

Since the data input or the command input can be performed by appropriately combining the voice input and the pen input, the usability as a human interface is greatly improved.

Although there is an insufficient point in recognition accuracy regardless of whether it is a voice input or a pen input, when there is an input from both sides with respect to the same object, the one with a higher degree of recognition is adopted, Can be significantly increased.

While inputting data by voice, edit commands such as insertion / deletion can be input by the gesture of the input pen, so that smooth document creation can be performed.

Since the microphone of the input pen and the computer main body can be wirelessly connected to each other, there is no obstructive wiring and a comfortable working environment can be provided.

Since the memory for accumulating voice signals is provided in the input pen, the input pen can be carried even when the computer main body is not in operation or when working away from the computer main body. For example, you can input document data on the spot and send it to the computer at a time later, so you can remember the idea you came up with.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of an information input device of the present invention.

FIG. 2 is a diagram showing a state of voice input of a character and editing by a gesture.

[Explanation of symbols]

 1 ... input pen 1a ... microphone 1b ... memory 1c ... transmission / reception section 2 ... tablet 3 ... reception section 4 ... transmission section 5 ... speech recognition section 6 ... natural language processing section 7 ... character / command input section 8 ... Image processing unit 9 ... Character recognition unit 10 ... Image recognition unit 11 ... Gesture input unit 12 ... Position recognition unit 13 ... Position input unit 14 ... Image input unit 15 ... General processing unit 16 ... Voice generation unit 17 ... Display Control unit 18 ... Operating system 19 ... Application program

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification code Internal reference number FI technical display location G06F 3/16 B 7165-5B

Claims (5)

[Claims] 1. A microphone, an input pen having a function of transmitting a voice signal sampled by the microphone to a computer main body by wire or wirelessly, a tablet which also serves as a display for detecting contact of the input pen, and a microphone from the microphone. Means for performing voice recognition processing and natural language processing on the voice signal, character recognition processing for the input signal from the tablet,
It is characterized by further comprising means for performing image recognition processing and position recognition processing, and means for giving data or a command to an operating system or application program by combining the input from the microphone and the input from the tablet. Information input device. 2. When the input from the microphone and the input from the tablet are inputs related to the same object, one having a higher degree of recognition is adopted as the data or command given to the operating system or the application program. The information input device according to claim 1. 3. The information input device according to claim 1, wherein data is input from a microphone and an edit command such as insertion / deletion is input by a gesture with an input pen. 4. The input pen is provided with a memory for accumulating a voice signal picked up by a microphone.
The information input device described in 2 or 3. 5. The information input device according to claim 1, wherein a sound signal is transmitted from the computer body to an input pen and sound is reproduced by a microphone.