CN1358300A - OHAI technology user interface - Google Patents

Abstract

A variety of methods and system for presenting and delivering user application input choices. One-handed or vocal user input provides chords or syllables. The invention includes systems and methods for presenting user application input choices, chord or syllable based apparatus and sub-system for generating user application input choices in response to input signals associated with chords or syllables. The chord blocks have two parts: a row on the top called the chord map and a large of rectangle below called the selecting cell. One embodiment of input device resembles a joystick and includes a base and a key carrier extending from the base. A single key is provided along the top of the carrier, and four keys extend widthwise along a side surface. A system presents and delivers user application input choices. An input device receives chords and syllables. A processor associated with the input device receives chords or syllables and generates input signals, a second processor for receiving and evalulating the input signals and for sending a user application signal to the user application processor of the computer.

Description

User interface with One Hand Input (OHAI) technology

background of the invention

This patent application claims the benefit of Provisional Patent Application No. 60/039,390, filed February 27, 1997.

A workstation user environment is characterized by a room of sufficient size for a computer, a mouse or other accessories necessary for the work to be done using the PC as a tool. In a workstation user environment, most user applications that require input use a mouse and keyboard. As a result, the process of input involves two hands manipulating the two input devices. To send input signals to the application, the user types with two hands, moves one hand away from the keyboard to manipulate the mouse, and then returns the hand to the keyboard. When the hands are on the keyboard, the hand and fingers are constantly moving from one key to another when (typically) the user tries to see the screen or window of the user application and the keyboard at the same time. One Hand Input (One HAnd Input) technology - OHAI, is an option that allows the user to focus on the screen and eliminates the possibility of erroneous input due to typing the wrong key.

Overview of the invention

The present invention relates to Chord-based or syllable-based input systems, methods of representing and presenting user application system input choices, and methods for generating application system responses to user-generated acupressure (here combinations of chords or spoken syllables) A subsystem of inputs. OHAI represents a single input.

The present invention includes systems, methods and apparatus for providing user applications with input options, chord-based or syllable-based input devices and subsystems for generating application system input in response to user chords or incoming syllables.

The new system for representing and presenting user application system input selections includes memory for associating string series or syllable sequences with complex user application system input selections, an optional A display device, an input device for inputting chords or syllables, a first processor for generating input signals from chords or syllables, a first processor connected to the first processor for receiving input sequences from the first processor A processor for estimating the input signal sequence, connected to an optional display device for modifying the display in accordance with the estimated input signal sequence, and connected to the user application system for sending the user application system input selection to the user application system .

There are many potential input device embodiments. In a preferred embodiment, the input device resembles a joystick and includes a base and key trays extending from the base. The key tray includes a base attached to the base, a top surface, a pair of sides, a front surface and a back surface. Provides keys on one or more faces. In a preferred embodiment, a set of four keys are arranged along one side or front surface as viewed. A key is provided on the top surface. The keys are placed such that when a user grasps the device, the user's thumb rests on the key on the top surface, while the user's other fingers rest on a set of four keys. Each key can be pressed with a single finger or thumb without removing or repositioning the fingers.

In another preferred embodiment, the input device is integral with a laptop computer running the user application system. Provide at least one set of keys to press on the computer case. In a related preferred embodiment, two sets of five ergonomically placed keys are provided, one set for the keyboard portion of the chassis and the other set for the monitor portion of the chassis.

When the user application system resides on the wearable computer, one embodiment of the input device is a glove including sensors integrated into the fingertip of the glove. Another embodiment of the input device is a speech input processor sensitive enough to distinguish a small set of speech syllables. The preferred embodiment of the display device on the wearable computer is a grid showing the relationship to the chord or syllable, which has a display on the head with a small monitor.

The first processor monitors user-generated strings or syllables and sends corresponding input signals to the second processor. The first processor may include a programmed chip, and may be housed in the input device.

The current method of representing and presenting user application system input selections includes the step of associating complex user application system input selections with a sequence of chords or syllables in memory, optionally displaying chord syllables in the sequence of chords or syllables with complex user application system input selections The step of the system inputting a selected relation, the step of receiving a user-generated chord or syllable using an input device, the step of generating an input signal from the input chord or syllable, the step of estimating a sequence of the input signal, modifying an optionally displayed to prompt in connection with complex The step of user application input selecting the associated chord or syllable sequence of chords or syllables, and the step of sending the complex user application input selection to the user application.

In the preferred embodiment, the display is a grid arrangement of blocks, including a grid name block and a plurality of chord blocks. Each chord block includes a row of finger grids and a large selection grid placed adjacent to the row of finger grids. The finger grid of each piece is coded so that the user can identify which combinations of switches can be depressed and released to select the desired string note. The tentative sub-input selection can be made to only press a set of switches without releasing them. If the selected selection grid includes the name of a different grid, this subsequent grid will replace the current grid.

Displays can be displayed on a computer monitor.

The present invention can be used to input Chinese characters or other foreign language based characters into computer user application systems. For inputting Chinese characters, character selection methods include pinyin selection and Chinese character selection. Pinyin selection involves starting by choosing an appropriate pinyin initial, selecting an appropriate vowel string, and selecting an appropriate ending/tone combination. Once Pinyin is selected, Chinese characters are displayed and selected.

A similar approach can be used for other speech

The present invention can be used for word processing applications, for World Wide Web (WWW), and virtual reality navigation applications, for structured application development and operating system interface applications, for menu navigation in applications using menus.

These objects, as well as further and other objects of the present invention, will become apparent from the patent disclosure, including the above and below written description and claims and drawings.

A brief description of the drawings

Figures 1-3 show a side view of a workstation OHAI tool (WHO).

Figure 4 shows a WHO being grasped by hand.

Figure 5 shows an OHAI grid for English text entry.

Figure 6 breaks down the chord blocks of an OHAI grid and shows the relationship between the finger grid and the workstation OHAI handpiece.

Figure 7 is a workstation and a glove showing an OHAI window.

Fig. 8 shows a window of a WWW application system.

Figure 9 shows a WWW application system with an OHAI chord map overlay.

Fig. 10 shows an OHAI grid for OTIS (OHAI Language Text Input System).

Figure 11 shows the OTIS grid mandate.

Figure 12 shows the OTIS follow-up grid.

Figure 13 shows OTIS finite state automation.

Fig. 14 shows English text input using OTIS.

Figure 15 shows the OHAI system information flow.

Figure 16 is a chart of Mandarin syllables.

Fig. 17 is a pinyin counting table.

Figure 18 shows 42 Chinese characters with the same pinyin "Shi".

Figure 19 shows the first Twin Bridge software prompt window for "Yes".

Figure 20 shows the Twin Bridge software prompt window for "Yes".

Figure 21 shows CHIME (Chinese Character Input Method) combining Pinyin initials and vowels.

Fig. 22 is an overview of the Chinese character input method.

Fig. 23 is a chart of pinyin chord sequence of the Chinese character input method.

Figure 24 shows the input of the strings for producing the four Chinese characters of "Yes".

Fig. 25 has shown the five Chinese character input method chords of "贳".

Figure 26 shows the input method strings for the two Chinese characters of "give".

Figure 27 shows an example of two stringed Chinese characters.

Figure 28 has shown the three Chinese character input method chords of " white ".

Figure 29 shows an example of three stringed Chinese characters.

Figure 30 shows the grid for "AnyTone" for "Shi".

Figure 31 shows the integration of Chinese character input method and OTIS.

Figure 32 shows a diagram of an OTIS data file.

Fig. 33 is a flowchart in an OHAI diagram.

Fig. 34 is a flowchart of an OHAI grid set (Gridset) application system.

Figure 35 is a front view of a laptop with OHAI function.

FIG. 36 is a rear view of the OHAI laptop of FIG. 35 .

Figure 37 shows an OHAI glove for the left hand.

Figure 38 illustrates a headgear for use with wearable computing.

Figure 39 shows the handle of a joystick type OHAI.

Figure 40 shows strings and corresponding text for non-display applications.

Figure 41 is a sketch of a secure silent communication OHAI application system.

Detailed Description of Best Crane WHO: Workstation One-Handed Input Tool

OHAI means single input. As shown in Figure 1, the workstation OHAI tool 1 (WHO) either replaces the keyboard or co-exists with it in the workstation user environment. Figure 1 shows a preferred embodiment in which the hand tool 1 resembles a joystick and includes a base 2 and a key bracket extending from the base. The key tray comprises a top surface with a single key 4 and a set of four keys 5, 6, 7, 8 extending widthwise along the sides. The keys are placed such that when the user grasps the device as in Figure 4, the user's thumb 10 rests on the top key 4 and the user's other fingers rest on the four-key set 5,6,7,8. Each key can be pressed with the thumb or other finger without having to move or move the finger again.

This one-handed device performs the same function as a keyboard. The WHO in Figure 1 is for the left hand. To use it, the user grips it so that the little finger is on the lowest key, each sequentially upward finger is on the key above the previous one, and the thumb is on top. To send input to the user application, he presses and releases a combination of keys. Typically, the user puts one hand on the WHO and the other on the mouse during the entire man-machine conversation. While it is recommended to use the WHO as the only text input device and forego the keyboard, it is possible to use all three devices simultaneously. Some examples of hope are the early days of moving from the keyboard to the WHO, or the simultaneous use of two distinct input devices. For example, WHO can be set to send Chinese input while the keyboard is used for English input.

With WHO, the user never has to take his hands off the device, his fingers off the keys, or his eyes off the screen. The surface that WHO guides the fingers to their proper position to press on extends the entire length of the surface of the device, so no time and effort is wasted repositioning the fingers. Since there are only five keys, the finger does not have to move from one key to another, so there is no need to take the user's eyes away from the application software system. The WHO device alleviates many of the factors that cause repetitive movement disorders such as carpal tunnel syndrome. Its size can be easily and lightly adapted to each individual user.

Plus, a one-handed device frees up the other hand for other uses: mouse manipulation, phone calls, or holding your breakfast coffee cup.

The WHO is ideally suited for a workstation user environment, and an enhancement to existing keyboards as an alternative. As an added bonus, it takes up very little space (smaller than a mouse, and considerably smaller than a mouse pad). The shape of the WHO makes it easy to keep out of the way when not in use. The device is light (about the same as a mouse), so it can be picked up and used with the front wall resting on the work surface. In both cases, it can be positioned and repositioned to relieve stress causing repetitive motion disorders. In addition to being an enhancement of the keyboard, its replacement frees up more surface area and provides more freedom of movement for the user.

The advantages of WHO in the workstation user environment are simplicity, comfort and freedom of repositioning and rotation in three dimensions. In short, the user can be repositioned in any suitable manner. In addition, it requires little physical effort to use and does not require gaze correction for the finger's position on the device. Other forms of devices are equally well suited to other computing environments, as will be disclosed below. how to use the tool

As shown in Figure 4, the user holds the WHO with his thumb on the top key and his fingers on the four remaining keys. To select an input, the user presses and releases a combination of keys. Each combination is called a Chord and the action of depressing and releasing the key is called Chording. There are only 31 chords, and they can be mastered in only 1-3 hours with an OHAI training game (compared to more than 100 hours for partial mastery of the QWERTY keyboard). OHAI Marks and Terminology

Each chord is represented by a series of circles on the markings, with blackened numbered circles representing depressed fingers and empty, unnumbered circles representing undepressed fingers. Since the example grip is left-handed, leaving the right hand free for precise mouse work, the fingers are numbered as follows:

①＝Depress the little finger

②＝Depress the ring finger

③＝Depress the middle finger

④＝Depress the index finger

⑤＝Depress the thumb

The chord composed of all fingers (full chord) is expressed as ①②③④⑤, the chord composed of only the thumb is ○○○○⑤, and the chord composed of the little finger, index finger and thumb is ①○○④⑤. Other chords are represented in a similar manner.

Each chord either represents an input selection, or represents one of a sequence of subinputs (Subinpnt) which together specify an input (such as pressing the SHIFT or CAPLOCK key on the keyboard before pressing the letter to be capitalized). An OHAI window (as shown in FIG. 5 ) coexists with the user application system window on the workstation screen (see FIG. 7 ) to give the user an instruction to pluck the strings.

To show how the user uses WHO, the OHAI window displays a certain number of grids, which relate strings to user application system inputs (as shown in Figure 5). Figure 6 breaks down and labels an OHAI grid for alphanumeric input.

As shown in FIG. 5 , the grid 20 is a 4×8 array of 32 blocks 21 . The upper left block 22 is the name black and gives the name of the grid which provides an overview of the selection represented by that grid. The remaining 31 chord blocks 21 are divided into two parts; a rectangular row 23 called the chord diagram on the top, including finger grids 31, 32, 33, 34, 35 and a large rectangle 24 called the selection grid below.

In each chord block 21, the finger grids 31, 32, 33, 34, 35 are black or empty. A black finger grid indicates a finger press and release. An empty finger grid indicates an unpressed finger. For example, a chord diagram consisting of all five black finger grids (——) indicates the full chord ①②③④⑤), a chord diagram with the 1st, 4th and 5th finger grids turned black (——) indicates the chord ①○○ ④⑤, which specifies the lowercase character 'r' in the grid 21 shown in FIG. 5 . Other chords are similarly represented by the chord diagram.

The range of immediately available input and sub-input selections is given by the grid currently displayed in the OHAI window. For example, Figure 7 illustrates an a-z grid 26 displayed on a computer monitor screen. The selection box indicates to the user what will happen when the string of the chord diagram above it is plucked (pressed and released). Tentative (pressed but not released) input selections are indicated by highlighting the box corresponding to the unreleased string. OHAI application type

One of the many disadvantages of previous implementations of chord-based input systems is that they only allow English text input. Instead, the OHAI application system associates string sequences with any user-input group or multiples thereof. This is especially useful in tasks where the keyboard is not suitable for input, such as Chinese and Japanese word processing, World Wide Web (WWW) and Virtual Reality (VR) navigation. hx qh, OHAI brings the possibility of a unified user interface, which provides a single A mechanism to combine the functions of toolbars, drop-down menus, accelerator keys and dialog boxes. These concepts fall into three intuitive categories for user applications. Word processing for any speech is an example of an input-intensive user application; WWW navigation is an example of a content/selection user application; and structural development and operating system interfaces are examples of a selection-intensive user application. Formal definitions of these three categories are beyond the scope of this disclosure; in general a word processor differs from a web browser in the choices made by the user using them, and from a software development kit. Each of these user application categories has a corresponding type of OHAI application.

OHAI applications fall into three general categories: OHAI grid group applications, OHAI chord diagram arrangements, and OHAI-based interfaces. The OHAI grid set is intended for input-intensive user applications such as word processing and spreadsheets. With the OHAI grid application, the workstation screen has both the OHAI window and the user application window (Figure 7). The user application window is opened by the user application system. The OHAI window is provided by the OHAI processor software and displays a certain number of grids including the OHAI grid group application. The OHAI chord diagram arrangement is used for selective intensity applications. Such as WWW and VR navigation, where the user application system provides options such as WWW links or VR gestures. Using the OHAI chord graph arrangement, the user application system window (FIG. 8) is overlaid with the chord graph (FIG. 9), which provides the chord rotation of the selections that appear. The OHAI-based interface is a complete user application system, and the way it interacts with the user is the OHAI grid group and chord chart instead of various windows-icons-menus-popus-wimps (Windows-Icons-Menus-Popus-WIMPS) The user interface unit in OHAI; an example is ODS, which enables users to build and modify OHAI application systems. Examples of OHAI application systems and their types are:

OTIS (OHAI grid group application, OHAI text input system)

CHIME (OHAI grid group application, Chinese character input method)

HAJIME (OHAI grid application, hand-based Japanese input method)

OWN (OHAI Chord Diagram Arrangement, OHAI Web Navigation)

OVR (OHAI Chord Arrangement, OHAI Virtual Reality)

OMI (OHAI Grid Group Application, OHAI Mouse Interface)

ODS (OHAI-based interface, OHAI development system)

In OHAI grid group applications such as OTIS and CHIME, the OHAI window displays the grid, and its form remains unchanged; there is always a name block and 31 chord blocks, each showing a chord diagram and a selection grid. Only the content of the selection box changes. For OTIS, the selection box includes English text input and sub-input; for Chinese character input method, it includes Chinese character text input and sub-input. Sub-inputs are indicators that show subsequent grids for a subset of sequential inputs. If the selection box includes an input, such as a text character, the input is sent to the user application system when the chord in its corresponding chord diagram is plucked. If the grid includes the name of a different grid, the subsequent grid replaces the current (predecessor) grid in the OHAI window, giving the user the choice of yet another set of grids. Certain selection grids cause both input to be sent to the application and subsequent grids to appear.

These concepts are illustrated with OTIS - an example text input system. OTIS: An Introduction to OTIS, an Exemplary OHAI Application System

Almost every user application receives and processes English text input. This means that an input system must be able to send lowercase letters, uppercase letters, Arabic numerals, label symbols and common symbols to the application system. OTIS is an exemplary (modifiable) OHAI grid set application system that fulfills the purpose of these inputs. This selection presents and describes the various parts of OTIS, shows how to use OTIS, and then points out some design considerations and advantages, as well as some additional layouts. Description of OTIS

The OTIS application system consists of grids (Fig. 10), each of which fulfills a specific task. Figure 11 defines the primary and secondary input tasks associated with each grid. The first or basic grille to OTIS is the LOWER grille. The LOWER grid shows the user which chords to enter lowercase, space and backspace, and which chords to select their own grid. For example, ○○○○⑤ string tone input a space, ○②③④○ input lowercase ‘m’, ①②③④⑤ input backspace, ①②③○○ select UPPER grid for uppercase. Lowercase, space, and backspace chords have no subsequent grid; keep the LOWER grid in the OHAI window when plucking these chords. For UPPER, NUMBER and PUNCT chords, all have subsequent grids named by their selections. These allow the user to select from an input group of uppercase, numeric symbol characters. The ability to quickly select capital letters justifies the use of subsequent screens. If the user needs a capital "O", he will toggle ①③③○○ to cause the UPPER grid, then toggle ①○○○○ to 'O', after which its subsequent grid (LOWER) replaces the UPPER grid. OTIS is similar for the input of single numbers (grids of NUMBER chord blocks) and symbols (PUNCT chords and grids).

Figure 12 explicitly shows the relationship between selection grids and their subsequent grids. The arrows in Figure 12 show the subsequent grid for each chord. Because the arrows from each chord block can be confusing, the diagram uses arrows from the edge of the window to indicate subsequent grids for all chords not explicitly shown. For example, an arrow from a LOWERT grill shows that most chords have the LOWER grill itself as a subsequent grill, while an arrow from an UPPER grill shows that all chords have a different subsequent grill; LOWER in most cases. These are examples of selections that both cause input and subsequent grids. LOWER and UPPER selection grids are themselves examples of selections that have subsequent grids (named after them) but do not result in input.

Because Figure 12 may be confusing, Figure 13 is another representation of the same information. Each circle corresponds to an OTIS grid, and the marks on each arc indicate the inputs that OTIS sends to the user application when toggled, for which inputs each arc goes to subsequent grids. Use OTIS

Figure 14 shows how to dial the phrase "the OHAI handset". Column 1 shows the OHAI window, column 2 shows the chords, column 3 shows the selection of chords, column 4 shows the contents of the user's application system window after the toggle sequence, and column 5 shows the keys corresponding to the keyboard applied to those chords. key. In order to send text to the user application system, the strings ①○③○⑤, ○②○○⑤, ○○③④○○ and ○○○○⑤ input characters 't', 'h', 'e' and a space. Pizzicato ①②③○○ twice to select the UPPER grille, followed by the CAPLOCK grille. For the "OHAI" pizzicato ①○○○○, ○②○○⑤, ○②○④○. The 3rd pizzicato ①②③○○, cancel CAPLOCK and return to the basic grid. Pizzicato ○○○○⑤ Enter a space after "OHAI". Finally, for the "handset" plucked strings ○②○○⑤, ○○○④○, ○②③④⑤, ○○③○⑤, ①○③○○, ○○③④○, ①○③○⑤. OTIS Design Considerations and Benefits

Some of the outcomes considered in designing the OHTI grid group application system were "sequence minimization, intuitive chord input linkage, input localization capabilities, chord mapping with respect to function, and chord difficulty.

Sequence minimization means that each input in a grid group should use the fewest number of consecutive chords obtainable from the base grid (and as many other formats as possible).

Figure 12 shows that OTIS minimizes the size of the sequence by placing the names of the other grids in the lower left three selection boxes. This makes all grills reach all other grills with at least two chords, so at most two chords from the base grill should be able to derive all inputs. In addition, by placing the most frequently used input (lower case letters) in the basic grid of OTIS by hand, the most frequently used input can be derived with a single chord. An example of an intuitive chord input connection is the connection of "space" to the chord ○○○○⑤, which is produced using only the thumb of the left hand; not only is this an easy chord to produce, but this also corresponds to the "Space" input.

By displaying LOWER, UPPER and CAPLOCK inputs in alphabetical order, NUMBER and NUMLOCK inputs in numerical order, and attempting to put the most frequently used symbols first into the NUMBER, MUNLOCK and PUNCT grids, OTIS optimizes the positioning capabilities of the inputs.

The chord mapping for the function is exemplified not only by the uppercase letters produced by the same chords as the lowercase letters, but also by the same chords used to switch to UPPER and then back to LOWER if CAPLOCK is required. To see this, note that ①②③○○ dialing once allows a single uppercase letter to be entered via the UPPER grill using the same chord as a lowercase letter. Picking twice allows a string of capital letters to be entered via the CAPLOCK grill using the same chord. Dial 3 times to return to the lowercase input of the LOWER grid. Strings ①②③○⑤ have a similar situation to numbers and numeral symbols.

One of the benefits of a well-designed OHAI grille group application system is that the user's dependence on the OHAI window fades after a short period of time. When the chord is memorized, the user can completely ignore the OHAI window and concentrate on the user's application system. At this point, the OHAI can be removed from the screen or retained for reference purposes. However, OHAI can be trained on its own due to the continuous visual feedback and the small number of chords. Due to the large number of Chinese and Japanese characters, self-training in a well-designed OHAI grid group application system is more necessary in CHIME and HAJIME. OTIS User Cropping

An input system that generates only alphanumeric input and forces the user to a single chord-text association is of limited use, so methods must be provided for adding and rearranging input. Two indications of tailoring, scalability, and rearrangement are briefly discussed here.

Extensibility means providing additional inputs; one way to extend OTIS is to assign a user-defined subsequent grid to the blank selection grid for ①②③④○ chords in the PUNCT grid. For example, if the user uses Chinese or Japanese characters in his task, he can also assign the basic grid of CHIME and HAJIME to this block, so that the Chinese or Japanese input is two pizzicatos away from the basic grid of OTIS.

Rearranging means reassigning the input to the chord, and reflects the reassignment in the OHAI grid group applied grid. The arrangement in this application is designed to minimize input fix time and maximize memorability for a "typical" user. A more specific user—like a novelist—may decide to place the most-used characters on the basic grid, and the lesser-used characters on subsequent grids. Thus, 'q' 'x' and 'z' can be interchanged with periods (.), commas (,), and double quotes ("). Another obvious optimization involves research to determine which chords are most useful to users (whether "typical " or special users) are most difficult. Once a judgment is made, the most frequently used characters can be associated with the easiest chords, thus sacrificing short-term localizability for long-term ease of use, an obvious payoff for users prone to repetitive motion disorders Application. Practical use of OTIS

OTIS is an exemplary OHAI application to illustrate the OHAI concept, not a reason to use OHAI instead of a keyboard for English text input in a workstation user environment. Most people who do English text input alone are used to the keyboard. The advantages of OHAI over keyboards for English text input occur in other user contexts; for example, keyboards add weight and moving parts to finger computers, so for input of English text into one such device, a 5-press human The OHAI device composed of keys placed on the organic case (as shown in Figures 35 and 36) is just about applicability. It has advantages over the pocket keyboard in terms of weight and price. The use of OHAI for English text input in a workstation user environment is only desirable when active keyboards are not used for reasons including space savings, portability, physiological and workflow benefits, etc. Usual OHAI entry system

In the OTIS example, the WHO device and the OTIS application system work together in a workstation user environment, allowing the user to input English text with one hand. It should be remembered that WHO is a dedicated OHAI device and OTIS is a dedicated OHAI application system. The two variable parts of this program are OHAI equipment and OHAI application system. The equipment needs to change to suit the user environment, and the OHAI application system changes in form and content with the nature of the required input, WOH is suitable for the desktop user environment, and OTIS is suitable for English text input. Other OHAI devices and OHAI applications are suitable for other user environments and purposes.

OHAI devices and application systems are part of the OHAI system, as shown in Figure 15, which shows the path of information flow in the OHAI system. In FIG. 15, the path from "OHAI Application Grid" to "User's Eye" represents the OHAI application system used by the OHAI software to inform the user how to generate inputs and sub-inputs via windows or chord diagrams. At the same time, the window of the user application system provides the user with options (such as a document that needs to be input). The path from "user's hand" to "OHAI software" represents the user manipulating the device to select an input or sub-input. Using OHAI to apply grid information OHAI software interprets the signal sent from the device and updates the contents of the window, ensuring that the OHAI window keeps the user informed of his tentative input selections (by highlighting) and his current input selections (by switching grids) or chord diagram overlay). When an input is fully plucked, it is sent to the user's application, and the process repeats.

The presence of windows and the nature of the handpiece removes two factors against the alternative application of chord-based input systems. Instead of forcing the user to memorize the relationship between strings and input, the OHAI window tells how to complete the required input. With only 5 keys, OHAI's hand tool eliminates the finger movement involved in keyboards such as QWERTY, Twidder and BAT. Description of the OHAI application system CHIME: OHAI Chinese character input method

The most immediately obvious benefit provided by OHAI is with input groups not suitable for OWERTY keyboards, such as Chinese character text input. More than a quarter of the world's population speaks Chinese. More and more of them use computers in their daily life. It is hoped that these users can input Chinese characters into the user application system. Chinese character input problem

According to authoritative sources, the number of Chinese characters (Chinese characters) ranges from a few thousand to tens of thousands. It is generally believed that the number in modern applications does not exceed ten thousand. Even with only 10,000 Chinese characters, it is a technical challenge to minimize the user effort to select them for input into the user application system.

In addition to the sheer number of characters, each character is internally complex, consisting of anywhere from 1 to 29 strokes.

Another factor in the complexity of designing an easy-to-use input method for ten thousand complex Chinese characters is the lack of a single organizing principle. Each Chinese character has a number of graphical attributes for classification and organization into dictionaries; these organizational and classification attributes consist of strokes, constituent parts, sequence of strokes, number of strokes and "stem-radicals", which are cognitively primitive A finite set of components. Although some very clever input methods have been devised using the graphical properties of Chinese characters, these methods force the user to remember a large amount of information, take the same amount of time as handwritten Chinese characters in order to input Chinese characters, or involve specific expensive hardware, and thus cannot win public use .

The problem is to design a system for selecting characters that takes advantage of existing knowledge about characters, minimizes the additional learning required, and takes less time and effort once learned than the actual writing process. If additional hardware is required, it should be cheap and useful for another purpose. Chinese character input solution strategy

The input method, which requires only minimal effort on the part of the user, benefits from the fact that although ten thousand Chinese characters have a sequence number, the pronunciation numbers are the smaller numbers of the total (only 1,279 pronunciations in Mandarin). These input methods allow the user to narrow down the selection of Chinese characters by first selecting a pronunciation and then selecting from the Chinese characters with that pronunciation. The most popular approaches apply phonetic specification states in using Pinyin phonetic representation systems, so before explaining how these systems work, an overview of Pinyin is appropriate. Summary of Romanization of Pinyin

Pinyin, the most widely spread method of expressing pronunciation, is taught in elementary schools in all countries where Chinese characters are taught as part of the standard curriculum. It uses the Roman alphabet and pronunciation tables or numbers to represent the pronunciation of Mandarin Chinese.

Pinyin representations optionally begin with one of 21 consonant sounds:

b, p, m, f, d, t, n, l, c, s, ch, sh, r, j, q, x, g, k, h

Selections without an initial consonant sound in CHIME are themselves treated as initial consonants and denoted "φ". Regardless of how the character begins, all pinyin have one of 19 vowel combinations:

a, o, e, -i, er, ai, ei, ao, ou, I, ia, ie, iou, u, ua, uo, uai, uei, üe

Although using the same character, '-i' and 'i' are pronounced differently; '-i' is similar to the vowel sound in English "is", and 'i' is similar to the vowel sound in English "she". If there is no selected consonant before the vowel string, multi-letter vowel strings beginning with 'i' or 'u' are replaced in written pinyin by 'y' and 'w' respectively. Thus there are "ya-", "yao", "you", "yo-", "ye" and "wa-", "wo", "wai", "we-".

In addition to the optional initial consonant sound and the mandatory string of vowels, there are two optional endings:

-n, -ng

Selections without endings in CHIME are treated as endings themselves and denoted as 'φ'.

Each pinyin has one of four tones: the first tone is high and flat, the second tone is short and rising, and the third tone is long and falling followed by a slight rise. The fourth tone is short and falling. A lack of intonation, sometimes called a neutral intonation, is short and neither rising nor falling. Each non-neutral tone is indicated by a number following the letter, or by a diacritical mark on the main vowel of the vowel string:

ma1, ma2, ma3, ma4, ma

or

ma, ma, ma, ma, ma

Neutral tones are sometimes used with O or φ following a syllable or above a main vowel. express. The absence of a diacritic or a number after it is also sometimes used to indicate that the tone is unknown and irrelevant.

It is useful to distinguish between syllables and pinyin; for the purposes of this section, pinyin means full pronunciation including tones, and syllables mean pronunciation without tones. For example, "ma" is a syllable, "mai" and "ma" are pinyin, and "Mo" is a Chinese character.

Figure 16 shows all combinations of initials, vowel strings and endings that contain Mandarin syllables. Note that not all possible combinations of initials, vowel strings, and endings actually exist.

Figure 17 shows that there are multiple pinyin derived from the syllables of Figure 16 present in the dictionary we used as reference. Note that not all possible pinyins actually exist. Use Pinyin to select a range of Chinese characters

To illustrate Pinyin-based input, consider the input of a single Chinese character "是" {"Shi4" "is"}. As shown in FIG. 18, "Yes" 30 is one of 42 Chinese characters with the same pinyin.

To enter "yes", the user begins by selecting "s-h-I-4". The software that implements this input method then uses a visual method to introduce Chinese characters and allow the user to choose from them.

The means of "selecting" the pinyin and the visual means of eliciting Chinese characters once the pinyin is specified vary widely in the implementation of Chinese character input methods. Example keyboard implementation

One of the most popular pinyin-based input methods is TwinBridge Software Corporation's Chinesepartner(R) software, which translates keyboard strokes into Chinese characters. While Chinese Partner is a popular and very useful application, the nature of the keyboard and its input tasks make touch typing impossible. To see this, consider a task that takes "yes" as input.

The user enters "s-h-i4" for pinyin by using the corresponding keys on the keyboard, and then watches the TwinBridge prompt window (FIG. 19) to see if it is one of the 10 displayed Chinese characters. If the Chinese character is displayed, press 10 function keys to stand one to show that the Chinese character is sent to the user application system. If the Chinese character is not displayed in the first prompt screen, the arrow keys must be used in order to display another group of 10 Chinese characters with the same pronunciation in the indication window (FIG. 20). When the character appears, use one of the 10 function keys to select it.

This process is similar for all Chinese characters. Using Chinese Partner, the user must press the key for a single Chinese character at least 5 times, with an average of 7 times. If the Chinese character is mistaken for the first time, the maximum is 12 times. If he doesn't see the Chinese character in the prompt window when he uses the arrow keys, he has to use the arrow keys to locate the Chinese character through the previous screen in reverse, and then increase the number of keystrokes. Due to the frequent need to switch the visual focus and attention between the keyboard, the prompt screen and the user application system, it is common to not see Chinese characters when passing the prompt screen for the first time.

Chinese Partner is one of the most effective Chinese character input systems using keyboards, and its clumsiness is simply due to the nature of the task it was set to try to use for Chinese character input with a device that was originally intended to slow down English text input. Realization of OHAI

CHIME-OHAI Chinese Character Input Method - Improvements to the basic two-stage strategy used by Chinese Partner: the user first specifies a pinyin, and then selects among Chinese characters that match that pinyin. CHIME achieves the same goal by modifying the Chinese Partner's strategy and taking advantage of the OHAI features, thereby reducing the user's input workload. This section explains how to use CHIME, then presents some design considerations and advantages as well as other uses of the same interface. Overview of using CHIME to input CHIME in Chinese characters

Similar to OTIS, CHIME consists of formats, which are swapped in and out at the OHAI window. In this way, the grid guides the user to select the Chinese characters to be input into the user's application system. All Chinese characters can be input from the 2nd to the 5th string tone.

Similar to Chinese Partner, the process of selecting a Chinese character occurs in two stages: the Pinyin selection stage and the Chinese character selection stage. There are 3 steps or junctures in the phonetic selection phase, each distinguished by the display of the CHIME grid against that juncture. They are: initial (or fundamental) joins, syllable joins and pinyin joins. The junction points of the CHIME Pinyin stages are shown in Figure 22 and Figure 23. The Chinese character selection stage is composed of one joint point in most cases, and more than two joint points are needed only when the user does not know a Chinese character tone; the necessary Chinese character joint points are distinguished by numbers.

At the initial selection junction, the OHAI grid displays all pinyin initials including 'φ/w/y' (allowing the user to indicate those that have no initial consonants for the desired Chinese character). Note that some selection grids include more than one initial Letters, and some include two-letter initials. In order to input "is", the user starts from the pizzicato ○②③④⑤, which is connected with the pinyin initial 'sh-'. After completing the initial junction, the user enters the syllable selection junction.

At syllable-choice junctions, each choice is associated with a vowel string. The selection box displays all existing Mandarin syllables beginning with the initial letter and associated vowel string selected above. The syllable "shi" is shown in the selection box for the strings ○○③○○ associated with the '-i' and 'er' vowel strings. To illustrate what "display existing syllables" means, note that there is no "Shueng" in Mandarin for the vowel string '-ue-', but there are syllables pronounced as "Shui" and "Shun", so these are displayed in conjunction with the vowels String '-ue-' relevant chords ①○○④○ in the selection cell. Note that there is no Mandarin syllable 'Shion' or 'Shiong', so the options for strings ○②③○⑤ related to the semi-vowel '-iou-' are empty. At the syllable selection junction the user can select up to 3 different syllables, depending on the ending following the initial and vowel string. These are lumped into one at the pinyin selection junction.

At the pinyin selection junction, each selection case is associated with a combination of endings and tones. The selection box displays all existing pinyin according to the previously selected syllable group. At this junction the columns of the grid are associated with endings and the rows are associated with tones. To enhance locateability, use the second, third, and fourth columns (so that the user visually scans from left to right after the name block) instead of the first, second, and third columns. The first ends with 'φ' (indicating no end) the latter follows with '-n' and '-ng'. The first to fourth tones are obtained in turn, followed by "any tone selection" (the case where the tone is unknown), and then the neutral tone (it is rarely the only tone of a Chinese character). The pinyin "shi" is displayed in the second column (corresponding to the 'φ' ending) and the fourth row (corresponding to the fourth tone). The chords for such an ending and tone combination are ○②③○⑤. Once Pinyin is selected, its Chinese characters are displayed at the Chinese character junctions.

Although a Chinese character does not have a tone in the input text, it generally has an additional and original tone pronunciation, which is often used for positioning in CHIME. For example the Chinese character "我想-Isuppose" used at the end of a sentence is pronounced with a neutral or first tone, so can be located in a grid selected with first, arbitrary and neutral tones.

At character selection junctions, each selection cell is associated with a single Chinese character having a previously selected pinyin. Show as much as possible on the OHAI window. If there are more than 30 Chinese characters, then 29 are displayed, and the chord ①②③④○ causes the OHAI window to display the remaining Chinese characters (ie the second Chinese character junction). When the user picks one of them, OHAI sends the Chinese character to the current user's application system window and returns to the initial screen to wait for another Chinese character to be selected.

Using up to 3 chords, all pinyins are selected: the first chord selects the initial letter, the second chord selects the vowel string, and the third chord selects the sound and ending. Always select the chord at each joint and do the same selection for that part of the pinyin: for example, at the first joint chord point ○②○○⑤ always select the initial letter '1-', and at the second joint select element the sound string '-i-' (if such a vowel string exists for the previously selected initial letter), and at the third junction choose an unending third tone (if such an ending exists for the previously selected initial letter and vowel string harmony tone). Only those Pinyin syllables that exist and connect initials, vowels, endings, and tones are displayed when the third chord is selected.

Figure 23 has a table showing which string tone selects which pinyin part at each of the 3 joint points during the pinyin selection process. Once Pinyin is selected, the corresponding flow characters appear in the order of the number of strokes used to write that character on Japanese paper (much like the organizing principle used by anyone familiar with Chinese characters). This is due to the tendency that the most commonly used Chinese characters are the ones with the fewest strokes and the easiest to recognize, thus enhancing the localization capability.

Because the minimal pinyin has more than 30 Chinese characters, about 99% of Chinese characters select 4 or less strings, 95% of Chinese characters only 4 strings can be selected, 4% need less than 4 strings, and 1% need 5 strings. Since no pinyin has more than 59 Chinese characters, no more than 5th string is required. The following are examples of 4th, 5th, 2nd and 3rd strings that can be selected as Chinese characters. A typical Chinese character is the 4th string

The 4th string sound is enough to send the Chinese characters exceeding 95% of all Chinese characters to the user application system. Fig. 24 shows a 4th string sequence that requires input of "yes" ("shi"). Starting from the basic grid, the user picks the string ○②③④⑤ to select the initial letter ‘sh-’. The current letter is selected, and the second grid showing all syllables starting with the initial letter 'sh-' allows the user to select the syllable 'shi' with the chord ○○③○○. The final 3rd grid allows the user to select the complete pinyin with strings ○②③○⑤. The fourth grid shows 42 first 29 Chinese characters pronounced as shi; the user then picks the string ①○②③④⑤ to select “Yes”, which is sent to the user’s application system and makes the OHAI window display the basic grid again for selection a Chinese character.

String sequence ○①②③④⑤, ○○③○○, ○②③○⑤, ①○③④⑤ send Chinese character "is" (" shi") to all the strings needed by the user application system as word processing or spreadsheet. Note that 4 strings are less than the number of strokes (9) that Chinese characters are written on paper. Examples of 5 string-tone Chinese characters

In the case of more than 30 Chinese characters for a pinyin, the first 29 are displayed at the fourth junction and those 30th and above are displayed at the fifth junction. The Chinese character "贳" (shi4, "oath") marked at 31 in Figure 18 is an example. The ellipsis in Figure 25 indicates that there are more than 30 Chinese characters pronounced 'shi'. In order to enter these, the user first uses the same first 3 chords as "是" (○②③④⑤, ○○③○○, ○②③○⑤), and then enters the 5th chord diagram at the first Chinese character junction ①②③④○ . Finally, the pizzicato ○②○④⑤ gets “贳”. Here 5 strings (○②③④⑤, ○○③○○, ○②③○⑤, ①②③④○, ○②○④⑤) are less than the 14 strokes required for handwriting this Chinese character. Other pinyins for Chinese characters with 5 pizzicatos are

Examples of yi4, yu4, bi4, li4, zhi4, ji1, ji4, xil secondary chord Chinese characters

If there is only one character for a particular initial and vowel string, that character is displayed at the syllable selection junction, allowing the user to select it with only 2 strings. Fig. 26 shows the CHIME grid and chord sequence for inputting the Chinese character "起" (gei3 "起") with two chords. Starting from the basic grid of CHIME, the user plucked strings (①○○○⑤ select the initial letter 'g-'. Because gei3 is the only pinyin with the initial letter 'g-' and the vowel string '-ei', and "give " is the unique Chinese character of this pinyin, and " give " is displayed in the grid piece relevant with vowel string '-ei', and the user selects it with string tone ①○○③④○. To " give " word, twice string tone and 12 strokes is quite satisfactory. Additionally, some Chinese characters selected for the two-tone chord and the strokes required to write them are shown in FIG. 27 .

Figure 22 can be used to determine which two strings are input to them. Examples of Chinese characters with the third string

If there is only one character for a particular pinyin, but there are other pinyins that start with its initial and vowel string, that character will be displayed at the character selection junction, allowing the user to select it with only the third chord. Fig. 28 shows the CHIME grid and chord sequence for entering a Chinese character ("恩" bai1) with only 3 chords. Starting from the CHIME basic grid, the user selects the initial letter 'b-' with the string tone ○○○④○, and then selects the syllable 'bai' with the string tone ○○③○⑤.

Because bai1 has only one Chinese character, it is shown in the grid block associated with the first tone and without ending. Here, the tertiary strings (○○○④○, ○○③○⑤, ○○○○⑤) are quite satisfactory compared with the 12 strokes required to write "把". Other third-sound Chinese characters and stroke numbers are shown in FIG. 29 .

Figure 22 can be used to determine, which third chords are input to them. CHIME's design

The necessary thing of CHIME is to enable Chinese character input. If it is to be commercially viable, it is also important to minimize user effort in the process. It is also desirable and of secondary importance that it provide methods for other input functions. This section presents an overview of minimizing the effort for users of CHIME, enumerates certain optimizations for CHIME efficiency, and then shows how CHIME can be made suitable for other environments. Minimize workload

CHIME minimizes long-term workload as well as that of everyday users. Minimization of long-term user workload

CHIME minimizes the user's workload through a self-educating design, which reduces the need to keep looking at the OHAI window. This self-education is achieved through sequence and consistency of choice, that is, the same choice at the same juncture in the process always produces the same result. Combining minimization and consistency of sequence and selection enables users to remember chord/input associations by using them rather than by special effort. The result is that the user takes advantage of his daily efforts: for a short time he no longer needs to look at the OHAI window to select Pinyin; and for a slightly longer time he doesn't have to look at the window for the most frequently used Chinese character input. Daily user workload is minimized

Minimized daily workload means less effort is required to generate Chinese characters with CHIME than any other method (including writing). These are achieved through the basic design of CHIME, which performs the following functions:

1. Minimize the number of steps to produce a string sequence of a Chinese character.

2. Minimize the number of selections necessary for each step in the string sequence.

3. Use a sequence number in each step of selection to locate the desired input and sub-inputs.

The quality of these CHIMEs is accomplished through the optimizations listed below. optimization

If CHIME simply lets the user select the Pinyin letters one by one, and then choose from the Chinese characters corresponding to that syllable, it requires only a little less work than with the keyboard-based approach. CHIME has made a series of optimizations to improve its adaptability to Chinese character input tasks. Mutually exclusive initials, semivowels and strings of vowels

Combining selections as much as possible is an optimization that frees strings to be used for other purposes. Note that there are 21 initial consonants, which together with no initial consonants give 22 options for starting a pinyin. While the 22 options comfortably fit the 31 options in the OHAI grille, it is desirable to have as few as possible, especially in the basic grille, leaving excess for other uses. Observations about Mandarin syllables (Fig. 16) allowed us to leave 3 chord blocks for the basic (initial selection) grid and 1 chord block for the second (vowel selection) grid.

The first observation involves 2 sets of initials: (j,g,x) and (z,c,s) The initials (j,q,x) and (z,c,s) are mutually exclusive only in that they follow content; (j, g, x) before long sounds; and umlauts before ü; (z, c, s) before short sounds -1, a, e, o and u. This can be seen in the grayed out area in Figure 16. These differences and phonetic similarities of (j,z), (q,c) and (x,s) allow CHIME to combine this pair/column, reducing the number of chords by 3 for initial consonant selection.

There are 3 mutually exclusive ways to represent syllables that do not start with a consonant. For strings of vowels beginning with 'a', 'o' or 'e' start with a vowel. For vowel strings beginning with 'i', the Pinyin system substitutes 'y'. For vowel strings beginning with 'u', the Pinyin system substitutes 'w'. These 3 mutually exclusive choices are combined in the choice case associated with the string sound ○○○○⑤ lacking the initial consonant.

Similar observations allow CHIME to vacate a chord block during the vowel selection phase. There must be a consonant in front of the short sound '-i', and there is no consonant in front of '-er', so the same second string can choose both. FIG. 21 is the result of this folding process in the Mandarin syllable diagram in FIG. 16 . Strings of initials and vowels instead of letters

Choosing sub-input sequences instead of individual sub-inputs whenever possible is an optimization that reduces join points and thus speeds up input. In CHIME, this optimization manifests itself in the ability to select two-letter initials (such as 'sh-'), entire strings of vowels (such as '-iao'), and the tones and endings within a chord. This alternative selection scheme with OHAI is possible in the case of selecting letters one at a time, but this selection and additional optimizations allow CHIME to improve upon previous methods in terms of accessibility and speed.

Using this strategy, each CHIME junction saves the user effort: a basic junction saves one chord selection by including the double-letter initial in the basic grid; a syllable junction saves two by representing all vowel strings. Subchord selection; Pinyin junction saves subchord selection by combining tones and endings. The input process is accelerated because the user needs fewer string selections to generate a Chinese character. Minimization of choice at each junction

Memorability is improved by minimizing the total number of chord/input relationships and the total number of junctures and the number of ties at each juncture. Figure 23 shows that the number of relations at basic junctions is 24, the number of relations at syllable junctions is 21, and the number of relations at pinyin junctions is 18. Memorize 64 facts and generate 1279 pinyin. This number makes the whole pinyin selection process very easy to remember. Consistency of choice at each juncture

When the user looks at the OHAI window, he always sees the same selection at the same position of each junction. Improves localization by eliminating the need to glance at each junction. When being familiar with the 64 relations of the string sound and the pinyin part, the user's eyes are trained to directly watch the selection grid. This in turn speeds up the input selection process. Ability to select any tone

Sometimes local speakers are unsure of the tone for a particular character, so CHIME includes an "arbitrary tone" option at the Pinyin junction. When this selection is plucked, the Chinese characters of all tones can be displayed as much as the grid behind it can hold. Figure 30 shows the grid displayed after selecting any tone for "shi". This grid shows the Chinese characters of all tones for the syllable "shi" arranged in order of stroke number. Any tone grid is the only exception for no Chinese characters requiring more than 5 chords. Preview the next grille

Fig. 28 shows a mixed grid of pinyin and Chinese characters. When there are 4 or fewer Chinese characters associated with a particular pinyin, CHIME shows them instead of the pinyin. This has the advantage of indicating which Chinese character will appear in the next grid in what order. If the Chinese character is often used, it points out the required string tone at the position of the string tone block at the previous junction without looking at the next grid. Change your mind about optimization

Because there are many empty selection boxes in each of the first 3 junctions, CHIME allows SPACE (space), O-9, PUNCT (punctuation) and CANCEL (cancel) to all be placed in it, enabling the user to select At any point in the process enter a space, number, math symbol, punctuation mark or quad into the basic grid. This means that if the user changes his mind during the selection process, he does not have to go back to the basic grid. only show existing

In order not to distract the user's attention, CHIME only displays those syllables, pinyin and Chinese characters that exist at each junction point of the Chinese character text input process, and does not display those that may exist (for example, in the second grid, the selected Initials combined with all vowel strings, or selected syllables combined with all possible tones), this minimizes visual distraction and allows the user to locate selections more quickly. Intuitive selection grid content

By displaying the contents of a cell, tell the user as much as possible and clearly what will happen when the string associated with that cell is plucked. When plucking strings, the order of selection is "sh", "shi" and then "shi", not "sh", "-i", and then "φ(4)", the former is the actual meaning of the string. The user then sees what he expects to input instead of its bits and segments. Examples of Adaptation

In particular, proposing a minimum number of choices for the first and last sequential chords is consistent with CHIME's goal of adapting to the environment. Examples of adaptive functions are: user-defined symbols, selection of Pinyin input instead of Chinese character input, selection of multi-Chinese phrases starting with the selected Han Ding. Adaptation entries are inserted at the first and last junctures in CHIME; between these junctures, it is neither the beginning nor the end of the user's process of selecting a Chinese character until the adaptation function is ready.

One type adapts to the environment, and scalability includes enabling the user to do some work on a certain letter other than selecting a pinyin at the first CHIME grid. This is done by using empty grid blocks for extended functions. For example, if the user wants to be able to switch between Chinese and English input, it can use a checkbox at the first junction to select OTIS and return to CHIME with an empty OTIS checkbox, thereby integrating OTIS into CHIME. A scenario for this is given in FIG. 31 .

The basic grid applied by each OHAI grid group is relabeled with the name of the imported language for the user's convenience. All that is needed to accomplish this integration is a method to launch OTTS from CHIME and a method to launch CHIME from OTIS. All that is required to make OTIS accessible from CHIME is to use an empty chord block in the base grid to refer to the base grid of OTIS. Making CHIME accessible from OTIS is a trick, since all possessives are used. In fact there is only one space in the OTIS grid. Instead of doing a lot of rearranging and accepting the necessity of pizzicato into the PUNCT grid to get into CHIME, we use this empty grid to reference to CHIME's basic grid. Therefore, in order to start OTIS from CHIMW, the user plucks the string ①②○○○. In order to return to CHIME from OTIS, he plucked the strings ①②③④○, ①②③④○. Both are easy to remember. Advantages of CHIME over keyboard-based methods

TwinBridge only displays 10 Chinese characters at a time and forces the user to type a key to change to another 10 Chinese character groups, which takes time. CHIME displays all available Chinese characters in one or two grids. Because the OHAI gizmo is used, and the OHAI window occupies the same display window as the application window, the user makes fewer visual shifts. Users really don't have to backtrack from prompt screens. Only 1% of Chinese characters may be ignored on the first pass. Other uses of CHIME

The interface for selecting Chinese characters for input can be used in other application systems. An example of an application that uses the same interface is a Chinese character dictionary. Instead of sending the input, the dictionary application can display the definition of the selected character. Another possible use is as a teaching and translation tool: For non-fluent speakers of these languages, the same software can tell users the meaning of selected Chinese characters. In the search mode, OHAI displays the translation for the Chinese character of the plucked string, and only sends the Chinese character to the user's application system window if the string is selected again. Technology: OHAI's hardware and firmware under OHAI's shell

WON is an example of OHAI hardware. The OHAI hardware can take any form as long as it sends string information to the OHAI software. The OHAI firmware is the program resident on the chip in the hardware that translates the pressing of the strings into interpreted signals and sends them to the OHAI software on the user's computer. firmware description

All the firmware does is monitor the state of the 5 switch inputs corresponding to the fingers holding the OHAI device. When the collective state of these 5 inputs changes, the firmware causes the device to send the corresponding chords to the computer where the OHAI software resides. Figure 33 is a flowchart illustrating one method of applying the firmware. Note that this design ensures that no duplicate chords are sent in the sequence, and only a NULL chord is sent when a chord is fully released.

In the flowchart of Figure 33 apply the expression

16(F ₄ )+8(F ₃ )+4(F ₂ )+2(F ₁ )+T where F is finger press, T is thumb press (the lower the F _x the closer to the thumb .This expression guarantees that the integers associated with the chords are unique for each finger combination.

As shown in FIG. 33, a chord accumulator is used so that the "send chord" process sends only completed chords. hardware form

In user environments where right-handed and left-handed users use the same computer, it is wise to have a form of OHAI device that is easily usable in both hands. A device that meets these needs is shown in Figure 39.

The workstation OHAI handpiece shown in Figure 39 resembles a joystick and has a base and a key tray leading from the base. The key tray includes a top surface having a single push key switch 50 which is pressed by the user's thumb and a set of four push key switches 51, 52, 53, 54 extending along the front face of the tray facing the other side facing the user. . These push-button switches are positioned so that the device can be grasped and used by either the right or left hand.

The simplification of the OHAI hardware requirements makes possible a variety of designs, each suited to its particular user environment or user preference. For example, one principle for determining a user's environment is the degree of mobility required. A moderately mobile user uses a laptop. A very mobile user needs a very small computer that can be lifted and taken down in a fraction of the time; a wearable or palmtop computer is suitable for such an environment.

Figure 35 and Figure 36 show the styles of OHAI equipment suitable for laptops. OHAI's laptop includes one or two groups of five keys 40, 41 and 43, 44 which are ergonomically placed on the laptop computer chassis. While two sets of keys can work together, having two sets allows the user to alternate between them to eliminate hand fatigue caused by spending too much time in the same position. Similar button configurations can be placed on armtops, palmtops, and wearable computers.

In Figures 35 and 36, key 43 is the topmost of a group of keys used by the left hand on the monitor portion of the laptop computer. The key 43 is for the thumb of the left hand. Keys 44 are used by other fingers.

A set of keys 40 and 41 are thumb and finger keys respectively, ergonomically and optionally located on the laptop. The little finger presses the leftmost key 45 .

For maximum comfort, the user can use either the front set (40, 41 ) or the rear set (43, 44) at any time.

For wearable (especially military field use) computers, the OHAI glove (Fig. 37) is the most suitable variety of OHAI equipment. It includes 5 sensors integrated in the fingertip of the glove, OHAI firmware and a method to send string sounds to the wearable computer when the finger is pressed. If the wearable computer has a headgear (FIG. 38), the interaction of the glove with the OHAI software and the OHAI window in the headgear display is equivalent to the interaction of WHO, the OHAI software, and the monitor screen.

Fig. 39 shows an example of chords corresponding to letters for non-display application systems.

There is a special OHAI interface for the audio/audio interface, such as it can be used in military sites that require safe silent communication. In these cases, the computer is linked to a radio or other communication device. Since silent communication is required, Fig. 41 presents a scheme for such secure silent communication. Implement OHAI application type

All OHAI applications include software that receives string information from the OHAI hardware. The software is different for each of the 3 types of OHAI application systems. OHAI grid group application system

CHIME and OTIS are examples of OHAI grid application systems.

Because there are many different OHAI grid group applications and they can be modified by the user, the complexity is dealt with in the data file. Because the data files are separate from the OHAI software, they can be changed without risking breaking OHAI. Each of these will be described in sufficient detail to enable a suitably competent programmer to implement the entire system. There are many ways to optimize the system, these are left to individual implementors. Instead of presenting a sound implementation theory, we use OTIS extensively as examples. Because the data file determines what OHAI does, this section begins by describing an OHAI grid application file. Overview of data files for OHAI grid group application OHAI grid application files

The behavior of the OHAI grid application system is fully specified by the OHAI grid application document. The OHAI grid application file includes information specifying the content of the selection grid, user application input and subsequent grids of the selection grid.

For example, the selection box in the OTIS basic grid (Figure 5) to display the "space" chord causes a space character (ASCII32) to be sent to the user application, with the basic grid itself as a subsequent grid. Organization of OHAI Grille Group Application Files

Figure 32 shows the file structure of the application system for the OTIS OHAI grid group. The file includes entries for each selection cell. Each item consists of 3 fields; these fields are called display, input and follow.

The display field indicates what is displayed in the checkbox.

The input field indicates what to send to the user application.

The subsequent field indicates the subsequent grid.

If the NULL constant is in the display field, it indicates what is not to be displayed, and in the input field, it indicates what is not to be sent to the user application system. For input that cannot be displayed. Such as space, backspace, put a backslash (,) in front of the corresponding ASCLL code. If backslashes are entered use two backslashes as input fields. There is always a grid name in the subsequent field. There are 32 sequential items for the grid in the OHAI application system. Grid fields always refer to one of these grid groups. An Implementation of OHAI Application File

One way to implement an OHAI grid application file is to use tab characters to delimit fields and backs to delimit items. Using such an implementation, the line for OTIS would look like this:

LOWER NULL NULL

Space Space \032 LOWER

a a a LOWER

b b b LOWER

...

UPPER NULL NULL

Space 1032 LOWER

A A LOWER

B B LOWER

...

The file has 192 lines, 32 lines for each grid group arranged in an arc, so item 1 is the header block, item 2 is for the 1st chord (○○○○⑤), and item 3 is for the 2 strings (○○○④○), etc. With the exception of the basic grid, which must be the first, the other grid groups can be in any order. OHAI software applied to OHAI grid groups

Figure 34 is a flowchart showing how the OHAI software uses the OHAI grid application data file to process data from OHAI to send input to the user application system. It is beneficial to view the output of the OHAI software as the input of the user application system.

The first thing the software must do is present the contents of the OHAI window. The content of the first letter is determined by the first grid group of the file. All display fields in the first grid group item are displayed in the OHAI window from top to bottom and left to right, so the order in the file matches the order in the item.

After initializing the OHAI window, the OHAI software sets a variable that keeps track of the second most recent ("previous chord") chord to the NULL chord from the OHAI device.

Once these initializations are complete, enter the main loop of the software. The first thing this loop does is wait for a string to be received from the device. Once it receives a chord into a variable that holds the most recent chord (the current chord), there are three possibilities:

1. The current string tone is NULL string tone.

2. The current string is not a NULL string, and the previous string is a NULL string.

3. The current string is not a NULL string, and the previous string is not a NULL string.

Case (1) corresponds to the user pressing and releasing a string. Since the firmware cannot sequentially send the same chord twice, and send this NULL chord when all keys are released, case (1) can only happen if the previous chord was not NULL. This means that the user presses and releases a note, then selects it; the pressed note is recorded as the previous note and releasing it causes the current note to become NULL. The first action the software takes to process a selected chord is to de-highlight the previous chord (it is no longer tentative). After de-highlighting, any input related to the chord is sent to the user application (the input field is non-NULL). If the subsequent grid for the selected chord/input is different from the current grid, the current grid is adjusted to the subsequent grid, which is in the file, and the OHAI window updates it. After processing the display, input and subsequent grid of this chord, the software overwrites the previous chord with the current chord, so processing can start over again by waiting for the chord to come from the device.

If the current chord is non-NULL (case 2 and case 3), this means the user is starting or in the middle of a trial chord sequence.

Case (2) corresponds to the user pressing a string after having released the previous string, but not releasing it. In case (2), the previous string received from the device is NULL indicating that the previous string has been completely processed (or the first string received by the software at this time); this means that the current string is the first string in the trial string sequence one (and possibly the last). The only thing the software does for the first tentative selection is to highlight the selection box for that string in the OHAI window. After updating the OHAI window, the software overwrites the previous chord with the current chord, so processing can start over again by waiting for the chord to come from the device.

Case (3) corresponds to the user pressing a string after a different string without inserting and releasing the action. This is the case when he changes his mind or he momentarily presses another finger to complete a note before letting go; each finger he adds is a tentative note. With each change, the software de-highlights the previous chord and highlights the selection box for the current chord. After updating the OHAI window, the software overwrites the previous chord with the current chord, so processing begins again by waiting for the chord from the device. Loops dealing with chords go on infinitely

Although the invention has been described with reference to specific embodiments, modifications and variations of the invention can be constructed without departing from the scope of the invention as defined in the following claims.

Claims (26)

1. A system for representing and submitting input selections to user application systems, including memory for associating string sequences or syllable sequences with complex user application system input selections; a system for associating strings or syllables with complex user application system input selections; An optional display associated with; an input device for inputting chords or syllables; a first processor associated with the input device for generating input signals from the chords or syllables; a second processor connected with the first processor , for receiving an input signal sequence from the first processor, for estimating the input signal sequence, for linking to an optional display for modifying the display based on the estimated input signal sequence, and for linking to the user application system for applying the user application input selection sent to the user's system. 2. The system of claim 1, wherein the input device comprises a keypad having keys sized and placed in accordance with ergonomic principles such that a designation is accessible by and accessible to a finger having a plurality of fingers, And make any key, all keys or any combination of keys can be manipulated by fingers at any time. 3. The system of claim 1, wherein said input device is similar to a joystick and includes a base, a key bracket extending from said base, said key bracket having a A bottom, a top surface, a pair of sides, a front and a back, and a plurality of keys, wherein at least one key is provided along at least one side. 4. The system of claim 3, wherein the at least one key provided along at least one side includes 1 and 4 keys on the top side extending widthwise along a side or a front. 5. The system of claim 2, wherein said device is or is housed in a computer or computer peripheral. 6. The system of claim 2, wherein said input device is integrated with a computer, and at least one set of depressible keys or keys is provided on a chassis of said computer. 7. The system of claim 6, wherein a plurality of sets of ergonomically positioned keys are provided, at least one set on the keyboard housing and at least one set on the monitor housing. 8. The system of claim 2, wherein each key can be pressed using a single finger or thumb without having to remove or reposition any fingers. 9. The system of claim 2, wherein pressing and releasing a set of at least one key provides input to the first processor. 10. The system of claim 1, further comprising a monitor for displaying user input selections for enabling the system. 11. The system as claimed in claim 1, wherein the display shows the connection between the user application system input selection and the chord or syllable sequence by displaying a grid comprising a grid name block and a plurality of chord blocks. . 12. The system of claim 1, wherein a chord block includes a coded finger grid and a selection grid adjacent to the row of finger grids for identifying a combination of pressed and released keys. 13. The system of claim 12, wherein the coded finger grid is a lighted finger grid for identifying key presses and releases. 14. The system of claim 13, wherein at least one block in a grid includes a selection grid for calling up a subsequent grid. 15. The system of claim 14, wherein the selector, software and firmware are adapted to input Chinese characters or other complex characters. 16. The system according to claim 15, wherein the display grids for inputting Chinese characters include grids for selecting initials of Pinyin, grids for selecting vowel strings, and grids for selecting ending/tone combinations and grilles chosen for Chinese characters. 17. The system of claim 1, wherein said input device is a glove including keys integrated into fingertips. 18. The system of claim 1, wherein the display resides in a headgear having a small monitor. 19. The system of claim 1, wherein said user application includes an audio/voice interface for providing communication. 20. A method of representing and presenting to a user application system input selection comprising the step of associating a complex user application system input selection with a sequence of chords or syllables in memory; optionally displaying the chords in the sequence of chords or syllables or syllables associated with complex user application input selections; using an input device to receive user-generated chords or syllables; generating input signals from input chords or syllables; estimating input signal sequences, modifying optional displays based on estimated input signal sequences and prompting for a chord or syllable in the sequence of chords or syllables associated with the complex user application input selection; and sending the associated complex user application input selection to the user application. 21. The method of claim 20, further comprising providing a display and modifying the display based on the estimated sequence of input signals, and further comprising providing a grid on the display for associating chords with user application system inputs. 22. The method of claim 21, wherein providing the grid on the display includes displaying an arrangement of blocks including a grid name block and at least one row of finger grids and a large grid below the row of finger grids. The selection grid, the finger grid code is used to identify the combination of pressing and releasing the switch. 23. The method of claim 21, wherein providing a grid displaying chords associated with user application system input comprises displaying the grid on a computer monitor. 24. The method of claim 20, wherein the step of estimating the input signal sequence further comprises modifying a display representing the relationship of complex user application input selections to chords or syllables. 25. A means for representing and presenting user application system input selections, including the association of chords or syllables with complex user application system input selections; an input device for inputting chords or syllables; an input device for receiving chords or syllables and a first processor for generating input signals; a second processor connected to the first processor for receiving and evaluating input signals and input signal sequences, and for sending user application system input selection signals according to the input signal sequences to a user application system. 26. The apparatus of claim 25 further comprising a display coupled to the second processor for receiving a display modification signal from the second processor based on the estimated input signal received by and evaluated by the second processor.

Images