CN114047872B - Text input method and system - Google Patents
Text input method and system Download PDFInfo
- Publication number
- CN114047872B CN114047872B CN202111183508.6A CN202111183508A CN114047872B CN 114047872 B CN114047872 B CN 114047872B CN 202111183508 A CN202111183508 A CN 202111183508A CN 114047872 B CN114047872 B CN 114047872B
- Authority
- CN
- China
- Prior art keywords
- input
- user
- text input
- gesture
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04812—Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a text input method and a text input system, which can combine gestures and a ring character input interface, reduce the input memory burden of a user, improve the use acceptance of public places of the user, and realize efficient and accurate input. The specific scheme comprises the following steps: the start signal and end signal detection device detects the received trigger signal in real time and sends the trigger signal to the computing device. The user gesture detection device detects receipt of an input gesture identifying a user into the computing device. The display equipment is used for displaying the CGR environment and comprises a virtual character interface, a text input interface and a virtual environment; the computing equipment is provided with the following modules: the user gesture recognizer is used for recognizing the input gesture; the starting signal and ending signal identifier is used for identifying the trigger signal; and the text input controller predicts candidate characters, input characters and a word list of candidate words input by a user through a prediction algorithm according to the recognition result of the input gesture, and controls a text input interface and a virtual character interface to be displayed in a display.
Description
Technical Field
The invention relates to the technical field of Computer Generated Reality (CGR), in particular to a text input method and a text input system.
Background
At present, how to help a user to perform efficient and quick text input in a computer generated display CGR environment under the conditions of not increasing the memory burden of the user and improving the willingness of the user acceptance is an unresolved problem.
Patent: CN109126125A discloses a text input method based on a double-rocker controller in a virtual reality environment, comprising: positioning a text input position in a virtual reality environment, and popping up a virtual keyboard; shaking the area where the target character is positioned by the first rocker and shaking the target character by the second rocker; and judging whether the holding time of the second rocker position is larger than a set threshold value, and if so, judging that the continuous input of the target character is realized.
The input mode must rely on a two-rocker control. In a mobile mixed reality environment, the carrying of peripheral devices can cause inconvenience in use; second, this approach requires the use of two hands to accomplish the input, which the user cannot use with one hand. The mode is abrupt when being used in public places, and the use willingness of users is reduced.
Patent: CN 102567631a discloses a non-resident text input method for mobile virtual reality head mounted display, comprising: positioning a text input position in a virtual reality environment, and popping up a virtual keyboard; controlling the virtual cursor to move to the area where the target character is located by utilizing the rotation of the head; searching the recommended word closest to the input character in the pre-stored dictionary according to the input character, and dynamically displaying the recommended word outside the currently input character area; if the target word exists, the virtual cursor is controlled to enter the region where the target word exists to select the word by utilizing the head movement, otherwise, the character is continuously input.
This patent uses head movement to input text, requires no stopping of the head movement to input text, and is prone to motion sickness. The user needs to select characters on a virtual keyboard in the virtual environment through head movements, which causes a shielding problem in the virtual environment. The head-movement interaction is abrupt when used in public places, so that the use will of users is reduced.
Disclosure of Invention
In view of the above, the present invention provides a text input method and system, which can combine gestures and a circular/linear single-line character input interface, reduce the input memory burden of a user, improve the use acceptance of public places of the user, improve the input speed, and reduce the input error rate.
In order to achieve the above purpose, the technical scheme of the invention is as follows: a text input system includes a computing device, a user gesture detection device, a start signal and end signal detection device, a display device.
The start signal and end signal detection device is used for detecting the received trigger signal in real time and sending the trigger signal to the computing device.
A user gesture detection device for detecting receipt of an input gesture identifying a user into the computing device.
The display equipment is used for displaying the CGR environment and comprises a virtual character interface, a text input interface and a virtual environment; also for displaying a user partial or whole body avatar; the text input interface includes a list of candidate words and text that has been entered by the user.
The computing device is a computing device with a processor and a memory, and the computing device is provided with the following modules: a user gesture recognizer, a start signal and end signal recognizer, an application controller, and a text input controller.
The user gesture recognizer is used for recognizing input gestures, and the input gestures comprise three types: a first gesture, a second gesture, and a third gesture; the first gesture is used to pre-select a character or characters in the character interface as alternative characters; the second gesture is used for the user to confirm and select the alternative characters as input characters; a third gesture is also included for selecting a word from the list of candidate words as an input word.
The starting signal and ending signal identifier is used for identifying a trigger signal, and the trigger signal comprises a text input starting signal and a text input ending signal; the text input start signal is used for triggering the start of a text input process, and the text input end signal is used for triggering the end of the text input process.
And the text input controller predicts candidate characters, input characters and a word list of candidate words input by a user through a prediction algorithm according to the recognition result of the input gesture, and controls a text input interface and a virtual character interface to be displayed in a display.
The application controller is used for controlling the content related to the application program, including different text input scenes corresponding to different application programs.
Further, the start signal and the end signal identify devices, one of the following devices is adopted: an image sensor, a pressure sensor, a touch sensitive device, a microphone, a position sensor, or an orientation sensor. The start signal and end signal recognition device serves as a text input start signal or a text input end signal based on the detected environmental information or user behavior signal.
Further, the user gesture recognition device performs gesture recognition using one of the following devices: an image sensor, a pressure sensor, a touch sensitive device or an myoelectric detection device.
Further, the text input controller comprises a word predictor, a text input interface controller and a virtual character interface controller; the word predictor predicts the input words according to the input characters selected by the current user, adopts a character position probability model and/or a language model for prediction, predicts the input words through the character position probability model and/or the language model, and generates a word with the maximum probability and a preset number to form a candidate word list.
The position probability prediction model predicts candidate characters and input characters input by the user through the current position of the characters.
The language model judges the possible probability of inputting words according to the inputted content of the user, and a word list of candidate words is obtained.
The text input interface controller controls the display device to display a virtual character interface, wherein the virtual character interface comprises a ring display total interface of all characters, a ring display interface of alternative characters and a display interface of input characters.
The text input interface controller controls the display device to display a text input interface including a candidate word list display interface and an input word display interface.
Further, the user gesture detection device, the start signal detection device, the end signal detection device and the display device are connected through a communication network, wherein the communication network comprises a wired communication mode or a wireless communication mode, and each device comprises a data sending and receiving component.
Further, the prediction adopts a character position probability model and/or a language model, the prediction of the input words is carried out through the character position probability model and/or the language model, and a word forming candidate word list with the maximum probability and the preset number is generated, specifically:
The character position probability model determines the position probability P position of the character according to the input character selected by the user; the language model is divided into two parts, one part is an intra-word prediction model, and the character possibly input by the user at present is judged according to the occurrence probability of words in a word stock and through the character already input by the user, wherein the probability of the character input at present is P letter; the other part of the language model is a prediction model in a sentence, and the probability P word of the most likely word input at present is judged according to the probability of the word in the sentence in the language library and the character already input at present; the word stock and the language stock use the existing word stock and language stock at present, or dynamically adjust according to user input; the probability of entering a word is p=αp position+βPletter+γPword; where α is the weight of the preset character position probability P position, β is the weight of the preset character probability P letter, and γ is the weight of the probability P word of the preset word.
The invention also provides a text input method, which adopts the text input system to input text, and comprises the following steps:
S1, detecting and receiving a text input start signal, and starting the following text input flow.
S2, displaying the annular display total interface of all the characters by the display device.
S3, detecting a first gesture of the user, wherein the first gesture is used for selecting one or more characters in the input virtual character layout as candidate characters.
S4, displaying an annular display interface of the alternative character by the display device.
S5, detecting a second gesture of the user, and confirming and selecting one or more alternative characters by the second gesture to serve as an input character.
S6, displaying a display interface of the input characters by the display equipment.
S7, predicting candidate words according to the current input characters, and generating a candidate word list.
S8, displaying the candidate word list display interface by the display device.
S9, detecting a third gesture or other input signals of the user, and selecting a word from the word candidate word list as an input word.
S10, displaying an input word display interface by the display device.
S11, whether a text input ending signal is detected, if so, ending the text input flow, otherwise, returning to S3.
Further, a voice signal, a gesture signal, an eye movement signal, a head movement signal, or a position signal for representing the start of text input is preset as the text input start signal.
A voice signal, a gesture signal, an eye movement signal, a head movement signal, or a position signal for representing the end of text input is preset as a text input end signal.
Further, the annular display total interface of all the characters and the annular display interface of the alternative characters are specifically: the single or multiple characters are arranged in a circular or oval shape in a single line, and all the character lines are arranged to form a complete or partial ring, and are displayed completely or partially when displayed.
The beneficial effects are that:
1. According to the annular display text input system provided by the embodiment of the invention, the gesture and the annular/linear single-line character input interface are combined, so that the input memory burden of a user is reduced; the small gesture motion and the simple sliding gesture improve the text input speed; based on the thumb and index finger sliding micro-gestures, the user can place the hands at any position and even operate in clothes pockets by combining with a single-line character input interface, so that the use privacy of public places is improved, the abrupt property is reduced, the fatigue of the hands is reduced for a long time, and the use acceptance of the public places of the user is improved; the input text can be predicted by using an inaccurate input mode and combining a position probability model and a language model, so that the input speed is improved, and the input error rate is reduced.
2. According to the text input method with annular display, text input can be performed through gestures of a user in a CGR environment at present, and the method can be used in a mobile scene. In the mode, a user specific gesture is coupled with a specific character, so that the user needs to memorize the specific character, and the memory burden of the user is increased; secondly, larger hand movements can reduce the switching speed of the user between gestures, thereby reducing the text input speed; the larger hand motions are abrupt when used in public places, the privacy leakage can be caused by the coupling between gestures and characters, and the use will of users is reduced.
In the invention, a user performs text input by combining virtual character layout by using micro gestures of thumb and index finger of hand. The micro-gestures are combined with the virtual character layout, gesture input can be completed through simple relative sliding micro-gestures, no specific relation exists between the micro-gestures and the characters, and a user does not need to memorize; the user can quickly complete the micro gesture action, so that the text input speed is improved; the micro gesture motion range is small, the user can finish text input at any position, for example, the user finishes input in a pocket, the abrupt nature of a public place is reduced, meanwhile, a bystander cannot see the virtual environment where the user is located, the gesture is simply repeated, privacy leakage is avoided, and the use intention of the user in the public place is improved; the user's hand can be placed wantonly, and little gesture action range is less, can avoid user's hand fatigue when long-time text input.
3. In the text input system provided by the invention, a prediction algorithm is used, and the specific characters of the current virtual character layout at the hand position of the user and the characters which are already input are combined, so that the prediction of the input text is performed through a character position probability and/or language probability model. The user can realize quick and accurate text input through imprecise text input.
Drawings
FIG. 1 is a schematic diagram of a text input system according to an embodiment of the present invention;
FIG. 2 illustrates an example of a ring-shaped input interface provided by an embodiment of the present invention;
FIG. 3 is a diagram of an annular input interface displayed on the back of a hand according to an embodiment of the present invention;
FIG. 4 is a diagram of a multi-character rendered annular input interface provided by an embodiment of the present invention;
FIG. 5 is a diagram of an input interface presented at the back of the hand for a single character provided by an embodiment of the present invention;
FIG. 6 is an exemplary diagram of a first gesture with a ring interface at the back of the hand provided by an embodiment of the present invention;
FIG. 7 is an exemplary illustration of an annular interface at the index finger provided by an embodiment of the present invention;
FIG. 8 is an exemplary diagram of a second gesture provided by an embodiment of the present invention;
FIG. 9 is another exemplary diagram of a second gesture provided by an embodiment of the present invention;
FIG. 10 is an exemplary diagram of text entry provided by an embodiment of the present invention;
FIG. 11 is a flow chart of text entry provided by an embodiment of the present invention.
Detailed Description
The invention will now be described in detail by way of example with reference to the accompanying drawings.
The invention provides a text input method and a text input system based on finger micro gestures, which can be used for a Computer Generated Reality (CGR) environment, particularly can be used for a moving CGR environment, and can improve the text input efficiency and convenience of a user and the user's intention to use in public places.
The text input mode provided by the invention can be used for text input in a computer-generated reality (CGR) environment.
Computer Generated Reality (CGR) environments refer to fully or partially simulated environments perceived by users through electronic systems. The user can interact with it; the CGR environment may adjust one or characteristics of one or more virtual objects in the environment according to the user's behavior; for example, the CGR system detects a swipe gesture by the user, and in response, the CGR environment adjusts the presented image content. The user may perceive and interact with virtual objects in the CGR through any sensory, including visual, auditory, tactile, olfactory, and so forth; for example, a user may perceive and may interact with audio objects in a CGR environment.
Virtual Reality (VR) environments and Mixed Reality (MR) environments are special CGR environments. The mixed reality environment includes an Augmented Reality (AR) environment and an Augmented Virtual (AV) environment. Wherein the CGR environment comprises at least a portion of a Virtual Environment (VE). A virtual reality environment refers to a simulated environment that is entirely computer-generated. A mixed reality environment refers to a simulated environment that contains one or more inputs from a physical environment or representations thereof in addition to a computer-generated simulated environment; the mixed reality environment refers to a situation between the full physical environment and the virtual reality environment, but does not include both situations. An augmented reality environment refers to a simulated environment in which a representation of a physical environment is partially transformed by computer-generated information. Enhanced virtual environments refer to virtual or computationally generated environments in combination with simulated environments from physical environment inputs.
Many different types of devices may be used to display the CGR environment, enabling a user to perceive and interact with the CGR environment, including but not limited to head-mounted displays, smart phones, smart watches, tablets, heads-up displays, glass with display functionality, and the like.
Fig. 1 shows an exemplary system for accomplishing such text entry. The system comprises a computing device, a user gesture recognition device, a start signal and end signal recognition device and a display device.
The start signal and end signal recognition device is for detecting a particular signal or signals for triggering the system to enable a user to start entering text and end text entry. The device may include, but is not limited to, an image sensor, a pressure sensor, a touch sensitive device, a microphone, a position sensor, an orientation sensor, etc., or other device that may detect a signal of environmental information or user behavior, etc., as a start signal or an end signal. For example, the touch sensitive device may detect a tap or swipe input as the start signal.
And the user gesture recognition device is used for recognizing the input gesture of the user. The input gesture includes at least a first gesture and a second gesture. The first gesture is used to pre-select a character or characters in the character interface; the second gesture is used for the user to confirm and select the characters; in some cases, a third gesture may be included for selection of a word from the list of candidate words. The gesture recognition device may use a variety of devices capable of recording corresponding data for different gestures of a user to accomplish gesture recognition using a computing device, including but not limited to image sensors, pressure sensors, touch sensitive devices, myoelectric detection devices.
The display device is used to display the CGR environment, including virtual character interfaces, text input interfaces, and other virtual environments. In some cases, a user partial or full body avatar may be displayed.
The text input interface includes a list of candidate words and text that has been entered by the user.
The computing device is a computing device having a processor and a memory, including a user gesture recognizer, a start signal and end signal recognizer, a text input controller, and an application controller.
The user gesture recognizer is used for processing data acquired by the gesture recognition device and detecting whether the data is a specific gesture. The specific gestures may be preset for a program or user-defined.
The start signal and end signal identifier is used for processing the start signal and the data acquired by the end signal identifying device, and detecting whether the data is a signal for starting or ending text input. The signal may be pre-programmed or user-defined.
The text input controller predicts the user text input candidate word by a prediction algorithm and displays it on a circular linear character interface or text input.
The word predictor predicts the input word based on the character currently selected by the user, and may use a location probability model and/or a language model. And predicting the input words through the character position probability model and/or the language model, and generating the first three or five or other number of words with the maximum probability to form a candidate word list.
The position probability prediction model predicts the character most likely to be selected by the user through the current position of the character. For example, characters facing the user have the greatest probability of entry, with the character deviating from that direction having a lesser probability of being selected.
The language model judges the probability of the current input character and word according to the character already input by the user.
The application controller is used for controlling content related to the application program, including different text input scenes corresponding to different application programs, and the like.
The computing devices may be integral or separate. The user gesture recognition device, the start signal recognition device, the end signal recognition device and the display device are connected through a communication network, wherein the device can be integrated or independent in a wired/wireless mode. Each device contains data transmitting and receiving components. The processor may be a local processor or a cloud processor. The memory may be various types of local memory including RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, etc., or cloud memory. A processor may read information from, and write information to, the storage medium. For example, the computing device, the real device, and the start signal and end signal recognition devices of the virtual reality head mounted display are integrated into the head mounted display.
The present invention proposes a method for text input in a Computer Generated Reality (CGR) environment. Comprising the following steps: the start signal triggers text input; displaying a virtual character layout; the user continuously uses the first gesture to select and input one or more characters in the virtual character layout as alternative characters; the user uses the second gesture to confirm and select one or more characters; the system predicts the input text according to the current character; the user uses a third gesture or other input signal, such as voice, to complete the input of a word; the end signal ends the text input.
The start signal triggers text entry. In the application, the system determines whether the user is about to enter text at this time by detecting a start signal. When the system detects a start signal, the user can input text by using gestures at the moment; when the system does not detect the start signal, the user will not be able to perform the text input function using the gesture. The start signal may include, but is not limited to, a voice, a gesture, an eye movement signal, a head movement signal, a position signal, and may be capable of triggering a user text input operation as the start signal. For example, using the voice "start typing" as a start signal, when the application is running, the user speaks "start typing" and the system detects the voice as a start signal; when the user viewpoint stays at the position of the application program, which can input the text, for a certain time, the system detects the viewpoint as a starting signal; when the user makes an application predefined gesture, the system detects it as a start signal; when the hand or the body of the user is positioned at a specific position in the virtual environment, detecting the hand or the body of the user as a starting signal; clicking a virtual object in the virtual environment by the user may signal text entry. The start signal triggers the text input function of the application program and the user can start text input.
The end signal ends the text input. In the application, the system determines whether the user has completed text input at this time by detecting a start signal to end the current text input. The end signal may include, but is not limited to, a voice, a gesture, an eye movement signal, a head movement signal, a position signal. For example, the user uses "end input" speech as a signal to end the current text input, and when the system detects the speech, the user has completed inputting, and the gesture will trigger the system input function; detecting a user's hand or body as an end signal when it leaves a specific location in the virtual environment; when the user completes the input of a sentence, clicking the "send" button in the virtual environment may serve as an end signal.
The virtual character layout may be set to always be displayed, at which time the default system is always in a text-capable state, and text input may be performed directly using the first gesture. The virtual character layout is arranged to be displayed after the start signal is detected, and the system can perform text input. For example, in a head mounted display, the virtual character layout is located at the back of the user's hand, palm of the hand, fingers, etc. in a three-dimensional virtual environment; in a smart watch, a virtual character layout is displayed on a portion of the display screen of the smart watch.
The virtual character interface is a circular character interface in which single or multiple characters are arranged in a single line in a circular or oval shape or other similar shape. All character arrangements may form a complete or partial ring and may be displayed in full or in part when displayed.
An example of a ring-shaped input interface is shown in fig. 2, in which the ring-shaped character interface forms a ring for a single character, but the character is displayed as incomplete, and in which the ring-shaped interface may be displayed, or not displayed, but only the character. As shown in fig. 2, the annular character interface may be displayed at the second section of the user's index finger. In addition, the size of the circular character interface may be altered and located at other joints of the user's fingers or at the arm, back of the hand, palm of the hand, etc.
Fig. 3 shows the manner in which the annular input interface is displayed on the back of the hand.
FIG. 4 is an example of a multi-character presentation.
Fig. 5 is another example of a single character presentation at the back of the hand.
In particular, when a user is occupied with one hand, a virtual character interface is displayed at the finger, and a micro-gesture between the thumb and the index finger or other fingers can be used for text input. When the virtual character interface is positioned at the positions of the arm, the back of the hand, the palm of the hand and the like of the user, the other hand is used for carrying out first gesture operation, and text input is completed.
The user selects one or more characters in the single line character interface using the first gesture. The first gesture is a relative sliding gesture of a user's finger along an annular direction at and around the annular input interface. Fig. 6 is an example of a first gesture in which the ring interface is located at the back of the hand, and the index finger of the user slides up and down relative to the back of the hand, and meanwhile, the character interface is updated according to the gesture, and the sliding speed and distance of the character interface can be consistent with the moving speed and distance of the gesture, or can be set independently, and maintain a certain proportional relationship with the moving speed and distance of the gesture. The user may continuously update the character interface using the first gesture to pre-select different characters or groups of characters.
FIG. 7 is an example of an annular interface located at the index finger with sliding of the user's thumb relative to the index finger for preselection of characters.
The user confirms the preselected character or group of characters using the second gesture. The second gesture is a relative sliding gesture of the user's finger at the annular interface with respect to a vertical annular direction at and around the annular input interface, as shown in fig. 8, 9, or other gesture, such as a click, double click, or the like. The second gesture is distinct from the first gesture.
The user selects from the list of predicted words using a third gesture or other signal to complete the entry of a word. The third gesture is distinct from the first gesture. The third gesture may be consistent with the second gesture or may be distinguishable from the second gesture. When the third gesture remains consistent with the second gesture, the second gesture and the third gesture have a temporal order; the system judges whether the second gesture is completed or not, and if so, the current gesture is a third gesture; if the second gesture is not complete, the current gesture is a third gesture. The third gesture/signal may comprise one or more gestures/signals, which may be performed sequentially/cyclically to select a particular word in the word list.
The character position probability model and/or the language probability model make predictions of the input text. The character position probability model determines the probability of occurrence of each character using a state decision of the character input interface P position. The language probability model is divided into two parts, and can be calculated by using an N-gram language model; one part is an intra-word prediction model, and according to the occurrence probability of a word in a word stock, the character possibly input by the user at present is judged through the character already input by the user, and the probability of the character input at present is P letter; part of the (language probability model) is a prediction model in a sentence, and the probability P word of the most likely word input at present is judged according to the probability of the word in the sentence in the language library and the characters already input at present. The word stock and the language stock can be used for existing word stock and language stock at present, and can be dynamically adjusted according to user input. By the formula: p=αp position+βPletter+γPword gets the probabilities of different words of the character that have been entered so far, and gets a list of predicted input words. Where α, β, γ are weight values that can be changed in a computing device application. The specific set values are as follows: alpha is larger and is 0.5; beta is 0.2 and gamma is 0.3.
The character position probability can be built in the system in advance, and can be adjusted by a user to a certain degree.
Each user input may be a precision input or a fuzzy input. In the accurate input, the character or character group input each time is a unique character or character group, namely the character position probability is 1 or 0; in the fuzzy input, the character or character group itself to be input and the surrounding characters or character groups may be input, with a position probability between 0, 1. As shown in fig. 10A, the system sets the character in the user direction as the character to be selected, and in the precise selection, "t" is the character to be input, and the probability that the other characters are input is 0; in the fuzzy selection, "t" and its surrounding characters, such as "s" and "u", are inputted with a certain probability, but "t" has the largest probability, and other characters have a smaller different probability according to the degree of deviation from the direction of the user.
10A-10D illustrate an example of text entry, where a user selects a sequence of characters via successive first and second gestures. In fig. 10A, the user starts to input, at this time, there is no character already input, and the position probability of the character "t" is the largest, then "s", "u", and then "r" v ", and the word candidate list is obtained by combining the language model. When the user has completed selecting the character "t", the user continues character input, and updates the predicted word candidate list in combination with the language model. In 10C, according to p=αp position+βPletter+γPword, the word with the highest probability is "text", and then "test" …, at this time, the user may choose to perform the third gesture to select the word from the candidate word list, or may continue to input, as in 10D, where "term" will not be a candidate word any more.
Fig. 11 is an example of a text input method, which includes the steps of:
S1, detecting and receiving a text input start signal, and starting the following text input flow.
A voice signal, a gesture signal, an eye movement signal, a head movement signal, or a position signal for representing the start of text input is preset as a text input start signal.
S2, displaying the annular display total interface of all the characters by the display device.
S3, detecting a first gesture of the user, wherein the first gesture is used for selecting one or more characters in the input virtual character layout as candidate characters.
S4, displaying an annular display interface of the alternative character by the display device.
S5, detecting a second gesture of the user, and confirming and selecting one or more alternative characters by the second gesture to serve as an input character.
S6, displaying a display interface of the input characters by the display equipment.
S7, predicting candidate words according to the current input characters, and generating a candidate word list.
S8, displaying the candidate word list display interface by the display device.
S9, detecting a third gesture or other input signals of the user, and selecting a word from the word candidate word list as an input word.
S10, displaying an input word display interface by display equipment;
S11, whether a text input ending signal is detected, if so, ending the text input flow, otherwise, returning to S3. A voice signal, a gesture signal, an eye movement signal, a head movement signal, or a position signal for representing the end of text input is preset as a text input end signal.
The annular display total interface of all characters and the annular display interface of the alternative characters are specifically: the single or multiple characters are arranged in a circular or oval shape in a single line, and all the character lines are arranged to form a complete or partial ring, and are displayed completely or partially when displayed.
In summary, the above embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. A text input system, comprising a computing device, a user gesture detection device, a start signal and end signal detection device, a display device;
The starting signal and ending signal detection equipment is used for detecting and receiving a trigger signal in real time and sending the trigger signal to the computing equipment;
A user gesture detection device for detecting that an input gesture of a user is received and identified and sent to the computing device;
The display equipment is used for displaying the CGR environment and comprises a virtual character interface, a text input interface and a virtual environment; also for displaying a user partial or whole body avatar; the text input interface comprises a candidate word list and text input by a user;
the computing device is a computing device with a processor and a memory, and the computing device is provided with the following modules: a user gesture recognizer, a start signal and end signal recognizer, an application controller, and a text input controller;
The user gesture recognizer is used for recognizing the input gestures, and the input gestures comprise the following three types: a first gesture, a second gesture, and a third gesture; the first gesture is used for pre-selecting a character or characters in a character interface as alternative characters; the second gesture is used for confirming and selecting the alternative characters by the user and is used as an input character; a third gesture is further included for selecting a word from the word candidate word list as an input word;
the start signal and end signal identifier is used for identifying the trigger signal, and the trigger signal comprises a text input start signal and a text input end signal; the text input start signal is used for triggering the start of a text input process, and the text input end signal is used for triggering the end of the text input process;
The text input controller predicts candidate characters, input characters and a word list of candidate words input by a user through a prediction algorithm according to the recognition result of the input gesture, and controls a text input interface and a virtual character interface to be displayed in a display;
The application program controller is used for controlling the content related to the application program, and comprises different text input scenes corresponding to different application programs;
the text input controller comprises a word predictor, a text input interface controller and a virtual character interface controller;
the word predictor predicts the input words according to the input characters selected by the current user, predicts the input words by adopting a character position probability model and/or a language model, predicts the input words by the character position probability model and/or the language model, and generates a word with the maximum probability and a preset number to form a candidate word list;
the position probability prediction model predicts candidate characters and input characters input by a user through the current position of the characters;
the language model judges the possible probability of inputting words according to the inputted content of the user to obtain a word list of candidate words;
The virtual character interface controller controls the display device to display a virtual character interface, wherein the virtual character interface comprises a ring-shaped display total interface of all characters, a ring-shaped display interface of alternative characters and a display interface of input characters;
The text input interface controller controls the display device to display a text input interface, wherein the text input interface comprises a candidate word list display interface and an input word display interface;
The prediction adopts a character position probability model and/or a language model, predicts input words through the character position probability model and/or the language model, and generates a word with the maximum probability and the preset number to form a candidate word list, specifically:
the character position probability model determines the position probability P position of the character according to the input character selected by the user; the language model is divided into two parts, one part is an intra-word prediction model, and the character possibly input by the user at present is judged according to the occurrence probability of the word in a word stock and the character possibly input by the user at present, wherein the probability of the character input at present is P letter; the other part of the language model is a prediction model in a sentence, and the probability P word of the most likely word input at present is judged according to the probability of the word in the sentence in the language library and the character already input at present; the word stock and the language stock use the existing word stock and language stock at present, or dynamically adjust according to user input; the probability of entering a word is p=αp position+βPletter+γPword; where α is the weight of the preset character position probability P position, β is the weight of the preset character probability P letter, and γ is the weight of the probability P word of the preset word.
2. The system of claim 1, wherein the start signal and end signal identifying devices employ one of:
an image sensor, a pressure sensor, a touch sensitive device, a microphone, a position sensor, or an orientation sensor;
The start signal and end signal recognition device serves as a text input start signal or a text input end signal according to the detected environmental information or user behavior signal.
3. The system of claim 1 or 2, wherein the user gesture recognition device performs gesture recognition using one of:
an image sensor, a pressure sensor, a touch sensitive device or an myoelectric detection device.
4. The system of claim 1 or 2, wherein the user gesture detection device, the start signal and end signal detection device and the display device are connected via a communication network, including a wired communication or a wireless communication, each device including a data transmitting and receiving component.
5. A text input method, characterized in that it uses the text input system as claimed in any one of claims 1 to 4 for text input, comprising the steps of:
s1, detecting and receiving a text input start signal, and starting the following text input flow;
s2, displaying an annular display total interface of all characters by the display equipment;
s3, detecting a first gesture of a user, wherein the first gesture is used for selecting one or more characters in the input virtual character layout as alternative characters;
s4, displaying an annular display interface of the alternative character by the display equipment;
s5, detecting a second gesture of the user, wherein the second gesture confirms and selects one or more alternative characters as input characters;
s6, displaying a display interface for inputting characters by the display equipment;
s7, predicting candidate words according to the current input characters to generate a candidate word list;
s8, displaying a candidate word list display interface by display equipment;
S9, detecting a third gesture or other input signals of a user, and selecting a word from the word list of candidate words as an input word;
S10, displaying an input word display interface by display equipment;
s11, whether a text input ending signal is detected, if so, ending the text input flow, otherwise, returning to S3.
6. The text input method of claim 5, wherein a voice signal, a gesture signal, an eye movement signal, a head movement signal, or a position signal for representing a start of text input is preset as the text input start signal;
a voice signal, a gesture signal, an eye movement signal, a head movement signal, or a position signal for representing the end of text input is preset as a text input end signal.
7. The text input method according to claim 5 or 6, wherein the annular display total interface of all characters and the annular display interface of the alternative characters are specifically:
The single or multiple characters are arranged in a circular or oval shape in a single line, and all the character lines are arranged to form a complete or partial ring, and are displayed completely or partially when displayed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111183508.6A CN114047872B (en) | 2021-10-11 | 2021-10-11 | Text input method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111183508.6A CN114047872B (en) | 2021-10-11 | 2021-10-11 | Text input method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114047872A CN114047872A (en) | 2022-02-15 |
| CN114047872B true CN114047872B (en) | 2024-07-23 |
Family
ID=80205256
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111183508.6A Active CN114047872B (en) | 2021-10-11 | 2021-10-11 | Text input method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114047872B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116069169A (en) * | 2023-03-29 | 2023-05-05 | 深圳市光速时代科技有限公司 | Data processing method and system for inputting virtual text based on intelligent watch |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104718544A (en) * | 2012-10-16 | 2015-06-17 | 谷歌公司 | Partial gesture text entry |
| CN111831112A (en) * | 2020-05-26 | 2020-10-27 | 北京理工大学 | A system and method for text input based on eye movement and finger micro gestures |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11314396B2 (en) * | 2018-05-09 | 2022-04-26 | Apple Inc. | Selecting a text input field using eye gaze |
-
2021
- 2021-10-11 CN CN202111183508.6A patent/CN114047872B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104718544A (en) * | 2012-10-16 | 2015-06-17 | 谷歌公司 | Partial gesture text entry |
| CN111831112A (en) * | 2020-05-26 | 2020-10-27 | 北京理工大学 | A system and method for text input based on eye movement and finger micro gestures |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114047872A (en) | 2022-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10593346B2 (en) | Rank-reduced token representation for automatic speech recognition | |
| EP3420442B1 (en) | System and method for multiple input management | |
| CN108089727B (en) | Handwriting keyboard for screen | |
| JP6140668B2 (en) | Multi-modal text input system for use with mobile phone touchscreen etc. | |
| US20180349346A1 (en) | Lattice-based techniques for providing spelling corrections | |
| US20150220265A1 (en) | Information processing device, information processing method, and program | |
| US20150261310A1 (en) | One-dimensional input system and method | |
| US20110063231A1 (en) | Method and Device for Data Input | |
| US12340083B2 (en) | Key function execution method and apparatus, device, and storage medium | |
| EP3465407B1 (en) | System, method and computer readable medium for input recognition linguistic resource management | |
| US20030234766A1 (en) | Virtual image display with virtual keyboard | |
| CN114047872B (en) | Text input method and system | |
| CN119156594A (en) | Predictive input interface with improved robustness to handling low accuracy inputs | |
| WO2021178255A1 (en) | Electronic input system | |
| US12026366B2 (en) | System and method for coarse and fine selection keyboard user interfaces | |
| Yamada et al. | One-handed character input method without screen cover for smart glasses that does not require visual confirmation of fingertip position | |
| Yamada et al. | A character input method for smart glasses that allows you to enter one character in one step with one thumb | |
| EP4254140A1 (en) | System and method for generating text data from limb motions | |
| Chang | Camera-Based Virtual Input | |
| CN113589958A (en) | Text input method, device, equipment and storage medium | |
| WO2017207103A1 (en) | System, method and computer readable medium for input recognition linguistic resource management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |