JP2012098891A - Information processing system and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing system and an information processing method for easily inputting characters in a tablet type computer.SOLUTION: An information processing method of the present invention includes: a step (a) for accepting an input of handwritten characters by a user; a step (b) for identifying characters by recognizing the characters on the accepted handwritten characters; a step (c) for displaying the identified characters in a second display area provided by note application software; a step (d) for generating text characters corresponding to the identified characters; and a step (e) for storing the characters displayed in the display area and the generated text characters. A tablet type computer is configured to display text expressions including conversion candidates of the accepted handwritten characters and the accepted handwritten characters as prediction conversion candidates in a first display area of the tablet type computer in response to the character recognition, and to identify the characters in response to the selection of one of the displayed prediction conversion candidates.

Description

  The present invention relates to an information processing system and an information processing method that facilitate character input. More specifically, the present invention relates to an information processing system and an information processing method that facilitate character input in notebook application software executed on a tablet computer.

  In recent years, tablet computers have spread and people's lifestyles have changed greatly. For example, a user of a tablet computer transmits and receives electronic mail, accesses the Internet, searches various information, reads an electronic book, and watches video and music. The user can perform these actions anywhere (eg, at home, in the office, or in the city).

  Some tablet computers have a touch panel display with few switches and buttons. The user can perform most operations by touching the display with a finger and / or stylus pen. The tablet computer can be connected to a network via a cellular network or a wireless LAN.

  A user can use various application software on a tablet computer. These application software can be downloaded via a network. Application software includes business applications that can be used for work.

  Notebook application software is known as a business application. The notebook application software is used whenever a user takes notes on a notebook. That is, the user can use the software with the same feeling as a paper notebook.

  The notebook application software provides multiple input interfaces. For example, the plurality of input interfaces include a handwriting input interface using a stylus pen and an input interface using a keyboard displayed on a display. It is known that an input interface using a keyboard is provided as a standard function by a tablet computer. There are two types of handwriting input interfaces. One type is an input interface for recognizing handwritten input and displaying it after replacing it with a corresponding character, and the other type is an input interface for displaying handwritten input as it is.

  Although many improvements have been made for handwritten character recognition (see Patent Documents 1 and 2), the need for improving the usability of notebook application software is increasing.

JP 2003-308484 A JP-A-6-332610

  When a user imagines taking notes on a paper notebook, there are scenes using a plurality of languages (for example, Japanese, English, Chinese, etc.) and scenes using figures and tables. When using the note application software in these situations, the user wants the same or better usability as taking notes on paper notes.

  However, the conventional notebook application software in a tablet computer has not provided a user-friendly usability. For example, there are drawbacks of poor character recognition accuracy, insufficient input support function, and inflexible input (cannot be drawn at a desired position).

  The present invention has been made in view of the above problems, and an object thereof is to provide an information processing system and an information processing method that facilitate character input in notebook application software executed on a tablet computer. is there.

  In order to solve the above-described problems, the present invention provides an information processing method for facilitating character input used in notebook application software executed on a tablet computer. The method executed by the tablet computer includes: (a) receiving a handwritten character input by a user; and (b) performing character recognition on the received handwritten character to identify a character, the tablet type In response to the character recognition, the computer uses one or more accepted handwritten character conversion candidates and a sentence expression including the accepted handwritten characters as a predicted conversion candidate in the first display area of the tablet computer. Displaying and identifying the character in response to selection of one of the displayed predictive conversion candidates, and (c) a second provided by the note application software for the identified character. Displaying on the display area; and (d) corresponding to the identified character. Features and generating a text, to include a step of storing (e) displayed in the display area character and the generated text characters.

  According to the present invention, it is possible to provide an information processing system and an information processing method that facilitate character input in notebook application software executed on a tablet computer.

1 is a system configuration diagram of a tablet computer according to the present invention. It is a figure which shows the external view of note application software. It is a figure which shows the keyboard input interface which concerns on this invention. It is a figure which shows the keyboard input interface which concerns on this invention. It is a figure which shows the 1st handwriting input interface which concerns on this invention. It is a figure which shows the 1st handwriting input interface which concerns on this invention. It is a figure which shows the 1st handwriting input interface which concerns on this invention. It is a figure which shows the 1st handwriting input interface which concerns on this invention. It is a figure which shows the 2nd handwriting input interface which concerns on this invention. It is a figure which shows the 2nd handwriting input interface which concerns on this invention. It is a figure which shows the program module of the note application software performed in a tablet type computer. It is a flowchart of the character input through the keyboard input interface which concerns on this invention. It is a flowchart of the character input via the 1st handwriting input interface which concerns on this invention. It is a flowchart of the character input via the 2nd handwriting input interface which concerns on this invention.

  Hereinafter, exemplary embodiments of the present invention will be described in detail.

(System configuration)
FIG. 1 is a system configuration diagram of a tablet computer 100 on which notebook application software is executed.

  The tablet computer 100 includes a control unit 101, a main storage unit 102, an auxiliary storage unit 103, an interface (I / F) unit 104, a display unit 105, a recognition result DB 106, a dictionary DB 107, and an individual history DB 108. These components 101 to 108 are connected to each other via a bus 120, and the components 102 to 108 are collectively controlled by the control unit 101. The tablet computer 100 can receive user input via the stylus pen 160 or a finger. In FIG. 1, the stylus pen 160 is indicated by a dashed arrow to indicate that it is not connected to the tablet computer 100 by wire.

  The control unit 101 is also called a central processing unit (CPU), and controls each component 102 to 108 and calculates data. The control unit 101 reads out various program modules stored in the auxiliary storage unit 103 to the main storage unit 102 and executes them. The main storage unit 102 is also called a main memory, and stores various data received by the tablet computer 100, computer-executable instructions, data after arithmetic processing by the instructions, and the like.

  The auxiliary storage unit 103 is a storage device such as a hard disk (HDD) and is used for storing data and programs for a long period of time. Since the main storage unit 102 has a relatively smaller storage capacity than the auxiliary storage unit 103, it is used for temporary data storage, arithmetic processing, and the like, whereas the auxiliary storage unit 103 requires necessary data and information. Used for long-term memory. That is, when the control unit 101 executes a program to perform data calculation, the necessary data or program is read from the auxiliary storage unit 103 to the main storage unit 102 and the calculation result data is stored for a long time. The control unit 101 writes calculation result data in the auxiliary storage unit 103.

  The interface (I / F) unit 104 serves as an interface for transmitting and receiving data to and from the stylus pen 160 for inputting handwritten characters. Data received from the outside of the tablet computer 100 is temporarily stored in the main storage unit 102. The interface unit 104 provides an interface (such as an application program) that accepts various commands and input data from the user to the display unit 105. The display unit 105 displays data processed by the tablet computer 100 and the interface.

  The recognition result DB 106 stores user-input handwriting information (stroke data) recognized by the tablet computer 100 and one or more character string information (text data) combined with the handwriting information. A set of stroke data and text data is associated with attribute information (note identifier, page number, line number, position, etc.).

  The dictionary DB 107 stores dictionaries of a plurality of languages and idiomatic expressions used in the notebook application software. The plurality of languages include a plurality of languages (Japanese, English, Chinese, Korean, etc.) in which characters are written from left to right, and are not limited to specific languages. These dictionaries may be built in the tablet computer 100 in advance or may be downloaded via a network. In addition, these dictionaries hold management information and the like that can be used by applications running on the tablet computer 100 as metadata (for example, information on the priority order according to the usage frequency). By using this management information, the tablet computer 100 can rank words and expressions and present them to the user according to the ranking.

  The personal history DB 108 is a database that stores the handwriting information (stroke data) input previously, the corresponding text-based characters, and the usage history of the dictionary and the idiomatic expressions stored in the dictionary DB 107 for each user. Use of this database can lead to improved accuracy of character recognition. That is, the notebook application software can quickly and appropriately provide the characters intended by the user based on the handwriting characteristics of each user.

  As the stylus pen 160, any known stylus pen can be used as long as it can draw characters in the input field provided by the notebook application software.

  FIG. 6 shows a program module of notebook application software executed in the tablet computer 100. The program module is executed on the main storage unit 102 by the control unit 101 in response to a command (input) from the user or a specific occurrence event. This program module includes a handwritten character recognition module 601, a text search module 602, a network service module 603, a predictive conversion module 604, and an encryption module 605.

  The handwritten character recognition module 601 estimates the candidate of the character having the highest likelihood from the dictionary DB 107 based on handwriting information (including input timing information and stroke start position information) of the handwritten character input by the user, The estimated character is displayed in a predetermined area of the screen. If the user determines that the estimated character candidate is erroneous, in response to the user request, the handwritten character recognition module 601 selects one or more candidates with a relatively high likelihood. Estimated from the dictionary DB 107 and displayed.

  The text search module 602 provides a function of searching text-based character string data stored in the recognition result DB 106. Handwritten characters input by the user are stored in the recognition result DB 106 as a set of handwriting information (stroke data) and one or more corresponding text-based character string data. The tablet computer 100 can use the text search module 217 to search for one or more character string data.

  The network service module 603 has an electronic mail transmission / reception function, an SNS access function, and a function for downloading and storing information existing on a network such as the Internet, and uploading and sharing various data to the network I will provide a. The tablet computer 100 can transmit notes (that is, files) created by the user to the network via the network service module 218, and can use services by various applications provided from the network.

  The predictive conversion module 604 has a function of predicting and presenting a commonly used sentence expression (that is, a subsequent character string) including a conversion candidate of the character and the input character corresponding to the character input by the user. provide. This function provides, for example, a post-conversion word or a subsequent character string that is expected when a part of characters is input when the characters are input. The predictive conversion module 604 performs predictive conversion using data stored in the dictionary DB 107.

  The encryption module 605 is a module that can encrypt each note (that is, file) created using the note application software. This notebook application uses a well-known encryption technique. By encrypting each notebook, it is possible to increase security when the notebook is transmitted to another user via the network and when the tablet computer 100 is in the hands of a third party.

(Overview of note application)
FIG. 2 shows an overview of the notebook application software. This notebook application can handle characters, figures, etc. that are input as if a user took notes on a paper notebook.

  Reference numeral 201 represents various modes. Various modes 201 include a handwriting input mode, an erase mode, and a keyboard input mode. The handwriting input mode is an input mode using a stylus pen or a finger. The erase mode is a mode for erasing characters / graphics drawn in the notebook application. The keyboard input mode is a mode for displaying a keyboard corresponding to various languages and receiving an input from the user.

  Reference numeral 202 indicates an input field in which input characters are displayed. The input field 202 can be moved to a desired location in the note using a stylus pen or finger, and the user can display input characters at the moved location.

  Reference numeral 203 indicates an input interface. The input interface 203 includes a handwriting input interface and a keyboard input interface. The handwriting input interface is an interface that can be input via stylus pen or finger input. The keyboard input interface is an interface that can be input via an input keyboard.

  The input field 202 displays characters (stroke data) input via the input interface 203. The characters displayed in the input field 202 can be bitmap data obtained by Bezier interpolation of the stroke data. The stroke data means one of data formats for expressing the shape of a character on a computer, and handwriting information is expressed by an array of points (x, y). Then, bitmap data is obtained by obtaining a Bezier curve based on this point information and performing interpolation.

  Note that the user can directly draw characters and figures on the notebook displayed on the display without using the input interface 203. When using a paper notebook, the user may draw a figure as well as a letter. Therefore, this notebook application software also has a function that allows the user to draw directly. The directly drawn figure is displayed as bitmap data, and the corresponding stroke data is stored in the recognition result DB 106. Since a figure or the like may not include text data for search, the text data may not be saved.

(Input interface)
The input interface usable in the notebook application software will be described below with reference to FIGS.

  FIG. 3 shows an example of a known keyboard input interface that is mounted on the tablet computer 100 as a standard. FIG. 3A shows an example of a keyboard for inputting alphabets and symbols, and FIG. 3B shows an example of a keyboard for inputting kana characters and symbols. This notebook application software supports multiple languages (Japanese, English, Chinese, Korean, etc.) that write characters from left to right. For example, when inputting Chinese, a user inputs pinyin which is a pronunciation of Chinese using the keyboard shown in FIG. 3A, and selects a desired Chinese character. When inputting Korean, the user uses another keyboard to select consonant characters and vowel characters.

  Characters input using the keyboard shown in FIGS. 3A and 3B are displayed in the input field 202 as bitmap data obtained by Bezier interpolation of stroke data. The tablet computer 100 recognizes characters displayed in the input field 202 and identifies corresponding text characters. The tablet computer 100 stores the stroke data corresponding to the characters displayed in the input field 202 and the corresponding text characters in the recognition result DB 106. The text characters stored in the recognition result DB 106 are used when a user searches for a desired sentence.

  Next, two types of handwriting input interfaces will be described with reference to FIGS.

  FIG. 4 shows a first handwriting input interface. FIG. 4A includes an input field 202, a first input area 401, and a first function button 402. In the first input area 401, the area is divided character by character, and accepts an input with a stylus pen or a finger. The first function button 402 has various functions used in the first handwriting input interface (for example, character deletion, numeric and symbol input keyboard call, line feed, input mode switching, space input, and other keyboard call). Provides a button to execute.

  Characters input to the first input area 401 are recognized by the handwritten character recognition module 601 and replaced with characters having the highest likelihood. FIG. 4B shows an example of a display screen in which characters are input in the first input area 401. A conversion candidate 403 indicates a prediction conversion candidate corresponding to the character input in the first input area 401. FIG. 4C shows another example of the display screen in which characters are input in the first input area 401. In this example, the character “DESIGN” is input to the first input area 401, and the candidate for prediction conversion corresponding to the input character is shown in the conversion candidate 403.

  More specifically, the character with the highest likelihood is formed as stroke data. Bitmap data is created by performing Bezier interpolation based on the stroke data, and the bitmap data is displayed in the input field 202. The stroke data is not displayed in the input field 202 but is temporarily stored in the main storage unit 102 and used when stored in the recognition result DB 106. When the characters (bitmap data) displayed in the input field 202 are deleted, the stroke data stored in the main storage unit 102 is also deleted.

  Here, the “predictive conversion candidate” refers to a frequently used sentence expression including the converted character of the input character and the input character. The tablet computer 100 performs character recognition of characters input using the handwritten character recognition module 601 and selects prediction conversion candidates for characters recognized using the dictionary DB 107.

  The language of predictive conversion depends on the language stored in the dictionary DB 107. In this specification, examples of Japanese and English have been described, but other languages can be implemented. The languages that can be implemented are languages that write characters from left to right (eg, Chinese, Korean, etc.).

  FIG. 4D shows an example of an input interface that corrects an error when there is an error in the character input in the input area 401. When there is an error, the correction candidate list 404 is displayed when the user touches an area including the error. In this case, the handwritten character recognition module 601 displays a plurality of candidate characters corresponding to the characters input by the user.

  FIG. 5 shows a second handwriting input interface. FIG. 5A includes a second input area 501, a second function button 502, and an input field 202. Unlike the first input area 401, the second input area 501 is not divided as an area for each character, and accepts input characters as they are. The accepted characters are displayed in the input field 202 as bitmap data obtained by Bezier interpolation of the stroke data. In addition, the tablet computer 100 separates characters input to the second input area 501 one by one based on information such as a space between characters, and uses the handwritten character recognition module 601 to characterize each character. Recognize and identify the corresponding text characters. The identified text characters are temporarily stored in the main storage unit 102 and stored in the recognition result DB 106 together with the stroke data at a predetermined timing. As described with reference to FIG. 4, the bitmap data is displayed in the input field 202, and the corresponding stroke data is stored in the main storage unit 102.

  A second function button 502 is a button for executing various functions used in the second handwriting input interface (for example, deletion of characters, movement of input fields, line feed, and other keyboard calls). FIG. 5B shows an example of the display screen after the line feed button is selected via the second function button 502. The input field 202 has moved to the next line of the character input immediately before, and nothing is input in the second input area 501.

(Processing flow)
FIG. 7 is a flowchart of a process for inputting characters using the known keyboard input interface of FIG. In step S701, a desired keyboard input interface is selected. For example, the keyboard for entering alphabets and symbols of FIG. 3 (a) is selected, and alternatively, the keyboard for entering kana characters and symbols of FIG. 3 (b) is selected.

  In step S702, character input is performed using the selected keyboard. In step S703, the input character is displayed as bitmap data in the input field 202. In step S704, the stroke data corresponding to the characters displayed in the input field 202 and the corresponding text data are stored in the recognition result DB 106 when this processing flow ends. In this case, the recognition result DB 106 stores attribute information such as a note identifier, page number, line number, and position, stroke data, and search text characters.

  FIG. 8 is a flowchart of a process of inputting characters using the first handwriting input interface of FIG. This processing flow is repeated until the character input is completed.

  In step S <b> 801, characters are input to an area that is divided character by character in the first input area 401. For example, characters are input as shown in FIG. In step S802, the tablet computer 100 performs character recognition using the handwritten character recognition module 601 and replaces the input character with the character having the highest likelihood.

  In step S803, the tablet computer 100 refers to the dictionary DB 107 and performs predictive conversion based on the replaced character. Specifically, as shown in FIGS. 4B and 4C, the tablet computer 100 predicts frequently used sentence expressions including conversion candidates for the replaced characters and the characters, and displays them on the display. indicate.

  In step S804, when one character string is selected from the conversion candidates and sentence expressions displayed by the predictive conversion, the tablet computer 100 displays the selected character string in the input field 202. The tablet computer 100 also identifies a text-based character string corresponding to the character string displayed in the input field 202 and temporarily stores it in the main storage unit 102. Thereafter, the process returns to step S801 again.

  Note that when this processing flow ends, the stroke data corresponding to the characters displayed in the input field 202 and the text-based characters stored in the main storage unit 102 are stored in the recognition result DB 106 as described above. This is the same as the embodiment. In this case, the recognition result DB 106 stores attribute information such as a note identifier, page number, line number, and position together.

  FIG. 9 is a flowchart of a process of inputting characters using the second handwriting input interface of FIG.

  In step S901, a character is input to the second input area 501. In step S902, the tablet computer 100 uses the handwritten character recognition module 601 to estimate a character having the highest likelihood based on the input character. When this estimation is performed, the tablet computer 100 separates characters input to the second input area 501 one by one based on information such as a space between characters, and uses the handwritten character recognition module 601. Character recognition is performed for each character, and the corresponding text character is identified. The estimated characters are temporarily stored in the main storage unit 102 in a text format.

  In step S903, the tablet computer 100 displays the input characters in the input field 202 as bitmap data. In step S904, the tablet computer 100 stores the stroke data corresponding to the characters displayed in the input field 202 and the text data of the characters stored in the main storage unit 102 in the recognition result DB 106. In this case, the recognition result DB 106 stores attribute information such as a note identifier, page number, line number, and position together.

(Other embodiments)
When character recognition is performed in step S802 of FIG. 8 and step S902 of FIG. 9, the tablet computer 100 searches the personal history DB 108 to identify a character corresponding to the handwriting information of the input character. it can. The personal history DB 108 stores previously input handwriting information (stroke data) of a user and a set of corresponding characters. That is, at the time of storing in the recognition result DB 106 described above, data is also stored in the individual history DB 108. For this reason, the tablet computer 100 can perform character recognition with higher accuracy. When the tablet computer 100 collates handwriting information while referring to the individual history DB 108, the tablet computer 100 uses the handwriting information input by the user as necessary with reference to the handwriting information in the individual history DB 108. The collation is performed while enlarging or reducing, or by gradually shifting.

(Summary)
The methods, functions, and means described in the above-described embodiments can be implemented as a computer-executable program, or can be implemented as a computer-readable recording medium that stores the program.

  The embodiment described above is merely an example, and the present invention is not limited to the above embodiment, and various modifications are possible. That is, as long as it is based on the technical idea described in the scope of claims, the modifications are included in the technical scope of the present invention.

100 tablet computer 106 recognition result DB
107 Dictionary DB
108 Personal history DB
160 Stylus Pen 201 Various Modes 202 Input Field 203 Input Interface 401 First Input Area 402 First Function Button 403 Conversion Candidate 404 List of Correction Candidates 501 Second Input Area 502 Second Function Button 601 Handwritten Character Recognition Module 602 Text search module 603 Network service module 604 Predictive conversion module 605 Encrypted speech recognition module

Claims (1)

An information processing method for facilitating character input used in notebook application software executed on a tablet computer,
The method performed by the tablet computer includes:
(A) receiving a handwritten character input by a user;
(B) performing character recognition on the accepted handwritten character and identifying the character, wherein the tablet computer responds to the character recognition and one or more of the accepted handwritten character conversion candidates And the sentence expression including the accepted handwritten character is displayed as a predictive conversion candidate in the first display area of the tablet computer, and the character is selected in response to selection of one of the displayed predictive conversion candidates. Identify, step,
(C) displaying the identified characters in a second display area provided by the notebook application software;
(D) generating a text character corresponding to the identified character;
(E) storing the characters displayed in the display area and the generated text characters.

Images