JP2000268045A - Information terminal equipment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide information terminal capable of reducing the operation burden of an operator in the construction of a data base to utilize voice recognition. SOLUTION: An evaluation point calculating part 38, a code book 16 and a field processing part 39 collate a label string provided by a vector quantizing part 36 with a grammer corresponding to the field of a designated voice input object and determine a field to store data (input data) expressed by that label string on the basis of this collated result. Grammars are previously applied to respective plural fields of a personal information managing(PIM) data base 20 and the grammars are provided as grammer data 18 containing symbols expressing the characteristics of fields and specified models expressing characters capable of being inputted to these fields.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information terminal device for recognizing a voice uttered by an operator and using the voice recognition result as input data to various databases, and more specifically to an information terminal device for managing personal information. The present invention relates to a portable personal information management (PIM) device.

[0002]

2. Description of the Related Art In recent years, voice recognition techniques such as dictation, spoken dialogue, and automatic speech translation have attracted attention. Some devices have been proposed which recognize voices emitted from an operator by using such voice recognition technology and use the voice recognition results substantially equivalently to input data obtained by, for example, a keyboard or a mouse. It has become For example, JP-A-6-3484
The computer system described in No. 52 converts speech into a command of a computer program, and the portable information terminal described in Japanese Patent Application Laid-Open No. Hei 8-227398 uses a voice to input event contents registered by a schedule management function. The system described in Japanese Patent Application Laid-Open No. H10-70613 creates name / address data at a high voice recognition rate when receiving a telephone order.

[0003] Speech recognition in such a conventional system presupposes the combined use of so-called keyboard or mouse operations, or simply replaces the command operation itself by such input means with speech input.
Improve operability of input using voice recognition,
There is a need to improve the user interface.

[0004] For example, among information terminal devices that have been rapidly spread in recent years, in particular, in portable devices, a housing and input means such as a keyboard or a mouse generally provided on the housing are configured to be small. There is a problem in that inputting a large amount of data by operating this for a long period of time imposes a heavy operation burden on the operator. Also, some types of information terminal devices do not have input means such as a keyboard, and transfer a database created by a personal computer or the like, and do not perform data input on the terminal. Alternatively, speech recognition on the assumption that a mouse or the like is used in combination cannot be applied.

In general, when the number of input data is large,
Cursor operation or the like for moving a field in creating a database is extremely troublesome, and such a problem of operation burden in data input applies not only to a portable information terminal device but also to other devices.

[0006]

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide an information terminal device capable of reducing the operation burden on an operator in constructing a database using voice recognition. .

[0007]

SUMMARY OF THE INVENTION In order to solve the above problems and achieve the object, the present invention is configured as follows.

[1] An information terminal device according to the present invention is a data terminal device for managing information by constructing a database storing a plurality of records each including at least a plurality of fields. Designating means for allowing an operator to designate a field to be input in advance before voice input; recognizing means for recognizing a voice emitted from the operator; and specifying means among grammars previously given to each of the plurality of fields Determining means for comparing the grammar corresponding to the field of the voice input target designated by the recognition result obtained by the recognition means with the grammar corresponding to the field, and determining a field for storing the recognition result based on the matching result; To the decision field in the same or different record Writing means for writing the input data to brute comprises a.

[2] The personal information management device of the present invention inputs a personal information database in which a plurality of records each including at least a plurality of fields are stored, and a plurality of fields related to predetermined records in the personal information database. Display means for displaying an input screen for inputting data, input means for inputting a voice uttered by an operator, and control means for controlling data input to the personal information database. A screen for displaying a screen, a designation unit for allowing an operator to designate in advance a field to be voice-inputted among the plurality of fields before voice input, a recognition unit for recognizing voice input from the input unit, and a recognition unit for the plurality of fields. Of the grammars given in advance to each, the voice input target Determining a field for storing the recognition result based on the matching result by comparing the recognition result obtained by the recognition means with a grammar corresponding to the same; A writing unit for writing input data based on the recognition result to the decision field, and a display control unit for displaying the input data on the input screen.

[3] A recording medium according to the present invention is a recording medium on which a program for managing information by constructing a database storing a plurality of records each having at least a plurality of fields is recorded. Designating means for allowing an operator to designate a field to be subjected to voice input from among a plurality of fields before voice input, recognition means for recognizing voice uttered from the operator, and pre-given to each of the plurality of fields The recognition result obtained by the recognition means is compared with the grammar corresponding to the field of the voice input target specified by the specification means in the grammar, and a field for storing the recognition result is determined based on the comparison result. To the decision field in the same or different record of the database It is obtained by recording a program for executing a writing means for writing the input data based on the recognition result.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS "Configuration" FIG. 1 is a block diagram showing a schematic configuration of an information terminal device according to an embodiment of the present invention. The information terminal device shown in the figure is a device that performs various database processes such as inputting information using a database, searching for desired information, extracting and displaying the information,
It comprises a control unit 1, a database 2, a display unit 3, a designation unit 4, and a voice input unit 5. In particular, this embodiment relates to the construction of the database 2, and more specifically, to data input to the database 2 based on voice recognition of an operator.

The database 2 stores a plurality of records each including at least a plurality of fields. The display means 3 is a means for displaying an input screen for inputting data to a plurality of fields of a predetermined record of the database 2, and is, for example, a display.
The designating means 4 is a means for allowing an operator to designate a field for data input before data input based on voice recognition, and includes, for example, a keyboard or a mouse. When the field for inputting the data is fixed, the specifying means 4 may be omitted.

The voice input means 5 is a means for inputting a voice uttered by an operator, and comprises, for example, a microphone. As will be described later, when a file in which a series of voice input data is recorded is used to collectively input data to the database 2, a means for reading voice input data from the file is provided instead of the voice input means 5. Can be

The control means 1 controls the display means 3, the designation means 4, and the voice input means 5 in connection with data input to the database 2, and includes a CPU. The control means 1 may be realized by software recorded on a computer-readable recording medium.

Hereinafter, data input to a database based on voice recognition by an operator will be described with reference to a more specific embodiment of the present invention.

FIG. 2 shows the present invention for personal information management (PIM).
It is a block diagram showing composition of an embodiment applied to an apparatus. The personal information management apparatus according to the present embodiment includes a main body 10, a microphone 12, an A / D converter 14, a codebook 16, a word HMM
(Hidden Markov Model) 18, P
It comprises an IM database 20, a display 22, a keyboard or mouse 24, and a control unit 30.

The control unit 30 provided in the main body 10 comprises a CPU (not shown), and an initial setting unit 32, a frequency analysis unit 34, a vector quantization unit 36, an evaluation point calculation unit provided as a program operating on the CPU. 38, and a field processing unit 39.

FIG. 3 is a diagram showing the external appearance of the main body of the personal information management device of the present embodiment.

The personal information management apparatus according to the present embodiment can manage four pieces of personal information of, for example, "To_Do_List (to-do list)", "schedule", "telephone directory", and "memo". There are four applications corresponding to each of these personal information. These applications are selectively activated from a specific menu screen displayed on the display 22 of the main body 10. I
1 to I4 are icons used by the operator to select and start a desired application from these applications, I1 is a “To_Do_List” application, I2 is a “schedule” application, and I3 is a “phonebook”. And the icon of the "memo" application. The activated application performs database processing on the PIM database 20. Also,
B1 to B4 are buttons for activating these applications other than on the display 22.
_Do_List "," Schedule "," Phonebook "," Memo "
Are provided corresponding to the respective applications. The PIM database 20 has a storage area for one or more records corresponding to each of “To_Do_List”, “Schedule”, “Phonebook”, and “Memo”. 1
One record includes one or a plurality of fields unique to each record type.

FIG. 4 is a diagram showing an example of a record input screen of the PIM database. The record input screen displayed on the display 22 includes a field name display and an area 40 for inputting / displaying data of the field.
And a (check box) area 42 for setting a voice input target field.

The initial setting section 32 shown in FIG. 2 is a means for performing the processing relating to the above-described application start-up and the processing relating to the setting of the voice input target field as the initial setting processing. The voice input target field setting is performed by designating a desired voice input target field by operating the keyboard or mouse 24 and performing an operation of checking an arbitrary box in the check box area 42 as shown in FIG. It is performed by FIG. 4 shows that the operator has “name”, “telephone number (home)”, “E_Mail”,
A case is shown in which a voice input is performed in the field of “address” and “address”.

The microphone 12 shown in FIG. 2 is a means for inputting a voice uttered by an operator. An audio signal obtained by the microphone 12 in an analog manner is converted into an A / D converter 14
, And supplied to the frequency analysis unit 34.

The frequency analysis unit 34 and the vector quantization unit 36 are recognition means based on the input audio signal.
The frequency analysis unit 34 is a means for analyzing the frequency of the audio signal and obtaining a frequency spectrum as the analysis result. The vector quantization unit 36 obtains a vector value of the frequency spectrum obtained by the frequency analysis unit 34, refers to the codebook 16,
This is means for performing vector quantization by expressing the elements of the vector collectively by one code.

The evaluation point calculation unit 38, the grammar data 18, and the field processing unit 39 compare the label sequence obtained by the vector quantization unit 36 with the grammar corresponding to the designated field of the voice input target, and perform the matching. Based on the result, it is a deciding means for deciding a field for storing data represented by the label string (that is, input data).

Each of the plurality of fields of the PIM database 20 is given a grammar in advance. The grammar is defined by a symbol representing a characteristic of the field and a specific model (here, a hidden Markov model) representing characters that can be input to the field. ) Is provided as grammar data 18. Table 1 below shows grammatical data according to the present embodiment as a schematic table. The grammars shown in Table 1 are not absolute, and the operator can arbitrarily edit (add, change, delete, etc.) by providing a specific editor.

[0026]

[Table 1]

The evaluation point calculation unit 38 includes the vector quantization unit 3
This is a means for evaluating the label sequence obtained in Step 6 using a hidden Markov model included in the grammar data 18 and calculating an evaluation point (likelihood).

Here, the expression of a word by a hidden Markov model (HMM) will be described.

When the word “SUM” is represented by an HMM, as shown in FIG. 5, a transition between three labels “s”, “u”, and “m” is represented by a finite state model represented by an arc. Become. The transition probability is calculated in advance using a voice database or the like. In this example, the label "s"
, The probability of stopping at the label “s” is 8
2%, probability of transition to label “z” is 5%, label “s”
The probability of transition to "h" is 3%, and the probability of transition to label "u" is 10%. Although omitted in FIG. 5, in a more practical model, about 2
There are 00 arcs. At the time of speech recognition using such a hidden Markov model, a plurality of word candidates are obtained using a label time series obtained from input speech, and a word having a maximum transition probability (likelihood) is recognized from among them. Result.

The evaluation point calculator 38 of the present embodiment
The label sequence obtained by the vector quantization unit 36 is evaluated by a hidden Markov model of the grammar data 18,
Calculate the evaluation point (likelihood).

The field processing section 39 includes an evaluation point calculation section 38
The field for storing the recognition result is determined on the basis of the evaluation points obtained in the step (1), the state transition process of the grammar is performed according to the grammar data 18, and the appropriate field in the same or different record of the PIM database 20 is determined based on the recognition result. This is a means for writing input data.

There are several methods of uttering the operator related to the input of voice data, from a method of uttering each word separately, to a so-called dictation that utters continuously with spoken words. In the present embodiment, no particular limitation is imposed on the utterance of the operator. However, in order to increase the recognition rate and prevent erroneous field determination, it is necessary to separate the utterance at least between the fields and between the first and last names in the "name". preferable.

Further, for example, when a field follows from "address" to "memo", it is difficult to determine the boundary of the field. In this case, it is preferable that the operator can directly designate a field by uttering a "memo", that is, a field name at the end of the voice input of the "address". Specifically, if you input "address: Hachioji-shi ..." followed by "memo: right next to the station,"
When you say "Josho ... Hachioji ... Memo ... Echinosugusoba", the input field is determined to be the highest priority in the "Memo" field when the field name "Memo" is recognized. To be configured. By doing so, a field determination error can be prevented, and data can be input more reliably. In addition, a predetermined reading of a field delimiter (“esc” in the example shown in Table 1) is uttered (for example, “escape”, “esc”).
Etc.), the fields may be forcibly separated. Further, as the field directly specified by the operator, not only the immediately following field as in this example, but also any field can be specified (field direct specification by voice command).

[Operation] FIG. 6 is a flowchart showing an operation relating to database construction in the personal information management apparatus according to the present embodiment.

[Step S1] First, the operator operates the icon I1 displayed on the display 22 as described above.
The user designates an application by operating .about.I4 or buttons B1 to B4. Initial setting unit 32 according to this designation
Starts the corresponding application. Thereby, the database to be processed is determined.

[Step S2] Next, as shown in FIG. 4, on the record input screen of the PIM database, an operator specifies a field to be subjected to voice input.

[Step S3] The voice data is acquired up to the break, and after the field processing accompanied by the voice recognition processing, the processing of storing the voice input data in the PIM database 20 is performed.

[Step S4] If there is an instruction from the operator to end the database construction process by voice recognition, the present process is terminated. Otherwise, the process returns to step S3 to continue the database construction process. In addition,
Although not described in detail here, for fields other than the voice input field designated in step S2,
Data input (for example, keyboard input) not based on voice recognition is performed in step S4 and subsequent steps.

FIG. 7 is a flowchart showing the flow of the voice recognition process in step S3.

[Step S21] The A / D converter 14 converts the audio signal obtained in an analog manner by the microphone 12 into a digital signal (for example, in PCM format).
Convert to The obtained digital signal is transmitted to the frequency analyzer 34.
Supplied to

[Step S22] The frequency analysis unit 34 analyzes the frequency of the digital audio signal obtained in step S21. A frequency spectrum is obtained as a result of the analysis.

[Step S23] Vector quantization section 36
Calculates the vector value of the frequency spectrum obtained by the frequency analysis unit 34, refers to the codebook 16, and collectively expresses the elements of the vector with one code to perform vector quantization. According to the vector quantization, the frequency spectrum is classified into about 100 types of groups, and can be represented by the names (label strings) of the groups.

[Step S24] The evaluation point calculator 38 calculates
The label sequence obtained by the vector quantization unit 36 is converted into a hidden Markov model (HMM) (for example, the “super”, “pharmacy”, and “bank” HM
M is indicated), and an evaluation point (likelihood) is calculated. More specifically, a word to be recognized (codebook 1
6 (stored in No. 6), the respective appearance probabilities are calculated, and the word having the highest probability is determined as a recognition result. In this case, a low evaluation score is regarded as a low probability that the recognition result is correct.

The voice recognition processing for obtaining such a recognition result is executed in the field processing described below.

FIG. 8 is a flowchart showing the flow of the field processing and the database writing processing in step S3.

[Step S31] The processing counter of a record relating to the database for which the construction processing is to be performed is I, the processing counter of the field is J, and initialization is performed by substituting 1 into these counters I and J. Also,
Loads the grammar of the first field among the specified fields for voice input. Here, since the operator does not always input the voice of the first field first, the loading is temporary.

[Step S32] The speech recognition process described with reference to FIG. 7 is started, and the recognition result of the speech uttered by the operator is obtained. In addition, without performing the voice recognition process in real time, a series of voice recognition results are obtained in advance,
This speech recognition result can be processed collectively (batch processing). In this case, a data file in which a series of speech recognition results is recorded is input.

[Step S33] In the case where there is an end instruction from the operator and no speech recognition result is obtained even after repeating the predetermined number of times or after the lapse of the predetermined time, or when performing the batch processing described above. If the end of the audio data file is detected, the operation of this flow ends.

[Step S34] An evaluation point (recognition score) of the recognition result obtained in step S33 is determined. If this evaluation point exceeds a predetermined threshold, step S35
The process proceeds to step S39 if it is below the threshold.

[Step S35] Since the recognition score in step S34 is high, the current field is determined as an input field, and grammatical transition in the field is started. The end of the grammar transition means the end of the field. The recognition result obtained in step S32 is written in the I-th record and the J-th field in the PIM database 20. FIG. 9 is a diagram illustrating an example of a grammar transition.

[Step S36] The grammar transition in the field is advanced. For example, in FIG. 9, when the recognition score of “PM” (or “AM”) is high in the state of st1, the state transits to st2.

[Step S37] The end of the field is determined based on the state transition of the grammar. If it is in the middle of the field, the process proceeds to step S32, and the input of the field is continued. If the field has ended, the process moves to step S38.

[Step S38] The processing counter J in the field is incremented by one. That is, assuming the following fields,
The grammar is loaded, and the operation from step S32 is repeated. Note that the counter J has the highest value (J_MA
If X) is taken, it is determined that the field is the last field in the record, the processing counter I of the record is incremented by 1, and the processing counter J of the field is initialized. That is, the next record is processed.

[Step S39] If the recognition score falls below the threshold value in step S34, a field having a higher recognition score is searched for a field near (3 to 5 fields ahead) from the current field.

[Step S40] If the search is successful and a field with a high recognition score is found, then step S40 is performed.
It moves to 41. On the other hand, when a field having a high recognition score cannot be found, the recognition result is stored in any one of the fields even though the recognition score is low, and a mark is given to the field to make it identifiable. Alternatively, a message that the field cannot be determined may be displayed on the display 22, and the operator may select a field to be stored.

[Step S41] The field processing counter J is incremented by one, and a grammar corresponding to the counter J is newly loaded.

[Step S42] The current field is determined as an input field, and grammatical transition in the field is started. The recognition result obtained in step S32 is written in the I-th record and the J-th field in the PIM database 20.

In steps S35 and S42, a field for storing a speech recognition result is determined, and the recognition result is converted into input data including character data and symbol data. This input data is displayed on display 2
2, the input data can be edited by the operator at this point. After the appropriate editing, the input data is written in an appropriate field of the PIM database 20 according to an instruction from the operator.

According to the above-described embodiment, the speech recognition result is compared with the grammar representing the characteristics of the field,
It is configured to determine the field for storing the input data represented by the recognition result based on the collation result, and it is not necessary to perform an operation such as moving the field with a keyboard or the like.
In other words, the operator pre-specifies the field for voice input and simply utters the input data in order, eliminating the need for an operation for moving the field.

Therefore, when constructing a database in which the number of fields and the number of records are enormous, the operation burden on the operator can be greatly reduced.

The present invention is not limited to the above-described embodiment, but can be implemented with various modifications. For example, in the above-described embodiment, the present invention is applied to a personal information management device. However, the present invention is applied to so-called personal computers (PCs), workstations, and various other information terminal devices, and the present invention is applied. Is also good.

Further, for example, when the postal code or the telephone number of the telephone number is inputted in the input of the "address", the candidates such as the place name can be narrowed down based on the information.
Therefore, by additionally providing a postal code-place name or a station number-place name database, it is possible to prevent erroneous characters in selecting a kanji for the place name.

[0063]

As described above, according to the present invention,
It is possible to provide an information terminal device capable of reducing an operator's operation burden in constructing a database using voice recognition.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of an information terminal device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of an embodiment in which the present invention is applied to a personal information management (PIM) device.

FIG. 3 is an exemplary view showing the appearance of the main body of the personal information management apparatus according to the embodiment;

FIG. 4 is a diagram showing an example of a record input screen of a PIM database

FIG. 5 is a view for explaining word expression by a hidden Markov model (HMM).

FIG. 6 is an exemplary flowchart showing an operation related to database construction in the personal information management device according to the embodiment;

FIG. 7 is a flowchart showing the flow of a voice recognition process;

FIG. 8 is a flowchart showing the flow of field processing and database writing processing.

FIG. 9 is a diagram showing an example of a grammar transition

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Control means 2 ... Database 3 ... Display means 4 ... Designation means 5 ... Speech input means 10 ... Main body 12 ... Microphone 14 ... A / D converter 16 ... Code book 18 ... Grammar data 20 ... PIM database 22 ... Display 24 ... Keyboard or mouse

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G10L 15/18 G06F 15/40 310G 15/00 G10L 3/00 535A 15/22 537Z // G06F 15/02 325 551B 561C

Claims (7)

[Claims] 1. An information terminal device for constructing a database storing a plurality of records each including at least a plurality of fields and managing information, wherein a field to be voice-input among a plurality of fields of the record is input before voice input. Designating means for allowing an operator to designate in advance; recognizing means for recognizing a voice emitted from the operator; and a grammar previously given to each of the plurality of fields; Determining a field for storing the recognition result based on the matching result by comparing the recognition result obtained by the recognition means with the grammar corresponding to the grammar; and determining the field in the same or different record of the database. Writing input data to the field based on the recognition result above The information terminal apparatus characterized by comprising writing means for performing a. 2. A method according to claim 1, wherein a grammar previously given to each of the plurality of fields is used for recognizing voices corresponding to the characteristics of the field and symbols, characters, and character strings that can be input to the field according to the characteristics. A storage unit for storing as a grammar data representing the specific model; and a reading unit for reading the grammar data from the storage unit, wherein the determination unit determines a symbol read by the reading unit. The information terminal device according to claim 2, wherein the grammar is compared with the recognition result based on a state transition of the grammar based on the grammar. 3. The recognition result by the recognition means is a label sequence obtained by frequency-analyzing the voice of the operator and vector-quantizing a spectrum obtained as the analysis result, and the specific model is a hidden Markov model. 3. The method according to claim 2, wherein the determining unit performs the matching based on the likelihood obtained by evaluating the label sequence using a hidden Markov model.
An information terminal device according to item 1. 4. The information terminal device according to claim 2, further comprising changing means for changing contents of grammatical data in said storage means. 5. A recognition result storage means for storing a series of speech recognition results obtained by said recognition means, reading a series of speech recognition results stored in said storage means, and said determination relating to said read recognition results. 5. The information terminal device according to claim 1, further comprising: a collective control unit that collectively determines a field by the unit and writes input data by the writing unit. 6. 6. A personal information database storing a plurality of records each including at least a plurality of fields, and display means for displaying an input screen for inputting data to a plurality of fields related to predetermined records of the personal information database. And input means for inputting a voice uttered by an operator; and control means for controlling data input to the personal information database. The input means displays the input screen on the display means, and displays the plurality of fields. Designating means for allowing an operator to designate a field to be voice-inputted before voice input; recognition means for recognizing voice input from the input means; and a grammar previously given to each of the plurality of fields. The grammar corresponding to the field of the voice input target specified by the specifying means is added to the recognition Determining means for comparing the recognition result obtained by the means and determining a field for storing the recognition result based on the matching result; and performing the recognition on the determination field in the same or different record of the personal information database. A personal information management apparatus comprising: a writing unit that writes input data based on a result; and a control unit that includes a display control unit that displays the input data on the input screen. 7. A recording medium storing a program for managing information by constructing a database in which a plurality of records each including at least a plurality of fields are stored, wherein a field to be subjected to voice input among a plurality of fields of the record Means for designating an operator in advance before inputting a voice, a recognition means for recognizing a voice emitted from the operator, and a grammar previously given to each of the plurality of fields specified by the specification means Determining means for comparing a recognition result obtained by the recognition means with a grammar corresponding to a field to be voice-input, and determining a field for storing the recognition result based on the matching result; Based on the above recognition result in the above decision field in the record Recording medium for recording a program for executing a writing means for writing the input data.