JPH0981557A - Natural language processing device and natural language processing method - Google Patents

Abstract

(57) [Abstract] [PROBLEMS] To easily know whether an output natural language expression is known as co-occurrence. In the kana-kanji conversion processing, for example, the edit control unit 102 checks whether or not the word group held in the word string candidate holding unit 108 matches the co-occurrence information. When the word group in the language information does not match or does not match the co-occurrence information, the edit control unit 102 attaches co-occurrence presence / absence information (for example, underline) indicating that fact to the candidate character string and displays it. Output to 103. As a result, whether the expression is determined to be known as a co-occurrence based on the co-occurrence information attached to the character string output to the display unit 103, or
You can easily know such expressions without accumulation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a natural language processing apparatus for creating, editing or displaying a document in natural language, and more particularly to performing natural language processing such as kana-kanji conversion using co-occurrence information between words. The present invention relates to a natural language processing device and a natural language processing method.

[0002]

2. Description of the Related Art In natural language processing, co-occurrence information is used as one of the measures for evaluating the certainty of natural language expressions. Co-occurrence information is information that links words that are semantically related.

For example, in a kana-kanji conversion device used in a Japanese word processor or the like, a large number of word string candidates are generated from an inputted kana character string as a conversion conversion result. Therefore, these candidates are provided with grammatical information such as a part of speech. After narrowing down by using the method, a method of referring to the co-occurrence information and determining the priority is used.

In such natural language processing, a method is used in which a word pair based on the input natural language information is compared with pre-stored co-occurrence information and the subsequent processing is switched depending on the presence or absence of the co-occurrence information. There is.

However, as a result of comparison between the word pair and the co-occurrence information,
In other words, the information about the presence or absence of co-occurrence information of a word pair to be processed is information that can be directly used only on the system side,
This information could not be directly referenced by the user.

[0006]

As described above, in natural language processing using co-occurrence information, it is known to the user that a given expression in the output natural language expressions is known as co-occurrence. No means for directly and simply referring to information such as a judged expression or an expression without such accumulation has been provided. Therefore, the user cannot sufficiently use the co-occurrence information when evaluating the output natural language expression.

The present invention has been made in view of the above points, and a natural language processing device and a natural language processing method capable of easily knowing whether or not an output natural language expression is known as co-occurrence. The purpose is to provide.

[0008]

According to the present invention, a holding means for holding language information constituting a natural language expression obtained by converting an input, and a word group in the language information held by the holding means are provided. A co-occurrence determination means for determining whether or not the co-occurrence information matches, and if the co-occurrence determination means determines that the word group in the language information does not match or coincides with the co-occurrence information, it notifies that fact. An output unit for outputting the co-occurrence presence / absence information attached to a natural language expression is output.

According to this structure, it is examined whether or not the word group in the natural language expression obtained by converting the input matches the co-occurrence information. Then, when the co-occurrence information does not match or when the co-occurrence information does match, co-occurrence presence / absence information for notifying that effect is added to the natural language expression and output.

Therefore, the user says that the expression is judged to be known as co-occurrence from the output co-occurrence information given to the output natural language expression, or an expression without such accumulation. You can easily know that. As a result, the output natural language expression can be understood, and the quality of the document at the time of editing and the efficiency of document creation can be improved.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings. Here, a Kana-Kanji conversion device will be described as an example of a natural language processing device that performs natural language processing using co-occurrence information.

FIG. 1 is a block diagram showing a schematic configuration of a kana-kanji conversion device according to an embodiment of the present invention. This kana-kanji conversion device is composed of a kind of information processing device such as a computer, and has a plurality of hardware parts such as an input part, a display part and a storage part, and a plurality of software parts constructed as software on a program. It consists of and.

That is, the kana-kanji conversion device is roughly divided into an input unit 101, an edit control unit 102, and a display unit 10.
3, a word search unit 104, a phrase candidate generation unit 105, a phrase candidate selection unit 106, a connection table 107, a word string candidate holding unit 108, a word dictionary 109, and a co-occurrence dictionary 110.

The kana-kanji conversion process in the kana-kanji conversion device having such a configuration is roughly divided into two processes. The first process is a process for recognizing the range of a phrase for an input kana character string by using information on the connectivity between independent words and adjuncts, information on the connectivity between adjuncts and adjuncts, and the like.

This process corresponds to the process in the input unit 101, the word search unit 104, and the phrase candidate generation unit 105 in FIG. Since this process is described in detail in Japanese Patent Laid-Open No. 3-265061, it will be briefly described here.

First, the kana information to be converted is input from the input unit 101 and sequentially sent to the word search unit 104 through the edit control unit 102. In the word search unit 104,
Word candidates are extracted by referring to the word dictionary 109. The word candidates extracted by the word search unit 104 are sent to the phrase candidate generation unit 105. The phrase candidate generating unit 105 refers to the connection table 107 to generate a phrase candidate from a plurality of word candidates, and sends the result to the phrase candidate selecting unit 106. The connection table 107 stores connection information of independent words and adjunct words, and adjunct words and adjunct words. Phrase candidate generation unit 10
The result of the process of 5 is stored in the word string candidate holding unit 108.

The second process is the 1 generated in the first process.
This is a process of determining a first candidate from a plurality of word string candidates. The phrase candidate selection unit 106 selects the first word string candidate from the first.
Select a candidate. 1 selected by the phrase candidate selection unit 106
The individual word string candidates are sent to the editing control unit 102. The edit control unit 102 is supplied with a result of Kana-Kanji conversion, a Kana string before conversion, a list of other candidates for homonyms, and the like, which temporarily stores the character string to be output and displayed. Etc. are sent to the display unit 103. The editing control unit 102 also receives from the input unit 101 inputs related to normal editing operations such as cursor movement, deletion of character strings, and selection of homonyms, and performs predetermined editing-related processing.

As shown in FIG. 2, a word number, reading information, heading information, part-of-speech information, and feature information are set in the word dictionary 109 for each word. For example,
For the word "opposition", the word number is "00
1 ”, the reading information is“ Igi ”, the heading information is“ object ”, the part-of-speech information is“ noun ”, and the feature information is“ suppress ”.
The same applies to other words.

As shown in FIG. 3, the co-occurrence dictionary 110 stores word numbers of co-occurring words, word numbers of co-occurring words, and relationship information indicating a co-occurrence relationship between two words. Remembered The word number is the word dictionary 10 shown in FIG.
It is a word number set to 9. For example, information is registered that "objection" of "001" and "chant" of "005" are used in a form of "protest" with a particle "wa" sandwiched between them. The same applies to other co-occurrence information.

The reference processing of the co-occurrence dictionary 110 and the comparison processing of the co-occurrence information and the phrase sequence candidate are included in the phrase sequence candidate selection unit 106 here. No. 265061, Japanese Patent Application No. 5-52
The method described in Japanese Patent No. 245 publication can be used.

The language information forming the natural language expression is the phrase sequence candidate held in the word string candidate holding unit 108 here. Next, the processing of the edit control unit 102 according to the first embodiment will be described.

The edit control unit 102 includes a phrase sequence candidate selection unit 1
When the first candidate determination process in 06 is completed, the result is sent to the display unit 103. At that time, the editing control unit 102 determines that the co-occurrence information matches based on the comparison result of each word of the word string candidates of the word string candidate holding unit 108 obtained by the phrase string candidate selecting unit 106 and the co-occurrence information. The display unit 1 is provided with the highlighting instruction information for displaying the red underline on the non-displayed portion.
Output to 03.

More specifically, the word dictionary 109 will now be described.
It is assumed that the dictionary information as shown in FIG. 4 is registered in, and the co-occurrence information as shown in FIG. 5 (a) is registered in the co-occurrence dictionary 110.

[0024] For example, "when you take a study"
When a kana character string is input, candidate character strings "when performing a material" and "when sponsoring a material" are obtained, and the candidate character string "when performing a material" in each word Display unit 1 as the first candidate based on the frequency information of
03 is displayed.

Here, since there is no word pair that matches the co-occurrence information in the candidate character string "performance the material", as shown in FIG. 6 (a), the part "performance the material" is performed. A red underline will be displayed as co-occurrence information (indicating that no co-occurrence occurs here).

On the other hand, in response to the input of a kana character string "when playing Shibai," candidate character strings "when performing a play" and "when sponsoring a play" are obtained. The candidate character string “when performing a play” is displayed on the display unit 103 as the first candidate. In this case, “play” and “performance” in “when performing a play”
Is a word that matches the co-occurrence information, there is no red underline display as shown in FIG.

FIG. 7 is a flow chart showing the processing operation in the first embodiment. When a kana character string to be converted is input from the input unit 101, the word search unit 10
In 4, the word candidates corresponding to the kana character string are extracted,
The phrase candidate generating unit 105 generates phrase candidates from the word candidates. The result obtained by the phrase candidate generating unit 105 is stored in the word string candidate holding unit 108.

At this time, the phrase candidate selecting unit 106 refers to the co-occurrence information regarding each word of the phrase candidate using the co-occurrence dictionary 110, and if there is a word that matches the co-occurrence information, it indicates that. Flag information is added.

The edit control unit 102, through the phrase candidate selecting unit 106, outputs the first result to be output as the conversion result.
When a candidate character string for rank is obtained (step A11), the first
The first phrase in the candidate character string for rank is referred to, and it is checked whether or not the coincidence flag of the co-occurrence information is set for that phrase (step A12).

As a result, if the coincidence flag of the co-occurrence information is not set in the phrase (No in step A13), the edit control unit 102 adds the highlighting instruction information to the surface character string information corresponding to the phrase. (Step A14).
On the other hand, if the coincidence flag of the co-occurrence information is set (Yes in step A13), such emphasis display instruction information is not added.

In this way, the editing control unit 102 is the first
By checking the presence / absence of co-occurrence information for each clause of the candidate character string of rank (steps A15 and A16), if a clause without co-occurrence information is detected, highlighting instruction information is added to the clause character string. Then, it outputs to the display part 103 (step A17). Thereby, when the candidate character string is output, it becomes possible to easily confirm whether or not the expression is known as the co-occurrence by the presence or absence of the red underline.

Next, the processing of the edit control unit 102 according to the second embodiment will be described. In the edit control unit 102, when the first candidate determination process in the phrase sequence candidate selection unit 106 is completed, the result is sent to the display unit 103. At that time,
The edit control unit 102 determines a portion where the co-occurrence information matches based on the comparison result of each word of the word sequence candidates of the word sequence candidate holding unit 108 obtained by the phrase sequence candidate selection unit 106 and the co-occurrence information. Light gray characters (dot thinned characters)
Display unit 1 with added highlighting instruction information for displaying in
Output to 03.

More specifically, the word dictionary 109 will now be described.
It is assumed that the dictionary information as shown in FIG. 4 is registered in, and the co-occurrence information as shown in FIG. 5 (a) is registered in the co-occurrence dictionary 110.

[0034] For example, "when you need to study"
When a kana character string is input, candidate character strings "when performing a material" and "when sponsoring a material" are obtained, and the candidate character string "when performing a material" in each word Display unit 1 as the first candidate based on the frequency information of
03 is displayed.

Here, since there is no word pair that matches the co-occurrence information in the candidate character string "performance material", they are displayed in normal (default) characters. On the other hand, in response to the input of the kana character string "when playing Shibai", candidate character strings "when performing a play" and "when sponsoring a play" are obtained. The candidate character string “when performing” is displayed as the first candidate on the display unit 103. In this case, “play” and “performance” in “play the performance” match the co-occurrence information. Since it is a word, this part will be displayed in light gray characters.

FIG. 8 is a flow chart showing the processing operation in the second embodiment. When a kana character string to be converted is input from the input unit 101, the word search unit 10
In 4, the word candidates corresponding to the kana character string are extracted,
The phrase candidate generating unit 105 generates phrase candidates from the word candidates. The result obtained by the phrase candidate generating unit 105 is stored in the word string candidate holding unit 108.

At this time, the phrase candidate selecting unit 106 refers to the co-occurrence information regarding each word of the phrase candidate using the co-occurrence dictionary 110, and if there is a word that matches the co-occurrence information, it indicates that. Flag information is added.

The edit control unit 102, through the phrase candidate selection unit 106, outputs the first conversion result.
When the candidate character string for rank is obtained (step B11), the first
The first phrase in the rank candidate character string is referred to, and it is checked whether or not the coincidence flag of the co-occurrence information is set for that phrase (step B12).

As a result, when the coincidence flag of the co-occurrence information is set for the phrase (Yes in step B13), the edit control unit 102 adds the suppression display instruction information to the surface layer character string information corresponding to the phrase. (Step B14). On the other hand, if the coincidence flag of the co-occurrence information is not set (No in step B13), such suppression display instruction information is not added.

In this way, the editing control unit 102 is the first
By checking the presence / absence of co-occurrence information for each phrase of the candidate character string of rank (steps B15, B16), when a phrase with co-occurrence information is detected, suppression display instruction information is added to the phrase character string. Then, it outputs to the display part 103 (step B17). Accordingly, when the candidate character string is output, it becomes possible to easily confirm whether or not the expression is known as the co-occurrence based on the display density of the character.

Next, the operation of the editing control section 102 according to the third embodiment will be described. The editing control unit 102
The co-occurrence / non-occurrence information display processing unit and the editing processing unit are included. Then, if the character sent to the edit processing unit is not the code of the display key for displaying the co-occurrence information, the process corresponding to the code is performed.

If the key is the co-occurrence / non-occurrence information display key, the word string candidate holding unit 108 is referred to read the co-occurrence / non-occurrence information, and the display unit 103 displays the word pairs whose co-occurrence information does not match. A code for black-and-white reverse display is added to the above, and output to the display unit 108.

More specifically, the word dictionary 109 will now be described.
It is assumed that the dictionary information as shown in FIG. 4 is registered in, and the co-occurrence information as shown in FIG. 5 (a) is registered in the co-occurrence dictionary 110.

For example, "when you take a study"
When a kana character string is input, candidate character strings "when performing a material" and "when sponsoring a material" are obtained, and the candidate character string "when performing a material" in each word Display unit 1 as the first candidate based on the frequency information of
03 is displayed.

When the co-occurrence / non-occurrence information display key provided in the input unit 101 is pressed, the "Perform material" part has no co-occurrence information, so that the "Perform material" part is displayed. Black and white display is reversed.

On the other hand, in response to the input of a kana character string "when playing Shibai", candidate character strings "when performing a play" and "when supporting a play" are obtained. Although the candidate character string "when performing a play" is displayed on the display unit 103 as the first candidate. In this case, even if the co-occurrence / non-occurrence information display key is operated, the display unit 1
The display candidate of 03 does not change.

FIG. 9 is a flow chart showing the processing operation in the third embodiment. When a kana character string to be converted is input from the input unit 101, the word search unit 10
In 4, the word candidates corresponding to the kana character string are extracted,
The phrase candidate generating unit 105 generates phrase candidates from the word candidates. The result obtained by the phrase candidate generating unit 105 is stored in the word string candidate holding unit 108.

At this time, the phrase candidate selecting unit 106 refers to the co-occurrence information regarding each word of the phrase candidate using the co-occurrence dictionary 110, and if there is a word that matches the co-occurrence information, it indicates that. Flag information is added.

The edit control unit 102, through the phrase candidate selecting unit 106, outputs the first conversion result to be output.
If a candidate character string for rank is obtained (step C11), the first
The first phrase in the rank candidate character string is referred to, and it is checked whether or not the coincidence flag of the co-occurrence information is set for that phrase (step C12).

As a result, if the coincidence flag of the co-occurrence information is not set for the phrase (No in step C13), the edit control unit 102 adds the highlighting instruction information to the surface character string information corresponding to the phrase. (Step C14).
On the other hand, if the coincidence flag of the co-occurrence information is set (Yes in step C13), such emphasis display instruction information is not added.

In this way, the editing control unit 102 is the first
By checking the presence / absence of co-occurrence information for each clause of the candidate character string of rank (steps C15 and C16), when a clause without co-occurrence information is detected, the highlighting instruction information is added to the clause character string. Then, it outputs to the display part 103 (step C17).

Here, the display unit 103 displays the first-ranked candidate character string (step C18). At that time, if the display code of the co-occurrence information is input from the input unit 101 ( Step C19: Yes), edit control unit 10
In step 2, the portion to which the highlighting instruction information is added is displayed in black and white in reverse (step C20). Thus, when the candidate character string is output, it is possible to easily confirm whether or not the expression is known as a co-occurrence, by the black-and-white inverted portion that is changed by pressing the co-occurrence information display key.

The present invention can be used not only in the above-described embodiment but also in various other ways without departing from the spirit of the invention. In the kana-kanji conversion system, the output of the conversion result provided with the co-occurrence information may be interpreted and displayed by another system.

Further, in the process of kana-kanji conversion processing, with respect to the candidate character string output as the first candidate, the words of the second and lower candidates in which the corresponding reading character strings are at the same position match the co-occurrence information. In this case, information indicating that fact can be added at the same time and highlighted.

For example, if the co-occurrence information as shown in FIG. 5B is registered in the co-occurrence dictionary 110 in response to the input of the character string "SHIBAI KOU SHI EN", the first place In addition to the expression "Perform a play", which is the candidate character string of, the co-occurrence information also matches the second-ranked candidate character string, "Sponsor a play."

Therefore, in the example of the first embodiment described above, a thin red underline is displayed in the "play performance" portion. Thereby, it is possible to know in advance that the candidate having the next matching co-occurrence information will be output by the next candidate operation.

In addition to the kana-kanji conversion processing, the Japanese output in the natural language processing, that is, the machine translation processing in which the input is the first language, for example, English, and the output is the second language, for example, Japanese. By providing a means for referring to and comparing the co-occurrence information with respect to the linguistic information that composes, it is possible to highlight a portion that does not match the co-occurrence information in the translated text or a portion that matches.

The co-occurrence information to be referred to may be a co-occurrence dictionary owned inside the system, a part of the word dictionary owned inside the system, a dictionary learned during the previous processing, or a dictionary outside the system. It does not matter whether it is retrieved from the document data inside or outside the system.

Further, not only the word-to-word co-occurrence information, but also co-occurrence information between a code representing a word group (so-called semantic feature) and a word or a word group code may be used. Words that do not match the word-to-word co-occurrence information or the code-containing co-occurrence information are shown in red underline, and the words that match only the code-containing co-occurrence information are shown in light red underline. Good.

Further, co-occurrence information of three or more words may be used. Further, the word pair matching with the co-occurrence information may be provided with information indicating a place where the matching co-occurrence information is located, for example, information for referring to the place may be provided as co-occurrence information.

The information to be referred to may be address information of the dictionary if the co-occurrence information is a dictionary inside the system, or if it is an expression in document data on a CD-ROM outside the system. It may be information indicating the place.

The type of co-occurrence information referred to, for example, syntactic relation between words (relationship between compound words, subject and predicate), frequency of co-occurrence information, field of co-occurrence information, co-occurrence The usefulness of the co-occurrence information can be enhanced by providing the co-occurrence information with a type according to the type of information such as the age of the information, the data to which the co-occurrence information belongs, and information about the likelihood of the co-occurrence information.

By combining with a process of checking whether a word pair in the language information matches this condition by a preset part-of-speech sequence condition, for example, a condition of part-of-speech + particle + verb, the condition is met. By detecting a word pair that does not match the co-occurrence information, the amount of co-occurrence presence / absence information to be used as a reference can be optimized.

Further, by combining with a process of checking whether a word pair in the language information matches this condition by a preset part-of-speech sequence condition, for example, an adverb + verb condition, for example, a word that matches the condition By detecting a word pair that is a pair and does not match the co-occurrence information, the amount of co-occurrence presence / absence information to be referenced can be optimized.

By combining with a process of checking whether a word pair in the language information matches this condition by a preset part-of-speech sequence condition, for example, a condition of noun + verb, for example, a word that matches the condition. By detecting a word pair that is a pair and does not match the co-occurrence information, the amount of co-occurrence presence / absence information to be referenced can be optimized.

Further, by combining a part-of-speech condition set in advance, for example, an adverb + verb, with a process for checking whether a word pair in the language information matches this condition, for example, a word that matches the condition. By detecting a word pair that is a pair and does not match the co-occurrence information, the amount of co-occurrence presence / absence information to be referenced can be optimized.

Furthermore, the features (2
It may be a value or a multivalue, and may be called property information or property), for example, a word in language information has a feature assigned to "a noun susceptible to adnominal modification of a noun" By combining it with the process of checking whether it has or not, for example, it has no features and is subject to adnominal modification
Moreover, by detecting a word pair that does not match the co-occurrence information, the amount of co-occurrence presence / absence information to be referred to can be optimized.

[0068]

As described above, according to the present invention, whether the word group in this language information matches the co-occurrence information for the language information that constitutes the natural language expression obtained by converting the input. When the word group in the above language information does not match or coincides with the co-occurrence information, the co-occurrence presence / absence information indicating the fact is output by attaching it to the natural language expression. When a natural language expression is output, it becomes possible to easily confirm whether or not the expression is known as co-occurrence. As a result, the output natural language expression can be understood, and the quality of the document at the time of editing and the efficiency of document creation can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a kana-kanji conversion device according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a word dictionary of the device.

FIG. 3 is a diagram showing a configuration of a co-occurrence dictionary of the device.

FIG. 4 is a diagram showing a specific example of the word dictionary.

FIG. 5 is a diagram showing a specific example of the co-occurrence dictionary.

FIG. 6 is a diagram showing a display example according to the first embodiment.

FIG. 7 is a flowchart showing a processing operation according to the first embodiment.

FIG. 8 is a flowchart showing a processing operation according to the second embodiment.

FIG. 9 is a flowchart showing a processing operation according to the third embodiment.

[Explanation of symbols]

 101 ... Input unit, 102 ... Editing control unit, 103 ... Display unit, 104 ... Word search unit, 105 ... Phrase candidate generation unit, 106 ... Phrase candidate selection unit, 107 ... Connection table, 108 ... Word string candidate holding unit, 109 … Word dictionary, 110… Co-occurrence dictionary.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Tatsuya Dewa, Inventor Tatsuya Dewa, 1 Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa Within the Corporate Research and Development Center, Toshiba Corporation (72) Yumi Mizutani, Komukai-Toshiba, Saiwai-ku, Kawasaki-shi, Kanagawa Town No. 1 Toshiba Corporation Research & Development Center

Claims (2)

[Claims] 1. A holding means for holding language information constituting a natural language expression obtained by converting an input, and whether or not a word group in the language information held by this holding means matches the cooccurrence information. The co-occurrence determining means for determining whether or not the co-occurrence determining means determines whether the word group in the language information does not match or coincides with the co-occurrence information by the co-occurrence determining means. A natural language processing apparatus, comprising: an output unit that outputs an expression. 2. Targeting the linguistic information that constitutes the natural language expression obtained by converting the input, it is checked whether or not the word group in this linguistic information matches the co-occurrence information, and the A natural language processing method characterized in that, when the word group of does not match with the co-occurrence information or when it matches, the co-occurrence presence / absence information indicating that fact is added to the natural language expression and output.