JP2000268034A - Automatic text pre-editing apparatus and method, and storage medium used therefor - Google Patents

Abstract

(57) [Summary] In a pre-editing process of a text described in a natural language, identification of a text type and selection of a pre-editing rule and a standard notation according to the text type are automated. SOLUTION: A dictionary table storing words, part-of-speech information, and morpheme information related to text, an identification rule table storing identification rules for identifying types of text, and a pre-editing rule for pre-editing text. A morphological analysis that extracts a part-of-speech information and a morphological information by performing a morphological analysis on each word of the input text by referring to a pre-editing rule table storing standard notations for each type of text, an input unit for inputting the text, and a dictionary table. And the identification unit that identifies the type of text from the extracted part-of-speech information and morpheme information by referring to the identification rule table and the pre-editing rule table that references the editing rule corresponding to the identified type. A pre-editing unit that detects a word in the portion to be edited and rewrites the word into standard notation.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is applied to a machine translation device or the like of a natural language processing system, and pre-edits a text described in a natural language to the extent that its meaning is not changed, thereby enabling natural language such as machine translation. The present invention relates to a text pre-editing apparatus and method for improving processing accuracy and a storage medium used for the same.

[0002]

2. Description of the Related Art Conventionally, a natural language processing system such as a machine translator usually processes various types of texts such as scientific and technical papers, patent specifications, equipment instruction manuals, and news articles.

However, in a conventional machine translation device,
Because the rules for parsing text correspond to standard expressions, depending on the domain and type of text,
Since special expressions are frequently used, parsing of text in a special expression format cannot be properly performed. For this reason, it is difficult for a translator using the existing parsing rules to properly translate a text in a special expression format, and even if it is translated, it takes a very long time to translate, and the translation result is also high. It is not desirable. Therefore, in general, countermeasures have been taken to extend the parsing rules so that text in a special expression format can be handled.

On the other hand, Japanese Patent Application Laid-Open No. Hei 6-139274 discloses a prior art in which a pre-editing process is performed prior to inputting a text into a machine translator while improving the accuracy of machine translation and minimizing user intervention. Has proposed a text automatic pre-editing device that detects a portion to which a pre-editing rule is to be applied by preliminarily morphologically analyzing an input text and automatically pre-edits the input text into a form that can be easily translated.

[0005]

However, a conventional machine translator that extends an existing parsing rule so that it can handle the translation of a text in a special expression format has the following problems, for example. (1) Since the scale of the parsing rules becomes very large, it is not easy in a practical system to add new rules while maintaining consistency with existing rules. (2) rules for handling text in a special expression form,
Mixing the rules for handling text in a standard representation format reduces the versatility of existing rules.

In the text pre-editing device described in Japanese Patent Application Laid-Open No. Hei 6-139274, when pre-editing a text in a special expression format into a text in a standard expression format, the text pre-editing device conforms to a wide variety of text types. The process of selecting the pre-editing rule group is not automated, and the user has to select the pre-editing rule group according to the type of text.

The present invention has been made in view of the above circumstances. For example, in a pre-editing process of a text described in a natural language, identification of a text type and pre-editing according to the text type are performed. Automatic text pre-editing apparatus and method for automating the selection of rules and standard notation, rewriting words in the pre-editing target portion of the text detected based on the automatically selected pre-editing rule into standard notation, and storage used for this Provide media.

[0008]

SUMMARY OF THE INVENTION The present invention provides a dictionary table storing words, parts of speech information and morpheme information related to a text described in a natural language, and an identification table storing identification rules for identifying types of text. A table memory consisting of a rule table, a pre-editing rule for pre-editing text and a pre-editing rule table storing standard notations for each type of text, an input unit for inputting text, and an input text by referring to a dictionary table. A morphological analysis unit that performs morphological analysis on each of the words to extract part-of-speech information and morphological information, and a part-of-speech information extracted by referring to an identification rule table;
Refers to an identification unit for identifying the type of text from the morpheme information and a pre-editing rule table, detects a word in a pre-editing target portion of the text from an editing rule corresponding to the identified type, and rewrites the word to a standard notation. An automatic text pre-editing device comprising a pre-editing unit.

According to the present invention, in the pre-editing process of a text described in a natural language, identification of a text type and selection of a pre-editing rule and a standard notation according to the text type are automated. Burden can be reduced. Further, it is possible to provide an automatic text pre-editing apparatus that rewrites a word of a pre-editing target portion of a text detected based on the automatically selected pre-editing rule into a standard notation.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In the structure of the present invention, (1) existing rules are not changed, and a rule group for handling a special expression is defined as:
A pre-editing rule table maintained in a format independent of the existing rules for handling standard expressions, and a special format expression so that existing rules can be properly processed before processing by the existing rules There is a pre-editing unit that rewrites the expression. (2) The type of text is automatically identified based on the identification rule table for various types of information included in the target text, and an appropriate preedit rule group is automatically determined from the preedit rule table according to the identification result. select.

In the present invention, the dictionary table, the identification rule table, and the pre-editing rule table include, for example, a tape system such as a magnetic tape or a cassette tape which can be separated from the main body, and a magnetic system such as a floppy (registered trademark) disk or a hard disk. Disc and CD-ROM / MO / MD / DVD
Such as an optical disk, an IC card (including a memory card) / an optical card, or a mask R
It may be configured by a storage medium that fixedly carries a program, including a semiconductor memory such as an OM, an EPROM, an EEPROM, and a flash ROM. The input unit may be configured by an input device such as a keyboard, a mouse, a pen, a tablet, a scanner, a character recognition device, and a storage medium reading device.
The identification unit and the pre-editing unit are, for example, CPU, ROM, RA
It may be configured by a computer including M and I / O ports, an ASIC including a CPU, and the like.

A pre-editing unit that refers to a pre-editing rule table and selects an editing rule corresponding to the type of text identified by the identifying unit; and a pre-editing unit that precedes the text corresponding to the selected editing rule. A configuration may also be provided that further includes a search unit that searches for the word of the edit target portion, and a rewriting unit that rewrites the searched word to standard notation. In this configuration, the selecting unit, the searching unit, and the rewriting unit include, for example, a CPU, an R
Computer consisting of OM, RAM, I / O port, C
An ASIC including a PU may be used.

[0013] The identification rule table may be configured to store identification rules for discriminating whether the type of text is a scientific or technical paper, a patent specification, an instruction manual for a device, or a news article for each field. Good.

According to this configuration, there is provided a machine translation system in which a machine translation device that performs machine translation between different languages performs a pre-editing process of translation using the automatic text pre-editing device of the present invention. Further, there is provided a natural language interface for connecting a natural language device and controlling the transfer of a text described in a natural language by using the automatic text pre-editing device of the present invention to pre-edit text. Further, there is provided a text summarizing system in which a text summarizing apparatus for automatically summarizing text described in a natural language performs a pre-editing process on a text using the automatic text pre-editing apparatus of the present invention.

Hereinafter, the present invention will be described in detail based on an embodiment shown in the drawings. The present invention is not limited by this.

FIG. 1 is a block diagram showing the configuration of an automatic text pre-editing apparatus according to an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes a control unit including a CPU (Central Processing Unit) of a computer, and the control unit 1 controls each unit by a control program stored in a program memory.

Reference numeral 2 denotes an input unit such as a keyboard, a mouse, a pen, a tablet, a scanner, and a character recognition device, and an input unit including a communication device connected to a communication line, a storage medium reading device, and the like. Input of text written in natural language, instruction of pre-editing process, communication of text,
Install control programs.

3 is a CRT (cathode ray tube) display, L
A display device 3a including a CD (liquid crystal display), a PD (plasma display), a thermal printer,
Shows an output unit composed of a printing device such as a laser printer or a communication device 3c connected to a communication line. The output unit 3 displays an input result from the input unit 2 and a translation result under the control of the control unit 1. The information is displayed on the device 3a, printed via the printing device 3b, or transmitted via the communication device 3c.

4 is a mask ROM, EPROM, EEPROM
OM, a semiconductor memory such as a flash ROM, a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy disk or a hard disk, or a CD-ROM.
The table memory 4 includes a storage medium including a disk system of an optical disk such as a ROM / MO / MD / DVD and a card system such as an IC card (including a memory card) / an optical card. A dictionary table 4a storing information and morphological information, an identification rule table 4b storing identification rules for identifying types of text,
A pre-editing rule table that stores pre-editing rules such as pre-editing rules for headlines of newspaper articles, pre-editing rules for patent specifications, and pre-editing rules for instruction manuals for text pre-editing, and standard notations for each type of text 4c.

5 is a mask ROM, EPROM, EEPROM
OM, a semiconductor memory such as a flash ROM, a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy disk or a hard disk, or a CD-ROM.
A program memory including a storage medium including a disk system of an optical disk such as a ROM / MO / MD / DVD and a card system such as an IC card (including a memory card) / an optical card is shown. The program memory 5 is a dictionary table 4a. , Morphological analysis of each word of the input text to extract part-of-speech information and morphological information, morphological analysis section 5a, and reference to identification rule table 4b, POS information and morphological information of the words extracted by morphological analysis section 5a Identifying unit 5b for identifying the type of text from the above, referring to the pre-editing rule table 4c, detecting the word of the part to be pre-edited of the text from the editing rule corresponding to the identified type, and rewriting the word to the standard notation. Each control program that functions as the editing unit 5c is stored.

The pre-editing section 5c includes a selecting section 5c-1 for selecting an editing rule corresponding to the type of text identified by the identifying section 5b from the pre-editing rule table, and a pre-editing section 5c-1 for selecting a text corresponding to the selected editing rule. It functions as a retrieval unit 5c-2 for retrieving the word of the edit target portion and a rewriting unit 5c-3 for rewriting the retrieved word into the standard notation.

6 is RAM, EEPROM, flash R
OM or other semiconductor memory, or tape system such as magnetic tape or cassette tape, magnetic disk such as floppy disk or hard disk, CD-ROM / MO / MD
/ A buffer memory composed of a storage medium including a disk system of an optical disk such as a DVD and a card system such as an IC card (including a memory card) / an optical card, and the buffer memory 6 is a text buffer 6 for storing input text.
a, a morphological analysis result buffer 6 for storing words, part-of-speech information, and morphological information morphologically analyzed by the morphological analysis unit 5a
b, an identification result buffer 6c for storing the type identified by the identification unit 5b, a selection result buffer 6d for storing the standard notation selected by the selection unit 5c-1, and a text rewritten by the rewriting unit 5c-3. It is provided in an area functioning as a rewriting text buffer 6e.

The control unit 1 stores data subjected to the matching process with the identification rule and data subjected to the matching process with the pre-editing rule in each buffer. Reference numeral 7 denotes a bus line to which control program data and address data are transferred. The control unit 1 realizes the text pre-editing device of the present invention by reading out a control program from the program memory 4 via the bus line 7 and controlling each unit.

8 is a mask ROM, EPROM, EEPROM
OM, a semiconductor memory such as a flash ROM, a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy disk or a hard disk, or a CD-ROM.
A storage for fixedly carrying a program constituted by a medium which can be separated from a main body including a disk system of an optical disk such as a ROM / MO / MD / DVD and a card system such as an IC card (including a memory card) / an optical card. A text pre-editing program of the present invention in a storage medium 8;
The buffer memory 6 via the storage medium reading device of the input unit 2
The pre-text editing function of the present invention may be realized by installing a pre-text editing program in the spare area. When the text pre-editing device includes a communication device connectable to an external communication network including the Internet, the storage medium downloads the program from the communication network via the communication device. It may be a medium that carries the program in a fluid manner. When the program is downloaded from the communication network as described above, the download program may be stored in the main device in advance, or may be installed from another storage medium. The content stored in the storage medium is not limited to a program, but may be data.

According to another aspect of the present invention, a dictionary table 4a storing words, parts of speech information, and morpheme information related to a text described in a natural language, and an identification rule for identifying the type of text are stored. A text automatic system comprising a table memory 4 comprising an identification rule table 4b, a pre-editing rule for pre-editing text and a pre-editing rule table 4c storing standard notations for each type of text, and an input unit for inputting text. A storage medium 8 that is used in a pre-editing device and stores a text automatic pre-editing program that can be read by a computer 1. The computer 1 refers to a dictionary table 4a, performs morphological analysis of input text, and performs part-of-speech information, The morpheme information is extracted, and the text is extracted from the extracted part-of-speech information and morpheme information by referring to the identification rule table 4b. To identify of Type refers to the pre-editing rules table 4c, to detect the word before editing target portion of the text from the editing rules corresponding to the identified type, it is possible to cause rewriting the word in standard notation.

FIG. 2 is a flowchart showing the procedure of the automatic text pre-editing process of this embodiment. Hereinafter, FIGS. 3 to 5
The pre-editing process of the text translated by the English-Japanese machine translation system will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of the input text according to the present embodiment. As shown in FIG. 3, texts E1 to E3 to be subjected to English-Japanese machine translation are input by the input unit 2 and stored in the text buffer 6a.
Subsequently, an instruction for automatic text pre-editing processing is input by the input unit 2.

In STEP 1, a morphological analysis of the target text is performed by the morphological analysis unit 5a, and vocabulary attributes such as parts of speech are extracted for each word which is a target expression included in the target text. In STEP 2, the value i of a target expression number counter that counts the number of target expressions included in one target text is reset. In STEP3 to STEP6, the identification unit 5b performs matching with each text identification rule stored in the identification rule table in order from the head of the target text one expression at a time, and determines the type of text according to which condition is matched. Identify.

FIG. 4 is a diagram showing an example of the identification rules stored in the identification rule table according to the present embodiment. When identifying the type of the input text, whether or not the target expression of the input text satisfies the identification rules 1) and 2) is determined by comparing the target expressions. In FIG. 4, since the input texts E2 and E3 shown in FIG. 3 satisfy the identification rules 1) and 2), the input text is identified as a newspaper article.
In the related art, it is necessary to specify in advance that the type of the target text is a newspaper article, for example. In this embodiment, the type of the target text is automatically identified.

In STEP 7, if the type of text cannot be identified by the identification rules stored in advance in the identification rule table, a certain value (for example, “general text”) is set as the type of text. In STEP 8, the value i of a target expression number counter that counts the number of target expressions included in one target text is reset.

In STEP9 to STEP11, the pre-editing unit 5c (selection unit, search unit, rewriting unit) performs matching with each pre-editing rule stored in the pre-editing rule group table in order from the head of the target text one expression at a time. Is performed, and by rewriting the expression that succeeds in matching,
Execute pre-edit processing. Note that n in STEP 9 is the number of expressions included in the input text.

FIG. 5 is a diagram showing an example of the pre-editing rule stored in the pre-editing rule table of the present embodiment. As shown in FIG. 5, the pre-editing rule table 4c stores, for example, pre-editing rules for headlines of newspaper articles. In the example of FIG. 5, when all of the pre-editing rules 1), 2), and 3) are satisfied, the pre-editing process rewrites “to” to “will”.

When pre-editing the text shown in FIG.
1st text “Agency to inspecthealth of 8 banks
Satisfies all of the pre-editing rules 1), 2) and 3) in FIG. 5, so "Agency will inspect health of 8 banks"
"Agency to inspect health of
8 banks ”is replaced by“ Agency will inspect healt
You may rewrite it as "h of 8 banks" if it is the headline of a newspaper article, otherwise it may not be rewritten.

On the other hand, the texts of E2 and E3 are
Since this editing rule is not satisfied, it cannot be rewritten. Whether or not the target expression of the text satisfies the pre-editing rule 1) in FIG. 5 is determined by checking whether the target expression matches, for example, a noun phrase pattern as follows.

Noun phrase pattern: {article} adverb} restricted adjective {noun *} noun Here, the elements (article, adverb, qualified adjective, noun *) surrounded by parentheses {} may or may not exist. The symbol "*" means that the element can appear repeatedly an arbitrary number of times.

Whether the target expression satisfies the pre-editing rule 2) is determined by searching for the word "for" on the target expression. Whether the target expression satisfies the pre-editing rule 3) is checked by, for example, the following procedure.

As a procedure for determining whether or not the target expression of the text satisfies the pre-editing rule, a fixed verb that can be a predicate is searched in order from the head of the target expression. If a predicate is found, it is checked whether a predicate candidate and a noun phrase whose head is a noun whose personality and number match are present in front. The noun phrase pattern is used for searching for the noun phrase. If such a noun phrase exists, it is regarded as a subject, the entire target expression is regarded as a sentence, and the pre-editing rule 3) is not satisfied.

The text "Agency to in
If you translate “spect health of 8 banks” into a conventional machine translation system that mainly deals with standard expressions, you can translate it into newspaper articles such as “Institutions for investigating the health of eight banks” Produces an inappropriate translation for the headline.

On the other hand, the morphological analysis processing in STEP1, the text identification processing in STEP2 to STEP7,
As can be seen from the description of the text pre-editing processing in STEP 8 to STEP 11, "Agency will inspect health of 8 banks" rewritten by the text pre-editing apparatus of the present invention.
Can be input into a conventional machine translation system and translated, for example, an appropriate translation can be obtained as a headline of a newspaper article, such as “the institution will investigate the health of eight banks”.

FIG. 6 is a diagram showing an example of an application system of the automatic text pre-editing apparatus of the present invention. The automatic text pre-editing device of the present invention is independent of the natural language processing system, and is a system utilizing pre-edited text output from the text pre-editing device of the present invention. There is.

For example, a machine translation system that performs machine translation between different languages can provide a machine translation system that performs pre-editing processing of translation using the automatic text pre-editing device of the present invention. In addition, an interface for connecting a natural language device and controlling the transfer of text written in a natural language can provide a natural language interface for performing pre-editing processing of text using the automatic text pre-editing device of the present invention. Further, it is possible to provide a text summarizing system in which a text summarizing apparatus for automatically summarizing text described in a natural language performs a pre-editing process on a text using the automatic text pre-editing apparatus of the present invention.

[0041]

According to the present invention, in the pre-editing process of a text described in a natural language, identification of a text type and selection of a pre-editing rule and a standard notation according to the text type are automated. Thus, the burden on the user can be reduced. Further, it is possible to provide an automatic text pre-editing apparatus that rewrites a word of a pre-editing target portion of a text detected based on the automatically selected pre-editing rule into a standard notation.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an automatic text pre-editing apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a procedure of an automatic text pre-editing process according to the embodiment;

FIG. 3 is a diagram illustrating an example of an input text according to the embodiment;

FIG. 4 is a diagram illustrating an example of an identification rule stored in an identification rule table according to the embodiment;

FIG. 5 is a diagram illustrating an example of a pre-editing rule stored in a pre-editing rule table according to the present embodiment.

FIG. 6 is a diagram showing an example of an application system of the automatic text pre-editing apparatus of the present invention.

[Explanation of symbols]

 Reference Signs List 1 control unit 2 input unit 3 output unit 4 table memory 4a dictionary table 4b identification rule table 4c pre-editing rule group table 5 program memory 5a morphological analysis unit 5b identifying unit 5c pre-editing unit 5c-1 selecting unit 5c-2 searching unit 5c -3 Rewriting unit 6 Buffer memory 6a Text buffer 6b Morphological analysis result buffer 6c Identification result buffer 6d Selection result buffer 6e Rewriting text buffer 7 Bus line 8 Storage medium

Claims (8)

[Claims] 1. Pre-editing a text and a dictionary table that stores words, part of speech information and morpheme information related to text described in a natural language, an identification rule table that stores identification rules for identifying types of text, and text. A table memory consisting of a pre-editing rule table storing pre-editing rules and standard notations for each type of text, an input unit for inputting text, and a dictionary table, and performing morphological analysis on each word of the input text. Part-of-speech information, a morphological analysis unit that extracts morphological information, and an identification unit that refers to an identification rule table, and an extracted part-of-speech information, an identification unit that identifies the type of text from morphological information, and a pre-editing rule table,
An automatic text pre-editing device comprising: a pre-editing unit for detecting a word in a pre-editing target portion of a text from an editing rule corresponding to an identified type, and rewriting the word into a standard notation. 2. The pre-editing unit refers to a pre-editing rule table and selects an editing rule corresponding to a type of text identified by the identifying unit, and a text corresponding to the selected editing rule. 2. The automatic text pre-editing apparatus according to claim 1, further comprising: a search unit that searches for a word in the part to be pre-edited; and a rewriting unit that rewrites the searched word into a standard notation. 3. The identification rule table stores identification rules for discriminating whether a type of text is a science and technology paper, a patent specification, an instruction manual for a device, or a news article for each field. 2. The automatic text pre-editing device according to claim 1, wherein: 4. A dictionary table storing words, parts of speech information, and morphological information related to text described in a natural language, an identification rule for identifying the type of text in an identification rule table, and The pre-editing rule and the standard notation for editing are stored in the pre-editing rule table for each type of text, and the text is input using the input unit,
Using the morphological analysis unit, refer to the dictionary table, morphologically analyze each word of the input text to extract part-of-speech information and morphological information, use the identification unit to refer to the identification rule table, and extract the extracted part-of-speech information Identifying the type of text from the morpheme information, using the pre-editing unit, referencing the pre-editing rule table, detecting the word of the pre-editing target portion of the text from the editing rule corresponding to the identified type, and Automatic pre-editing method, characterized by rewriting the standard notation. 5. A computer which refers to a dictionary table storing words, part-of-speech information, and morphological information related to a text described in a natural language, performs morphological analysis on the input text, and extracts part-of-speech information and morphological information. Refer to the identification rule table storing the identification rules for identifying the type of text, identify the type of text from the extracted part-of-speech information and morpheme information, and specify the pre-editing rule and standard notation for pre-editing the text. An automatic text editor that refers to the pre-editing rule table stored for each type of text, detects words in the pre-editing target portion of the text from the editing rules corresponding to the identified type, and executes processing for rewriting the words to standard notation. A storage medium storing a pre-editing program. 6. A machine translation system for performing machine translation between different languages by performing a pre-editing process of translation using the automatic text pre-editing device according to claim 1. 7. An automatic text pre-editing device according to claim 1, wherein an interface for connecting a natural language device and controlling the transfer of a text described in a natural language. Natural language interface. 8. A text summarization system for automatically summarizing text described in a natural language, wherein the text summarization system pre-edits the text using the automatic text pre-editing device according to claim 1.