JPH02254566A - Automatic excerpt generating device - Google Patents

Abstract

PURPOSE:To extract important words with a high precision to extract important sentences summarizing the points of contents by using information of a meaning paragraphic structure for extraction of important words and sentences to extract important words and sentences summarizing the points of respective meaning paragraphs and eliminating unnatural connection relations or the like from sentences including important word groups common to respective sentences to generates an excerpt. CONSTITUTION:A processor 3 which executes an excerpt generating program, an input device 1 which reads in document data, and an output device 2 which outputs the generated excerpt to a magnetic storage device are provided. Further, an excerpt generating program memory 4, a document memory 5, a work memory 6, a word table 7, a sentence structure table 8, an important word table 9, an important sentence table 10, an excerpt table 11, a Japanese- language dictionary 12, and a sentence structure rule dictionary 13 are provided. The sentence structure of an original text is analyzed up to the level of meaning paragraphs to extract important words and sentences, and important word groups common to respective sentences are adopted and unnatural conjunctions and demonstratives are eliminated, and the sentence structure of the original text is used again to generate the excerpt which has unity and consistency with respect to sentence. Thus, important words are extracted with a high precision to extract important sentences summarizing the points.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention automatically generates abstracts that concisely describe important contents of documents stored in the database from the original texts in order to create a document database. This relates to a device that generates

[Prior Art] When creating a database consisting of a large amount of documents such as manuals, newspaper articles, specifications of patent applications, and technical documents, it is necessary to create an abstract of the original text so that the outline of the contents of the document can be quickly grasped. It is necessary to create one. Conventionally, the following methods have been used for this purpose.

(1) A method for determining important sentences by understanding some text (2) A method for determining important sentences by analyzing the relationship between sentences (3) A method for determining important sentences by analyzing the relationship between sentences (3) Using the frequency of keywords, Methods for Determining Important SentencesHowever, each of these conventional methods has the following drawbacks.

In method (1), by using narrative grammar and causal relationships, the structural constraints and relationships between events in sentences are analyzed at a deep level, and summarization rules are applied to the data structure obtained. apply to determine important sentences.

This method generates an abstract using a large amount of knowledge and deep analysis of the subject, including world knowledge and inference rules. Therefore, it has the advantage of being able to create an abstract as a logical result of inference using various levels of knowledge such as world knowledge, common sense, linguistic constraints, intentions, content relationships, and causal relationships.

However, this method requires a great deal of world knowledge and deep reasoning, so it has the disadvantage that it can only be applied in a narrow field and to relatively short texts.

In method (2), the conjunctive relationship between two sentences (the logical relationship between two consecutive sentences) is analyzed, and the relative importance of the two sentences given for each concatenative relationship is calculated. An abstract is generated by selecting sentences in the original text based on the index. At this time, the conjunctive relationship between two sentences is determined from conjunctions and demonstratives, conceptual relationships between constituent elements between propositions, inference rules of verbs, nouns, and adjectives, scripts, and the like.

This method has the advantage that the abstract can be derived as a logical consequence of inference from rules regarding the relative importance of sentences. However, in many cases, the relationship between sentences cannot be determined using only syntactic information and requires a lot of knowledge and reasoning, so the target area is very narrow (limited). Importance obtained from conjunctions is effective for local importance evaluations, such as determining the most important sentence within a paragraph, but it is effective for evaluating local importance, such as determining the most important sentence within an entire sentence. Since it cannot be used for global importance evaluation, it has the disadvantage that it can only be applied to relatively short sentences or parts of sentences.

In method (3), the theme and core important words (keywords) of the written content are determined in advance using frequency statistics of sentences, etc., and sentences that contain many of these important words are extracted as importance (key sentences) and then abstracted. generate.

With this method, important content can be determined by comprehensively analyzing a sentence, and importance can be assigned to each sentence according to the frequency of important words. The advantage is that you can select as many degrees of importance as you need in this order. However, this method has problems such as the abstract containing sentences that have little to do with the development of the main theme of the text, or the output abstract being a list of sentences that are not related to each other and not being cohesive as a text. There were drawbacks.

To summarize, methods (1) and (2) use a large amount of knowledge to perform very deep analysis, so the problem is that the subject of abstract creation is limited to short texts in a limited field. was there. On the other hand, method (3) can be applied to a wide range of texts, but there are problems such as sentences that are unrelated to the development of the text's theme and that the abstract is not coherent as a text. there were.

[Problem to be solved by the invention] The present station is capable of extracting sentences that are unrelated to the development of the sentence,
The purpose of the present invention is to provide an automatic abstract generation device that solves the problem of the abstract generation method based on the frequency of important words, in which the abstract lacks cohesion as a sentence.

[Means for Solving the Problems] In order to solve the above problems, the present invention provides a processor that executes an abstract generation program, an input device that reads document data, and a device that outputs the generated abstract to a magnetic storage device. Output device, abstract generation program memory, document memory,
It is equipped with a working memory, a word table, a sentence structure table, an important word table, an important sentence table, an abstract table, a Japanese dictionary, and a sentence structure rule dictionary.

[Operation] The present invention utilizes the property that nouns that are frequently used in sentences that are subject to abstract creation processing are important words that form the core of the subject or description. Function words are completely removed, and important words, which are the core of the subject and description of a sentence, are extracted with high precision from the frequency and position information of common nouns and proper nouns.

In addition, a sentence is a structure whose constituent elements are headings, paragraphs, sentences, etc., and by utilizing the property that there are rules that define the structure between these constituent elements, similar to the case of syntactic analysis, First, we analyze the sentence structure using syntactic methods.

There are strong and weak content connections between formal paragraphs expressed by indentation, and this method takes advantage of the property that a series of paragraphs with high content opening speed can be grouped together as one semantic paragraph. Then, the usage of nouns in each formal paragraph is statistically analyzed, and the sentence is divided into semantic paragraphs.

Additionally, the first sentence in which an important word appears is often the one that introduces the topic or raises an issue, and the last sentence in which the important word appears is often a sentence that states the conclusion regarding the topic. The first sentence of a paragraph is often a sentence that summarizes the content of the paragraph, and the sentence that contains many important words often describes the central content of the sentence. Then, important sentences in a sentence are extracted from the frequency of important words contained in each sentence and the position of the sentence within the sentence.

When a sentence is composed of several semantic paragraphs,
Taking advantage of the fact that important words that express the central topic and important sentences that summarize the content often exist in each semantic paragraph, by extracting important words and important sentences for each semantic paragraph, When extracting important sentences, the gist of each relatively large component in the sentence structure of the original text is grasped, and when generating an abstract, the development of sentences and changes in topic in the original text are reproduced in the abstract.

When sentences containing the same group of important words are collected, there is a connection between the sentences, that is, cohesion, and the abstract has a cohesive content as a sentence. By arranging them according to the order in which they are written and giving them the same sentence structure as the original text, it is possible to reproduce the logical structure of the sentences and the order in which information is presented in the original text. By utilizing the property of being an abstract, an abstract that has both cohesion and consistency as a text is generated.

When the extracted important sentences are arranged in the order of appearance in the original text, it is possible to find that the meanings of the conjunctions and denotatives contained in these important sentences are inconsistent with the meanings in the original text, and that there are common word groups in the conjunctive phrases. Utilizing the property that cohesion can be maintained even if conjunctions and indicators are removed if they exist, in the extracted important sentences, conjunctions and indicators that are inconsistent with the context of the original sentence are detected and deleted. do. By doing this, an abstract with natural connections and no discomfort can be generated.

Through the above-described functions, the present invention takes into consideration the development of the sentence such as the presentation of the problem and the conclusion, prevents sentences unrelated to the subject from being included in the abstract, and changes the sentence structure of the original text to the level of a semantic paragraph. It is used to extract important words and important sentences by analyzing
When generating an abstract, each sentence includes a common group of important words, unnatural conjunctions and indicators are removed, and the sentence structure of the original text is reused to create an abstract that has cohesion and consistency as a text. Make it possible to create.

[Embodiment] FIG. 1 is a diagram configuring a system showing an embodiment for realizing an automatic abstract generation device of the present invention. 2 is an input device that reads document data recorded in character code in a magnetic storage device, 2 is an output device for outputting the generated abstract to the magnetic storage device, and 3 is a processor that executes an abstract generation program ( CPU), 4 is a program memory that stores an abstract generation program, 5 is a document memory that stores document data read by the input device 1, 6 is a working memory used when executing the abstract generation program, and 7 is a morphological analysis 8 is a sentence structure table that stores the sentence structure obtained by sentence structure analysis; 9 is an important word table that stores important words extracted from sentences; 10 is a sentence 11 is an abstract table that stores generated abstracts, and 12 is a Japanese dictionary that stores syntactic and semantic information necessary for morphological analysis and noun extraction. , 13 is a sentence structure rule dictionary that stores rules regarding the structure of a document to be processed. Various tables can be realized using semiconductor memory, magnetic disks, optical disks, etc.

FIG. 2 shows a functional block diagram of one embodiment of the present invention,
The input unit 21 corresponds to the input device 1 in FIG. 1, and reads a document file 28 to be processed into the document memory 5.

Next, the morphological analysis unit 22 uses the Japanese dictionary 29 to
The input sentence is looked up in a dictionary and divided into words, and a word string is generated in which each word is given a part of speech name and stored in a word table 7. At this time, compound words (long unit nouns) are divided into unit words and stored.

Next, the processes of the sentence structure analysis section 23 and the important word extraction section 24 are executed in parallel, and while mutually referring to the analysis results, the sentence structure information and the important word information are passed to the important word extraction section.

As shown in FIG. 3, the sentence structure analysis unit 23 includes a sentence form analysis means (step sl) that generates a recognizable document structure from the format, and a sentence structure analyzer (step sl) that generates a recognizable document structure from the format, and analyzes the content opening speed between formal paragraphs to analyze the meaning of the sentence. Paragraph chain analysis means for dividing into paragraphs (step s
2), and stores the constituent elements of a sentence (sentences, paragraphs, headings, meaning paragraphs, etc.) and the relationships between the constituent elements in the sentence structure table 8. Each step of the text analysis section 23 will be explained in detail below.

In the sentence form analysis in step s1, the input sentence is regarded as a character string, and the sentence structure rules stored in the sentence structure rule dictionary 30 are applied to this character string, and the title,
Recognize the basic components of sentences such as headings and sentences.

It then analyzes the relationships between these components to recognize larger text components, such as paragraphs and sections.

The sentence structure rules are a group of production rules in which "document" is the initial symbol and printable characters and format control characters are the terminal symbols.

As non-terminal symbols, "heading", "paragraph", "sentence", etc. are used. In many cases, a formal grammar description of a sentence structure can be performed within the framework of a context-free grammar (CFG). Using this rule, in sentence form analysis, a top-down CFG parser analyzes the sentence structure.

FIG. 4 is an example of the definition of formal grammar regarding sentence structure using sentence structure rules. The first line of the rule indicates that the document to be processed in this example has a document header consisting of order, affiliation, and author, and lines 6 to 9 indicate that the document body consists of sections. A section may or may not have a section header, and the section body indicates that it is a repeat of a paragraph. These rules are unique to the target document, and by rewriting the rules, text form analysis can be applied to various documents.

In the sentence form analysis (step sl), finally, the sentence structure information regarding titles, paragraphs, sentences, etc. obtained as a result of the analysis is stored in the sentence structure table 8, and the process ends. , we examine the degree of content relatedness between paragraphs by time-series analysis of the word cloud features calculated for each downfall (formal paragraph), and divide the text into several semantic groups (semantic paragraphs). become

FIG. 5 is a flowchart of the paragraph chain analysis means.

In paragraph chain analysis, first, nouns are extracted from each paragraph,
Create a paragraph word list consisting of the following items (
Step 511).

■ Unit word set (set of all nouns including tautology) ■ Headword set (set of all different nouns) ■ Total number of words (number of elements in unit word set) ■ Number of different words (number of elements in headword set) Next, in order to divide the text into semantic paragraphs, a feature value for indexing the strength of content connections is calculated for each paragraph (formal paragraph) (step 512).

Here, we focus on the changes in the number of different words and the total number of words in each paragraph as the text develops, and the repetition of the same words between adjacent paragraphs, and use them as indicators of the strength of the content connections between paragraphs. , we define the following two statistical measures: new word rate and term similarity.

k (i) : Number of different words that appear for the first time in paragraph i n (t) : Total number of words in paragraph i Term similarity D (i) = (1/ N(i)) 'E
−F(i, M) wise eV(i+1) V(i): Word cloud (set of different words) on paragraph i N(i)
: Total number of words in paragraph i M: Headword (different word) F (i, M): Frequency of use of M in paragraph i New word rate r
(1) is the average increase rate of headwords in paragraph i. Since new words tend to pile up at points where topics change, the point at which a new topic is introduced can be detected from the maximum point of the new word appearance rate.

In addition, the term similarity D (i) can detect the point of change in the topic from the minimum point of the term similarity, since common words are often used in paragraph i and the paragraph i-1 immediately before it. .

When a sentence is viewed as a time series of paragraphs, the rate of new words generally decreases and the degree of term similarity tends to gradually decrease as the sentence progresses. This is because the topic gradually converges to the conclusion as the sentence approaches the end. In order to take such trend fluctuations into account and detect changing points in the topic from a global perspective, first, we calculate the trend fluctuations of the new word rate and term similarity in the paragraph time series to a minimum of 2.
A linear function is approximated by multiplication (step 513).

In general, the series X (i) includes a trend variation T (i),
Other than that, let the residual sequence e (i) be X (i) = T
When expressed as (i) +e (i), the trend fluctuation T (i
) as a linear function T (i) = β. +β, *i, then e (i
) Q=Σ(e(i)) ” =Σ(X(i)−(β.+β
β such that t*+))' is minimized. and β can be determined. From this, the residual sequence e (i> is e(
It can be obtained as i)=X(i)-T(i>=X(i)-(β.+81 lines 1).

This method is based on the new word rate ``(i) and the term similarity D (i
), the residual e,(i) of the new word rate and the residual e,(i) of the term similarity are obtained.

Next, a paragraph whose residual series takes a value (the new word rate is a maximum value and the term similarity is a minimum value) is selected as a candidate for a break in the semantic paragraph (step 514). Specifically, a paragraph l that satisfies either of the following conditions is selected as a candidate for a semantic paragraph division point.

(a) e, (i) > O, er (i-1) < e, (
i).

e, (i) > e, (i+ 1) (b) eo(i) <0. an(i-1)>e
a(i).

eo (i>< eo (+ + 1)) Next, based on this candidate, the entire sentence is finally divided into two blocks of approximately the same size.
- Determine the meaning collapse so that it is divided into four parts (step 515).

The specific steps are as follows.

■ Arrange the dividing point candidates in descending order of max (le, (i)l, 1e0(i)I).

■ When both adjacent paragraphs are division point candidates, remove the smaller one of max (Ie, (i)1.le, (i)l) from the candidates.

■ IIIJIX (l e, (i) 1. l
Select a predetermined number of division points from the larger one of e-(i) l). However, if the original sentence is divided into several parts at the stage of sentence form analysis, Priority is given to the result, that is, the chapter structure obtained through analysis of format, etc.

Finally, the sentence form analysis stores the information of the semantic paragraph in the sentence structure table, and ends the process.

FIG. 6 is a flowchart of the important word extraction section 24.

First, in step S21, the important word extraction unit refers to the part-of-speech information of the words on the word table 7 and extracts words other than nouns and words that have function word roles such as temporal pronouns, numerals, pronouns, and formal nouns. Remove common nouns and extract only common nouns and proper nouns.

Next, in step s22, word frequency statistics are performed, and then in step s23, the headword set is arranged using the following information as a key.

1st key: Frequency of use (descending order) 2nd key: Appears in the heading/does not appear (the one that appears is written first) 3rd key: First appearance position (ascending order) For each sorted entry word, Create a frequency-ordered word table consisting of statistics.

■ Ranking of usage frequency (frequency) ■ Word appearance (headword) ■ Frequency of usage ■ Cumulative usage rate (total word coverage rate) ■ Different word coverage rate ■ Headline occurrence word flag ■ First appearance position In general, words with high frequency of use are , the m rate of being an important word is high. However, when examining the coverage of important words by a set of high-frequency words, the coverage of important words is proportional to the logarithm of the size of the set of high-frequency words (coverage of different words). In other words, the probability that a word in a group with a very high frequency will be selected as an important word is very high, but as the frequency decreases, the number of different words increases exponentially, so the word in the next most frequent group will be selected as an important word. The probability of being affected is much lower than in the previous group.

Therefore, in case it becomes necessary to rank the importance among the set of important words, the words in the first group are the most important words, and the words in the first and second groups are combined. We will call them important words and will treat them differently in subsequent processing.

Based on the above knowledge, in step s24, words with a cumulative usage rate of 15% or less are extracted as the most important words. Also,
In step s25, words with a cumulative usage rate of 25% or less are extracted as important words within a range where the different word coverage rate does not exceed 5%. Note that this threshold value was determined experimentally from the sentences used as samples so that the same amount of important words as an expert could be obtained. This method can be applied to various types of texts by changing the threshold depending on the nature of the target text and the nature of the abstract to be generated.

Finally, the important word extraction unit 24 stores the extracted most important words and important words in the important word table 9, and ends the process.

FIG. 7 is a flowchart of the important sentence extraction section. The important sentence extraction unit 25 first analyzes the morphological characteristics of each sentence in the original text based on the information in the sentence structure table 8 and the important word table 9 (step 531).

In the sentence feature analysis (step 531), the sentence features are indexed from the following three viewpoints based on the results of important word extraction and sentence structure analysis.

(a) The frequency of the most important words and important words in each sentence (b) The distribution of the most important words and important words in the chronological order of sentences (c) The position of the sentence within the sentence structure and the overall In order to reflect the context in the feature values, we prepare the following two types of important words based on the results of sentence structure analysis.

(i) Most important words and important words of the entire sentence (11) Most important words and important words of each semantic paragraph From these points of view, in the sentence feature analysis (step 531), each sentence is divided into sentences consisting of the following items. Create a feature table.

■ Meaning paragraph number, paragraph number, paragraph white sentence number ■ An important eel word that appears for the first time in that sentence in the entire sentence ■ An important eel word that appears last in that sentence in the entire sentence ■ Appears in that sentence Eel Important Words and Important Words of the Entire Sentence■ In step s32, the importance of the sentence is ranked based on the result of the sentence feature analysis for the important phrases of the semantic paragraphs that appear in the sentence. Sentences containing key words clearly describe some aspect of the subject of the text. However, when semantic information is not used, important aspects must be determined using probabilistic likelihood.

Conditions of importance that can be considered from the viewpoint of content extraction are as follows.

(a) Sentences that contain many important words (b) Sentences in which important words appear for the first time and at the end (c) Sentences at the beginning of paragraphs Sentences that contain many important words tend to state the central content of the sentence. In many cases, it is possible to associate many contents of the original text from that sentence.Sentences in which important words appear for the first time and at the end often contain many issues, conclusions, etc.

Furthermore, the sentence at the beginning of a paragraph is often a sentence that summarizes the content of the paragraph.

Conditions of importance that can be considered from the viewpoint of sentence generation are as follows.

(d) Each sentence contains a group of words related to the theme of the sentence (e) A sentence that contains an introductory description of the theme When the content is described using multiple sentences, there is cohesion between each sentence and completeness of the sentence itself. is required. To provide cohesion, each sentence must be descriptive of the same subject. Additionally, in order to provide completeness, it is necessary to include a description in the abstract that allows the reader to recognize the subject matter.

Based on these considerations, the importance of the sentences is ranked in the following order of ■■■■ (■ is the order of the number of important words).

■ The sentence in which the eel important word appears for the first time ■ The sentence in which the eel important word appears last ■ The sentence that contains the eel important word and is at the beginning of the paragraph ■ The next sentence that contains the eel important word and in which many important words appear , in step s33, the sentences are written in descending order of importance,
Select sentences until the number of sentences reaches 20% of the original sentence. Here, the threshold value was experimentally determined so that automatic abstraction would obtain the same amount of importance levels as the number of importance levels selected by experts in the sentences used as samples. By changing the threshold according to the nature of the target text and the abstract to be generated, this method can be applied to various types of texts.

Furthermore, if the information on the semantic paragraph is available, the importance of ■■ is extracted from the entire sentence, and the importance of ■■ is extracted for each semantic paragraph. At this time, the important words used are the combination of the important words of the entire sentence and the important words of each semantic paragraph.

Finally, the important word extraction unit 25 stores the importance determined in this way in the importance table 10, and ends the process.

FIG. 8 is a flowchart of the abstract generation section.

First, in step s41, the abstract generation unit 26 rearranges the selected importance levels in the order of the original text. This is to generate an abstract that is consistent as a text by reflecting the order of sentences in the original text and making the abstract a similar reduced form of the original text.

Next, in step s42, a conjunction or an indicator located at the beginning of a sentence in this string of sentences is detected using the part-of-speech information in the word table 7. This is because conjunctions and denotatives are words that indicated inter-verbal relationships in the context of the original text, so they have the potential to create conjunctions or reference relationships that are inconsistent with the context of the original text in the importance sequence. This is because there is.

Next, in step s43, among the conjunctions and indicators detected in step s42, those in the abstract that do not match the context of the original text are deleted.

Judgment as to whether or not it is consistent with the context of the original text is performed as follows using position information in the original text of importance.

Generally, when two consecutive sentences in the original text are selected for importance, there is a very high possibility that the connection and reference relationships between the preceding sentence and the subsequent sentence will be maintained even in the abstract. Furthermore, even if you remove the conjunction or the inter-indication at the beginning of a sentence, the sentence will not become a non-sentence (a grammatically incorrect sentence). Furthermore, if multiple sentences contain a common word group, there will be cohesiveness in the mata, so even if there are no conjunctions or indications at the beginning of the sentences, readers can retain the conjunctive and referential relations of the mata to a certain extent. can do.

Taking these into consideration, in step s43, the conjunctions and indications in the following sentences in two successive degrees of importance in the original text are preserved, and all others are deleted.

Next, in step s44, the title, author name, and abstract are formatted according to the format of the abstract to be generated, the results are stored in the abstract table 11, and the process of the abstract generation section 2 ends.

Finally, the output unit 27 stores the contents of the abstract table 11 in the abstract file 31.

FIG. 9 is an example of the operation of the present invention.

The original text of the input document is shown in FIG. 9e. The input document in the document file 28 read by the input R21 is subjected to morphological analysis by the morphological analysis unit 22.

Next, in the sentence structure analysis unit 23, first, sentence structure analysis is performed using the sentence structure rules in the sentence structure rule dictionary 30.
Analyze the structure of the input document. The results are shown in Figure 9a. Through this processing, each constituent element such as a title, paragraph, sentence, etc. in the input document is recognized, and the relationship between constituent elements, such as the relationship between a sentence and a paragraph, is also recognized.

Next, the paragraph chain analysis performs semantic paragraph segmentation based on the results of calculating the term similarity and new word rate of each paragraph. The calculation results of term similarity and new word rate, as well as the obtained semantic paragraphs, are shown in Figure 9. In this example, paragraphs 4 and 13 are selected from the maximum point of the error series of new word appearance rate, and paragraph 9 is selected from the minimum point of the error series of term similarity as the change point of the topic from a broader perspective. The paragraph is determined.

Next, the important word extraction unit 24 creates a frequency-ordered word table based on the word spirit statistics, and determines the most important words and important words of the entire sentence and the important words of each semantic paragraph. Frequency-ordered word list for the entire sentence,
Figure 9C shows the extraction results of the most important words and important words of the entire sentence and the important words of each semantic paragraph. In this example, "Competition", "Service", "Technology", "
``work'' and ``rA S K'' are selected as the most important words, and in addition to these, ``self,''``activity,'' and ``failure,'' which satisfy the condition that the cumulative usage rate is 25% or less, are selected as important words. Furthermore, important words for each semantic paragraph are determined in the same way based on the word spirit statistics for each semantic paragraph.

Next, the important part extraction unit 25 performs sentence feature analysis of each sentence.

The results of sentence feature analysis are shown in Figure 9d. In sentence feature analysis,
For each sentence, information is given such as the position of the sentence, the face of the sentence, the most important words and important words appearing for the first time and the last time, the most important word in the sentence, the meaning paragraph important word in the sentence, etc. Next, based on the result of the sentence analysis, the sentences are arranged in order of importance and the required amount of sentences is selected. The results of importance extraction are shown in Figure 9e.

In this example, 10 sentences are extracted from 48 sentences in the original text under the condition that sentences are selected until they exceed 20% of the original text.

Next, the abstract generation 11B26 rearranges the extracted importance levels in the order of the original text, and arranges them between the conjunctions at the beginning of sentences and between instructions.
Remove inappropriate items. The results of detecting conjunctions and indications to be deleted are shown in FIG. 9f. In this example, the four words ``but'', ``therefore'', ``kore ga'', and ``but'' are detected as prepositional conjunctions/demonstrations, but the latter two words are also detected in the abstract because the previous sentence in the original sentence is not included in the abstract. Since it is selected, it is not deleted, and the former two words are deleted.

Finally, the output unit 27 writes the completed abstract to an abstract file 31.

[Effects of the Invention] As explained above, the present invention extracts important words that are the core of the subject and description of a sentence with high accuracy from part-of-speech information, word frequency information, and position information in a Japanese dictionary. , based on the development of the original text, such as the problem statement and conclusion.
By extracting the importance that is important in developing the text and stating the gist of the content, and analyzing the sentence structure of the original text to the level of the semantic paragraph, information on the semantic paragraph structure can be used to identify important words and important sentences. It is used for extraction to extract important words and important sentences that describe the gist of each semantic paragraph. Also, when creating an abstract, sentences that contain important word groups common to each sentence are extracted based on the logical meaning of the original text. Since it generates an abstract by using the structure and removing unnatural connections and reference relationships, it is possible to extract important words that are the core of the text's theme and description at a compositional level, and improve the text development. The improvement effect was that it was possible to extract important sentences that were important and stated the gist of the content, and it was also possible to generate an abstract that had cohesion and consistency as a text. .

[Brief explanation of drawings]

Fig. 1 is a system configuration diagram of an embodiment of the present invention, Fig. 2 is a functional block diagram of an embodiment of the invention, Fig. 3 is a flowchart of the text structure analysis section, and Fig. 4 is used in text form analysis. An example of formal grammar and sentence structure rules, Figure 5 is a flowchart of paragraph chain analysis, Figure 6 is a flowchart of important word extraction section, Figure 7 is a flowchart of important sentence extraction section, and Figure 8 is a flowchart of abstract generation section. The flowchart in FIG. 9 is an example of the operation of the present invention. Input device Output device Processor (CPU) Program memory Document memory Working memory Word table Sentence structure table Important word table 2 Important sentence table: Abstract table: Japanese dictionary 2 Sentence structure rule dictionary 2 Input section: Morphological analysis section: Sentence structure analysis section 24: Important word extraction unit 25: Important sentence extraction unit 26: Abstract generation unit 27: Output unit 28: Document file 29: Japanese dictionary 30: Sentence structure rule dictionary 31: Abstract file

Claims (2)

[Claims] (1) An input section for inputting the original text, a Japanese dictionary that stores syntactic information and semantic information for morphological analysis, a morphological analysis section that performs morphological analysis using this Japanese dictionary, and headings and paragraphs. , a sentence structure rule dictionary that stores rules regarding the relationships between the constituent elements of sentences such as sentences, and this sentence structure rule dictionary and analysis results regarding the usage of nouns to recognize and structure the document components. A sentence structure analysis unit that analyzes logical relationships between elements; an important word extraction unit that extracts important words in a sentence using analysis results regarding the usage of nouns; and important words obtained from the important word extraction unit. an abstract generation unit that determines the presentation order of important sentences obtained from the word information and the sentence structure analysis unit and generates an abstract, and the sentence structure analysis unit further includes: , a sentence form analysis means that recognizes the constituent elements of a sentence from morphological features such as format, and analyzes the usage status of nouns for each paragraph of the paragraphs that are the constituent elements recognized by this means, and An automatic abstract generation device comprising paragraph chain analysis means for determining content relevance and segmenting sentences. (2) The abstract generation unit arranges the important sentences extracted by the important sentence extraction unit according to the frequency of appearance in the original text, and detects conjunctions and demonstratives that are inconsistent with the context of the original text in the series of important sentences. The automatic abstract generation device according to claim 1, further comprising means for deleting the information.