JPH0748217B2 - Document summarization device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION Object of the Invention (Field of Industrial Application) The present invention relates to a document summarizing apparatus capable of accurately and efficiently creating a summary sentence in which an important part of a document is extracted.

(Prior Art) Various document information is overflowing with the development of information society.
Various attempts have been made to automatically create a summary of a document for the purpose of efficiently classifying and organizing such a large amount of document information.

Then, the document summary is generally realized by taking out important parts of the document, connecting them, and putting them together in the form of a summary sentence. Therefore, it can be said that the technique for recognizing an important part of a document is a very important basic technique for creating the abstract.

By the way, as a method for recognizing an important part of a document, a method that generally focuses on the appearance frequency and distribution of words, a method that focuses on a vocabulary that represents assertions, evaluations, and opinions uses knowledge outside the language. Techniques Methods that focus on the document structure indicated by conjunctions have been proposed.

"A method that focuses on the appearance frequency and distribution of words" is
An important matter that is being claimed in a document may be used repeatedly and paraphrased with similar words, for the purpose of helping the reader understand. Based on the premise that there are many sentences, it recognizes that a sentence including a large number of words distributed / appearing over a wide range of a document is an important sentence, and collects these sentences to create a summary sentence.

In such a method, for example, it is only necessary to identify each word in the document by morphological analysis and totalize the distribution of words (words), so that the summarization processing can be realized relatively easily. However, on the other hand, a sentence that does not include a word (word) determined to be important is excluded from the extraction target for creating the summary sentence. As a result, for example, when an important word (word) is referred to by a pronoun, even if the sentence has an important meaning, it will not be used to create a summary sentence. Therefore, there is a problem that it is often the case that an appropriate summary cannot be created.

The "method of focusing on vocabulary that expresses assertion, evaluation, and opinion" is a method that is often used for summarizing editorial-like sentences (editorial texts), and is a document creator on the subject matter of the editorial text. It is based on the premise that the claims, evaluations, and opinions of are important. But with this method,
For example, opinions and opinions such as "I think ..." and "I think ..."
A sentence containing a vocabulary or a specific expression used when making a guess is judged and extracted as an important sentence, and a summary sentence is created.

However, it is very difficult to prepare in advance all the vocabulary and specific expressions used when describing important matters. For this reason, there is a problem that a sentence that is originally important but not extracted for creating a summary sentence appears, or a sentence that is not so important but included in the summary sentence appears. It is easy to occur.

Furthermore, the “method of utilizing knowledge outside the language” is not limited to linguistic knowledge such as part-of-speech information and grammar of each word used for sentence analysis, and what kind of concept given to the word in the real world Document summarization is performed by using knowledge that has a property, that is, knowledge other than linguistic knowledge. More specifically, rather than literally summarizing a sentence, information that describes a certain matter is extracted and organized from a document, and this is extracted according to the purpose and interest of the user, for example, in a table. It is used for applications such as outputting in format.

Such a method is very effective for document summarization in a specific field in which knowledge is prepared in advance. However, it is not only extremely difficult to prepare such a large amount of non-language knowledge, but it is also very difficult to hold that knowledge. In addition, there is a problem that the time required for the processing is very long and it is very difficult to realize it at the current technical level.

On the other hand, the “method of focusing on the document structure indicated by the conjunction, etc.” seeks the rhetorical structure between a plurality of sentences based mainly on the conjunction located at the beginning of the sentence.
When two sentences or a group of sentences are connected in a rhetorical relationship, one seeks which sentence is relatively important. Specifically, when two sentences are connected in a “forward” relationship by the connective “because”, the part (sentence) located after is detected as an important part. Also, when two sentences are connected by a "why" connective in a "reason" relationship, the sentence located before is detected as an important part.

By obtaining the sentence structure based on such a rhetorical relationship between sentences, it becomes possible to sequentially obtain sentences (parts) with higher importance from the structure. Moreover, in order to obtain the above-mentioned structure, it is necessary to recognize rhetorical relations between sentences, but in order to find rhetorical relations, it suffices to focus on connectives, and the number of these connectives is relatively limited. Therefore, it is possible to more accurately extract an important part in the document as compared with the other methods described above.

However, in some cases, there are few connectives that indicate rhetorical relationships between sentences, and as a result, it is often impossible to judge the importance. Moreover, if the relative importance of the two sentences or their groups connected by rhetorical relation is different from the predetermined one, there is a problem that it cannot be processed correctly. There is a problem in creating a highly accurate summary using only this method.

(Problems to be Solved by the Invention) As described above, in the various methods of creating a summary sentence that have been proposed in the past, since the recognition accuracy for extracting an important sentence from a document is poor, In many cases, the content of the sentence was significantly different from the original content of the document, and it was very difficult to obtain a highly accurate summary sentence.

The present invention has been made in consideration of such circumstances, and an object of the present invention is to improve recognition accuracy for determining the degree of importance of a sentence that constitutes a document and to provide a high-quality summary with a high content. An object is to provide a document summarizing device capable of creating a sentence.

[Configuration of the Invention] (Means for Solving the Problem) A document summarizing device according to the present invention analyzes a sentence constituting a document to be summarized and detects character modification information attached to the sentence. Information about the position in the sentence that indicates which part of the sentence the character modification information is attached to, which is used for comparison and collation with the character modification information to identify an important part of the document. The knowledge storage means for storing knowledge consisting of a numerical value representing the degree of importance set corresponding to information, the summary result using the analysis result by the sentence analysis means and the knowledge stored in the knowledge storage means. And a summary sentence creating unit that extracts a sentence from the document to be summarized to create an important sentence according to the determination result by the determining unit. By sentence creation means Creating Yakubun is singled out high numerical statement representing the importance in the required range of characters as summary in order, and performing by concatenating singled out statements in accordance appearance order.

(Operation) In the present invention, the sentence analysis means configures the document to be summarized and detects the character modification information attached to the sentence,
Position information indicating which part in the sentence the character modification information is attached, and this position information, which is provided for comparison and collation with the character modification information for identifying an important part in the document in the knowledge storage means. Knowledge that is stored in correspondence with the numerical value that represents the degree of importance set in correspondence with each other, and the determination means uses the analysis result of the sentence analysis means and the knowledge stored in the knowledge storage means to determine the importance of the document to be summarized. And a summary sentence creating means creates an important sentence by extracting a sentence from the document to be summarized according to the determination result by the determining means.

Then, the summary is created by the summary creating means by sequentially selecting sentences having a high numerical value representing the degree of importance within the range of the number of characters required as the summary and connecting the selected sentences in the order of appearance.

(Embodiment) Hereinafter, a document summarizing apparatus according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of the embodiment apparatus, and 1 is a sentence input unit. The input unit 1 has a buffer memory function for storing document information created by a keyboard or the like or a memory function for inputting document information created by another device and storing and holding the document information. It is assumed that the document image is input, the document information is recognized, and the document information encoded is stored as the sentence input unit 1.
Can also be configured.

The sentence analysis unit 2 individually extracts sentences constituting the sentence from the document information input from the sentence input unit 1 by using, for example, a punctuation mark, a period, a space, a paragraph, and the like, and extracts each of these sentences as a morpheme. To analyze. At this time, the sentence analysis unit 2 uses the character modification information attached to the document information provided for the morphological analysis, for example, an underline attached to a character, a shaded area,
Information such as emphasis and blanking is detected at the same time.

The character modification information detected in this way is given to the important sentence determination unit 3 together with the result of morphological analysis of the sentence, and the knowledge information about the important sentence stored in advance in the knowledge storage unit 4, that is, the important sentence is displayed. It is provided for comparison and collation with the character modification information 5 that can be specified. The important sentence determination unit 3 extracts a sentence including a character to which the character modification information is added as a highly important sentence according to the matching with the knowledge information, and
The above-mentioned conventional general important sentence extraction method is appropriately adopted to extract a highly important sentence in the document.

The summary sentence creating unit 6 inputs the highly important sentences extracted by the important sentence determining unit 3 as described above, creates a required sentence by connecting these sentences in the extraction order, and outputs this. To do.

Here, the character modification information attached to the characters of the sentences forming the document is given, for example, in the format as shown in FIG. For example, the upper 4 bits are used to specify the type of character decoration such as underlining, shading, emphasis, and hollowing, and the remaining lower 4
The format is configured so that a range of character decoration can be specified using bits, up to a maximum of 16 characters. The character decoration information provided in such a format is embedded in the code string of the characters forming the sentence, and is arranged in the document following the character decoration information designation symbol as shown in FIG. 3, for example.

That is, the document information is given in an internal representation format as shown in FIG. 3 (a), for example. This example shows the character code in hexadecimal notation, "This is an underline. Is given by the code string "a4b3a4ec ... a1a3FF".

Then, the character decoration information is the character decoration information designation symbol "01".
Is inserted into the character code string described above such as "82". By such character modification information, the character "under, line" indicated by the character code "b2bc, c0fe" in the sentence is underlined. As a result, the sentence to which such character modification information is added is displayed or printed out in a state where the character "underline" is underlined, as shown in FIG. 3B, for example.

Now, the sentence analysis unit 2 which analyzes the digitized document information including the character modification information as described above performs a morphological analysis on each sentence in the document one by one. At this time, the paragraph number in which the sentence appears and the sentence number in the paragraph are sequentially registered in the pointer table 2a in the sentence analysis unit 2 as shown in FIG. 4, for example. Also, the result buffer 2b in the sentence analysis unit 2
In the above, the word recognized by the above analysis process and the type of character modification added to the word are registered as a pair for each clause. Here, in the "pointer" column of the pointer table 2a, the subscript of the first element of the word information in the buffer 2b is substituted, and in the "pointer" column of the next element of the pointer table 2a, the word information Substitute the value obtained by adding 1 to the subscript of the final element. For example, the first element of the pointer table 2a in FIG. 4 is "this" and "underline".
In order to indicate that it is composed of two clauses, the subscript (for example, 1) of the buffer 2b in which the first word is stored is assigned to the "pointer" field of the first element, and the second element of the pointer table 2a In the "Pointer" column, the above subscript +2
(Ie 3) is substituted. When all the sentences have been analyzed, 0 is assigned to both the paragraph and the sentence number as in the third element of the pointer table 2a in FIG. If the analysis fails and no words can be extracted even though the sentence has been parsed, there is no word information that can be registered in the buffer 2b. The value of the column is the same as the value of the "pointer" column of the next element. Further, as will be described later, in order to check the importance of the sentence 1, it is necessary to refer to a word included in the sentence. This includes the element of the pointer table 2a pointing to the sentence 1 and the next element. It can be retrieved by referring to the value in the "pointer" column. Therefore, in determining the degree of importance, the values in the "pointer" column of the elements of sentence 1 and the next sentence (that is, sentence I + 1) are assigned to variables P1 and P2, as described in FIG. Refers to the word that appears in.

At this time, as the character decoration information in the result buffer 2b, the upper 4 bits of the character decoration information in the format shown in FIG. 2 are used. In addition, even if character modification is applied to only a part of the word, it is considered here that character modification is applied to all of the word, and the result buffer
In 2b, character modification information is added to the word.

The importance item in the pointer table 2a stores the importance determination result for each sentence determined by the important sentence determination unit 3.

Now, the knowledge information 5 for recognizing an important sentence registered in advance in the knowledge storage unit 4 includes items such as "type", "contents", "position information", and "importance" as shown in FIG. It is given as information to have. The content of knowledge described in the "content" item is described as importance judgment knowledge based on expressions such as vocabulary when the "type" item is "2", and the "type" item is " When it is 1 ", it is described as importance determination knowledge based on the character modification information.
When the “type” item is “1”, the knowledge based on the character modification information described in the “content” item is similar to the character modification information in the result buffer 2b described above, and the knowledge of the upper 4 bits described above is used. It is given as hexadecimal information corresponding to the information.

Further, the content described in the item of "positional information" is used to specify which part of the sentence should be focused when the condition described in the item of "content" is determined. For example, the beginning of the sentence, the end of the sentence if "1", any position in the sentence if "2", and the whole sentence if "3".

Further, the item of "importance" stores information indicating the degree of importance expressed as a numerical value in the range of [-1] to [1], for example. According to the numerical value indicating this importance, for example, the closer the value is to [1], the higher the importance is,
The closer to [-1], the lower the importance.

Knowledge information about the importance described in such a format is stored in the knowledge storage unit 4 in a table format as shown in FIG. 6, for example. In the example shown in FIG. 6, from the information in the first column, “if all the words in one sentence are underlined, the importance of the sentence is [1]. ], And from the information in the second column, “If any word in the sentence is highlighted, the importance of that sentence is [0.7]. ] Is shown. When the value of the “type” is 2, that is, when the importance judgment knowledge is based on expressions such as vocabulary, the “content” column shows the knowledge identification numbers in the various methods already proposed. Etc. should be described.

By comparing such knowledge information with the above-mentioned analysis result, the important sentence determination unit 3 obtains the importance of the sentence for each sentence obtained from the input document, and the information of the importance is described above. It is described in the "importance" item of the pointer table 2a. Then, the abstract sentence creating unit 6 pays attention to the item of "importance" of the pointer table 2a, extracts the sentences of high importance, and combines these sentences to create the abstract sentence.

FIG. 7 shows a procedure flow of the important sentence judgment processing in the apparatus (important sentence judgment unit 3) of this embodiment.

This processing procedure is started by initializing the control parameter I for sequentially referring to the above-mentioned pointer table 2a for each sentence to [1] (step a). Then, the importance of the pointer table 2a designated by the control parameter I is set to [0], and the variable P1 is set to the value in the "pointer" column of the element of the pointer table 2a designated by the control parameter I. , And the control parameter (I +
The values in the "pointer" column designated in 1) are set (step b).

The variable P1 is for designating the first element (clause) of the analysis result for the sentence specified by the control parameter I in the result buffer 2b in which the analysis result of the sentence is registered. The variable P2 is for designating the first element (clause) of the analysis result for the sentence following the sentence designated by the control parameter I.

By comparing the values of the variables P1 and P2 set in this way with each other, for example, when the variables P1 and P2 are equal, the degree of importance regarding the sentence is increased due to the reason that the analysis process of the sentence fails. It is detected that the element to be determined does not exist (step c). In this way, the control parameter I
When it is detected that the element that determines the importance does not exist in the sentence specified by, the control parameter I is incremented (step d), and the paragraph of the pointer table 2a indicated by the control parameter I is displayed. After confirming that the value is not "0", the procedure returns to the above-described processing procedure from step b, and the processing procedure based on the next control parameter I is similarly started (step e).

However, if it is confirmed in the above-mentioned variable check that an element for determining the degree of importance exists in the sentence, then control for sequentially referring to the knowledge 5 registered in the knowledge storage unit 4 is performed. The parameter J is initialized to [1] (step f). After that, the knowledge 5 in the knowledge storage unit 4 referred to by the control parameter J as the variable K
Step g) of setting a value indicating the storage position of That is, when the knowledge in the knowledge storage unit 4 is as shown in FIG. 6, and when the value of the variable J is 1, the variable K is
The value in the "positional information" column of the element, that is, 3, is substituted. After the initialization for such knowledge reference, the value of the above-mentioned position information shown in the variable K is any of [0] [1] [2] [3] having the meaning as described above. It is determined whether there is any (step h, i, j).

If the value indicated by the variable K is [0], the content of the knowledge is valid only at the beginning of the sentence, so only the content of the result buffer 2b designated by the variable P1 is examined. (Step k). And the result buffer
If the content of 2b does not satisfy the condition shown in the important sentence determination knowledge, the control parameter J is incremented (step r) to check whether the condition is satisfied similarly for the next knowledge, and all knowledge is acquired. This is repeated until the check for is completed (step s). As in the first and second elements of FIG. 6, when the value of the “type” of knowledge is 1, the knowledge is the important sentence determination knowledge based on the character modification information as described above. . In this case, in order to judge whether the element of the buffer 2b satisfies the condition indicated by the knowledge, the value of the "content" of the important sentence judgment knowledge is compared with the value of the "character modification" of the buffer 2b. Good. Further, it is assumed that the number of pieces of knowledge stored in the knowledge storage unit 4 is substituted for the variable T in step s.

On the other hand, when the content of the result buffer 2b satisfies the condition indicated by the important sentence judgment knowledge, the importance of the important sentence judgment knowledge specified by the control parameter J and the importance of the important sentence judgment knowledge are specified by the control parameter I. If the importance value on the judgment knowledge side is greater, the importance value for that sentence is changed to the above importance value. The value is updated using the value of the importance indicated by the sentence judgment knowledge (step q).

On the other hand, if the value indicated by the variable K is [1],
Since it is shown that the content of the knowledge consists of only the last clause of the sentence, the result buffer 2b specified by the value [P2-1] obtained by subtracting [1] from the value specified by the variable P2.
Check only the contents of (step m). Then, according to the contents of the result buffer 2b, the same processing procedure as in the case where the value indicated by the variable K is [0] is performed, and the value of the degree of importance for the sentence is obtained. That is, if the content of the buffer 2b designated by the value -1 of the variable P2 does not satisfy the condition of the important sentence judgment knowledge referred to by the variable J, the process proceeds to step r, and if the condition is satisfied, the step p Proceed to.

Similarly, when the value indicated by the variable K is [2], it is checked whether or not there is a clause for which the knowledge is satisfied in the sentence designated by the control parameter I (step n),
When the value indicated by the variable K is [3], it is checked whether or not the knowledge is satisfied for all the clauses forming the sentence specified by the control parameter I (step o).

In this way, whether or not the sentence specified by the control parameter I satisfies the condition indicated by the important sentence judgment knowledge is sequentially checked according to the judgment condition of each knowledge, and the value of the importance degree for each sentence is obtained. After that, by mutually judging the value of the importance calculated for each of those sentences,
For example, in the range of the number of characters required as a summary sentence,
A sentence having a high degree of importance is selected in order, and these sentences are connected according to the order of appearance thereof to create a summary sentence.

Thus, according to the present apparatus configured as described above, in addition to the conventionally-proposed various important sentence determination methods, attention is paid to the character modification information added to the characters that form the sentence, and the character modification is performed. Since the important sentence is extracted from the document while recognizing the sentence including the existing characters as the sentence having the high importance, it becomes possible to easily and effectively select and extract the sentence having the high importance. Moreover, since the importance of each sentence is recognized and judged while weighting the importance according to the knowledge information, it is possible to select and extract the sentence of high importance accurately. As a result, it is possible to effectively and automatically create a highly accurate and accurate summary sentence.

The present invention is not limited to the above embodiment. For example, it goes without saying that the above-mentioned various conventionally-proposed methods for determining important sentences may be used in combination, and these may be comprehensively determined to extract the important sentence. Further, the knowledge information used for recognizing and determining an important sentence is not limited to the above-mentioned example, and its condition, content, etc. may be determined according to the specifications. Other,
The internal representation format of the document to be summarized can be modified in various ways, and the point is that it can be modified in various ways without departing from the scope of the invention.

[Effects of the Invention] As described above, according to the present invention, the creation of a summary sentence is
In the range of the number of characters required as a summary sentence, a sentence with a high numerical value indicating the degree of importance (the degree of importance set corresponding to the position information indicating where the character modification information is attached in the sentence) is selected in order. Since the selected sentences are connected according to the order of appearance, it is possible to effectively create a high-quality summary sentence with rich contents.

[Brief description of drawings]

FIG. 1 shows a document summarizing apparatus according to an embodiment of the present invention. FIG. 1 is a block diagram showing a schematic configuration of the embodiment apparatus, and FIG. 2 is an example of a format of character decoration information added to a character. Fig. 3 is a diagram showing an internal representation format of document information and its output form, Fig. 4 is a diagram showing an example of a sentence analysis result, and Fig. 5 is a representation of important sentence knowledge used for important sentence determination. FIG. 6 is a diagram showing an example of a format, FIG. 6 is a diagram showing an example of important sentence determination knowledge, and FIG. 7 is a diagram showing a flow of important sentence determination processing. 1 ... Sentence input unit, 2 ... Sentence analysis unit, 3 ... Important sentence determination unit, 5 ... Knowledge storage unit, 6 ... Knowledge information regarding character modification information, 6 ... Summary sentence creation unit.

Claims (1)

[Claims] 1. A sentence analyzing unit for analyzing a sentence constituting a document to be summarized and detecting character modification information attached to the sentence, and a character for identifying an important part in the document. Knowledge that consists of position information indicating where in the sentence the character modification information is attached and a numerical value indicating the importance set corresponding to this position information, which is used for comparison and verification with the modification information The stored knowledge storage means, the determination means for determining an important sentence in the document to be summarized by using the analysis result by the sentence analysis means and the knowledge stored in the knowledge storage means, and the determination means And a summary sentence creating means for creating an important sentence by extracting a sentence from the document to be summarized according to the determination result, wherein the summary sentence creating means creates the summary sentence with the number of characters required as the summary sentence. Before in range A document summarizing device characterized in that sentences having a high numerical value representing a memorandum importance are selected in order and the selected sentences are concatenated in the order of appearance.