JP2000010986A - Retrieval support method for document data base and storage medium where program thereof is stored - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a search by displaying circumferential topics extracted under OR conditions and subject topics extracted under AND conditions when plural documents are retrieved. SOLUTION: The circumferential topics, generated by ORing cooccurrence networks of respective documents in a document group as a retrieval space, and subject topics, generated by ANDing cooccurrence networks of the respective documents, are displayed. Consequently, narrowed-down elements can accurately be shown to a retriever. The retriever further inputs narrowing-down conditions according to the narrowed-down elements to narrow down the retrieval space, and then the retrieval of document data which has few noise and high flexibility becomes possible. This system has its processes composed of a cooccurrence data base generating process 100 and a subject suggestion independent word network display process 200, and those processes 100 and 200 are performed by reading execution programs to a CPU from a large-capacity external storage device.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the use of a document database by extracting specific words from newspaper articles, technical documents, literary works, etc., and associating the words in a network format. The present invention relates to a technique that is effective when performing a search for a document.

[0002] The following methods are known as search methods for electronic data media such as networks and CD-ROMs (search space).

The first is a method in which a searcher inputs a keyword as a search condition in a search space, extracts documents matching the keyword from the search space, and narrows the search. A search engine on the Internet is also used. There are many of this kind.

[0004] Second, the search space is classified into several fields in advance, words indicating the characteristics of the documents belonging to those fields are presented from the system side, and the searcher selects the field according to necessary information. This is a method of narrowing down. This includes what is called a directory service.

A third method is a search method based on a combination of the first method and the second method, in which a search is performed using a keyword after limiting a search field in advance, and is separated from a search subject. It is effective for removing noise.

[0006] Most of the conventional retrieval methods are top-down processing, that is, a more abstract target is set and the target is gradually reduced from a wide range to a target having a lower level of abstraction (specific one). It was a way to narrow down.
On the other hand, on the other hand, there is a method of indexing a search field by generating a set of data with a higher degree of abstraction from a set of data with a lower degree of abstraction in an index manner. The generation of frequent data sets relied exclusively on operations based on human experience.

[0007]

However, none of these retrieval techniques could provide sufficient retrieval efficiency. The cause is as follows. (1) If the number of documents in the search space is not narrowed down, if the keyword is narrowed down by keyword matching, a word that does not show the content characteristic of the document will be matched with the keyword, and a document not desired by the searcher will be searched out. Will be.

(2) On the other hand, when the number of documents in the search space is narrowed down, search omission has already occurred due to a synonym, and a document intended by the searcher exists in addition to the documents belonging to the search space. There is that.

(3) It is rare that a document has only one topic, and a plurality of topics often exist. Furthermore, the way of perceiving topics varies depending on human subjectivity, and it is difficult to specify a field for one document even by using a method such as a directory service.

(4) A search key serving as a search condition for narrowing down often uses a full text, a keyword, a title, and a field, but these do not always clarify the characteristics of a document. That is, when trying to obtain an analysis result that was not intended by the searcher in advance from a collection of many documents (heuristic search), it is rather rare that the searcher can set an appropriate search key in advance. It is.

(5) In the process of narrowing down, the information that the searcher can know is often only the number of documents in the search space, and the search space is intended by the searcher only with that information. Can't judge.

(6) A document narrowed down by a search condition requires a searcher to finally read the entire text and grasp the contents to determine whether the document is necessary, and it takes time and labor. .

At present, several methods have been proposed to solve the above-mentioned problems in search. For example,
In some search engines, important words in a search space and words having a high frequency of appearance are presented from the search system side. Further, efforts are being made to present search spaces to searchers in the form of graphs and maps so as to increase the information obtained by the searchers.

In Japanese Patent Laid-Open Publication No. Hei 8-314980, the present inventor also extracts a self-sustaining word from a given document and creates a co-occurrence table for recording the co-occurring words and the number of times of co-occurrence for the independent word. A co-occurrence network display method that calculates the co-occurrence probabilities that indicate the strength of these co-occurrence relationships and displays these independent words with different links according to the co-occurrence probabilities makes it extremely easy to grasp the contents of the document. Proposed.

In the present invention, this technique is further developed,
An object of the present invention is to provide a search technology using a co-occurrence network from the viewpoint of topics covering a large number of documents.

[0016]

A first means of the present invention is as follows.
Extracting an independent word from the given first plurality of documents; creating a co-occurrence table for recording the co-occurring words and the number of times of co-occurrence for the independent words generated for each document; A peripheral topic generating step of generating a logical sum network from the co-occurrence table of each document, a central topic generating step of generating a logical AND network from the generated co-occurrence table of the document, and a logical sum including the logical AND network Displaying a network and prompting for a word to be narrowed down; narrowing down to a second group of documents containing the input word; creating the co-occurrence table with the second group of documents as the first group of documents This is a document database search support method that repeats generation of a topic topic and generation of a central topic.

From a plurality of document groups (first document group) as a search space, a logical sum of the co-occurrence network of each document is calculated to generate a peripheral topic, and a logical product of the co-occurrence network of each document is calculated. By generating the main topics and displaying them, it becomes possible to accurately show the narrowing-down element to the searcher. The searcher can further narrow down the search space by inputting narrowing conditions based on the search condition, and can search for document data with extremely little noise and high flexibility.

[0018] The second means is the first means,
As the central topic generation step, the strength of a co-occurrence pair that exists simultaneously among a plurality of documents is calculated using the mutual information amount between the plurality of documents.

That is, mutual information is defined with respect to the co-occurrence probability of a co-occurrence pair which exists simultaneously in a plurality of documents, and the mutual information indicates the strength of the connection of the co-occurrence pair, whereby a plurality of It is possible to obtain an index of how much a co-occurrence pair simultaneously included in a document group represents a central topic in the document group.

The third means is a storage medium storing the first means as a program. Here, the storage medium includes any medium that can be magnetically and optically recorded,
The form may be any form such as a disk, a tape, a memory cartridge and the like. Specifically, optical discs,
Examples include a magneto-optical disk, an IC card, and a magnetic tape.

[0021]

DESCRIPTION OF THE PREFERRED EMBODIMENTS For the purpose of understanding the present invention, a description will be given of a simplified example of extraction of an independent word, a configuration of a co-occurrence table, and a display example of a co-occurrence network.

In the present invention, the word "independent word" refers to a word that has meaning alone. For example, in the case of a sentence "I want to be a cooking teacher,""cooking,""teacher," and "naruto" are independent words. is there.

"Co-occurrence" refers to the relationship between two independent words that exist simultaneously in the same sentence. That is, when a plurality of independent words exist in the same sentence at the same time, these independent words are said to co-occur with each other or to have a co-occurrence relationship. When two independent words having a co-occurrence relation exist, one independent word is referred to as a co-occurring word with respect to the other independent word.

Next, a specific example of the co-occurrence table is shown in FIG. The figure shows an example in which a co-occurrence table is created based on two sentences: Document 1 “I want to be a cooking teacher” and Document 2 “I am a cooking school teacher”. In the figure,
“Cooking”, “teacher” and “naru” are extracted as independent words of the document 1, and “cooking”, “school” and “teacher” are extracted as independent words of the document 2. The co-occurrence words on the right side of the figure that are co-occurring with these independent words are extracted and the number of co-occurrence is calculated. For example, the independent word "cooking" is document 1.
And document 2 both exist, and for this independent word "cooking", the co-occurrence word "teacher" appears once in document 1,
Appears once in Document 2 and co-occurs twice. Therefore, the co-occurrence frequency of the co-occurrence word “teacher” with respect to the independent word “cooking” is 2.

At this time, a method of creating a co-occurrence table as one sentence by connecting document 1 and document 2 as shown in the left vertical direction of FIG.
There is a method of creating a co-occurrence table of document 2 and merging these tables to form a co-occurrence table of a plurality of documents. Absent. This will be described later. FIG. 2 shows the co-occurrence table obtained in FIG. 1 described above in a co-occurrence network. Here, since the text is very short, the calculation of the co-occurrence probability is not considered.

The "co-occurrence probability" is the probability P (cj / wi) that a co-occurring word cj for wi co-occurs with wi when a certain independent word wi appears, and is calculated by the following equation (1). It is possible. P (cj / wi) = M (cj / wi) / Nwi (1) where, wi: independent word (1 ≦ i ≦ NB, NB: total number of words in one document) cj: independent Co-occurrence word (1 ≦ J ≦ Nb (w
i), Nb (wi: the number of co-occurring words for wi) M (c
j / wi): Number of co-occurrence of independent word wi and co-occurring word cj Nwi: Number of appearance of independent word wi The co-occurrence probability calculated by the above equation (1) is the independent word wi
Is a conditional probability of occurrence of wi, and indicates a semantic connection from wi to the co-occurring word cj. Therefore, this information should be represented by a directional link such as “independent word wi → co-occurring word cj”. Can be.

Subsequently, the present invention executes a co-occurrence count expected value calculating step of calculating an expected value of the co-occurrence count. The expected value E (cj / wi) of the co-occurrence count can be calculated by the following equation (2). E (cj / wi) = Ns ｛{1- (1-Pcj) m- (1-Pwi) m + (1-Pcj) m ・ (1-Pwi) m} (2) where Ns : Total number of sentences in one document Ni: Total number of independent words m: Average number of independent words in one sentence Pcj = Ncj / NI Pcj: Probability of appearance of co-occurring word cj Pwi = Nwi / NI Pwi: Probability of appearance of independent word wi m− NI / Ns

Next, the expected value E (cj / wi) is compared with the actual number of co-occurrences M (cj / wi), and a combination of an independent word satisfying a certain condition and a co-occurring word corresponding thereto is extracted. A subject suggestion independent word group selection step is executed. As the constant condition, the following equation (3) can be exemplified. M (cj / wi)> E (cj / wi) (3)

If the above equation (3) is satisfied, the actual number of co-occurrences M (cj / w) is smaller than the expected value E (cj / wi).
i) is large, and it can be said that there is a semantic connection between the independent word wi and the co-occurring word cj. However, when the number of words NB is extremely small with respect to the total number of independent words NI in the document,
For example, when there are many repetitions with a small amount of words such as a story for children, even if the semantic connection is very weak, the combination of the independent word and the co-occurring word that satisfies the condition of the above equation (3) is Many will appear.

Therefore, considering the average number of appearances of one word, it may be determined that the co-occurrence relation is strong only when the following expression (4) is satisfied. M (cj / wi)> E (cj / wi) + αNI / NB (4) In the above equation (4), α can be obtained experimentally, for example.

Next, the relationship between the processing of this embodiment and the hardware will be described with reference to FIG. The processing of this embodiment is roughly divided into a co-occurrence database creation processing (100),
Subject suggestion independent word network display processing (200). In order to realize these processes, a 16-bit processing, preferably a 32-bit or higher CPU, a main memory, a large-capacity external storage device such as a hard disk device, and an external display / output device such as a CRT or a printer are provided. Computer system is used. The co-occurrence database creation processing (100) and the subject suggestion independent word network display processing (200) are executed by the CPU reading an execution program from a large-capacity external storage device. The input of the original text is input in advance by application software such as an editor or a word processor, and is stored in a large-capacity external storage device in, for example, a text file format.

The above co-occurrence database creation processing (10
0) and subject suggestion independent word network display processing (200)
May be realized by a single computer, or may realize a co-occurrence database creation device for realizing a co-occurrence database creation process (100) and a subject suggestion independent word network display process (200). It may be divided into a subject suggestion independent word network display device, and these devices may be connected by a communication line so that digitized document data, a co-occurrence database, and the like can be transmitted and received by communication. Further, it is natural that the data transfer between the co-occurrence database creation device and the subject suggestion independent word network display device is not limited to communication. For example, the data transfer is performed by recording on a recording medium such as an FD or a CD-ROM. Methods and the like can be exemplified.

The co-occurrence database creation process (10
0) and subject suggestion independent word network display processing (20
0) will be described. <Co-occurrence database creation process (100)> The co-occurrence database creation process (100) is based on digitized document data, and includes a co-occurrence table (TBLn), a co-occurrence probability table (TBMn), and an expected value table (TBNn). ) Is created, and this co-occurrence database (KDB) is recorded in a large-capacity external storage device (or main memory).

In this embodiment, the co-occurrence table (T
BL1), co-occurrence probability table (TBM1), expected value table (TBN1) and co-occurrence table of document 2 (TBL1)
2), the co-occurrence probability table (TBM2) and the expected value table (TBN2) are merged into a new compound document (1 to
n: Here, n = 2) co-occurrence tables (TBL1 to TBL1)
n), co-occurrence probability tables (TBM1 to TBM), and expected value tables (TBN1 to TBN) are generated.

The co-occurrence database creation processing (100)
As shown in FIG. 3, the independent word extraction process (110), the co-occurrence table creation process (120), and the co-occurrence probability calculation process (13)
0) and co-occurrence count expected value calculation processing (140).

In the independent word extraction process, an independent word is extracted from digitized document data, and the total number of independent words and the number of words in the document are calculated. The co-occurrence table creation process (120)
For each independent word extracted by the independent word extraction processing, a co-occurrence table (TBLn) for registering a co-occurring word for each independent word and its co-occurrence count is created.

The co-occurrence probability calculation process (130) calculates the co-occurrence probability of each independent word co-occurring with the co-occurring word corresponding to all independent words recorded in the co-occurrence table (TBLn). When a plurality of co-occurring words exist for one independent word, a co-occurrence probability is calculated for each co-occurring word. Further, in the co-occurrence probability calculation process (130), for each independent word,
A co-occurrence probability table (TBMn) for registering co-occurrence words and co-occurrence probabilities co-occurring with the co-occurrence word is created.

The co-occurrence count expected value calculation process (140) is performed on all independent words recorded in the co-occurrence table (TBLn).
An expected value of the number of co-occurrences of each independent word co-occurring with the co-occurring word is calculated. When a plurality of co-occurring words exist for one independent word, an expected value of the number of co-occurring times is calculated for each co-occurring word. Then, for each independent word, an expected value table (TBNn) for registering a co-occurring word and an expected value of the number of co-occurrences co-occurring with the co-occurring word is created. (Thematic Suggestion Independent Word Network Display Processing (200)) The theme suggestion independent word network display processing (200) creates a co-occurrence network based on a co-occurrence database and outputs the created co-occurrence network to a CRT, printer, or the like. This is a process of outputting to the output device as well as searching for the original text of the co-occurrence network. In this embodiment, a CRT is used as an output device.

Subject Suggestion Independent Word Network Display Processing (2
00), as shown in FIG. 7, a subject suggestion independent word group selection process (210), a co-occurrence network generation process (220),
Document search processing (230) and original text reference processing (240)
Can be divided into

The subject suggestion independent word group selection process (210)
A co-occurrence database (KDB) corresponding to the document data specified by the user, that is, a co-occurrence table, a co-occurrence probability table, and an expected value table are read from the large-capacity external storage device. Then, for all the independent words, the relationship between the independent word and the co-occurring word corresponding to the independent word (ie, the relationship between the expected co-occurrence count and the actual co-occurrence count) satisfies the above expression (4). Are determined, and a table for registering these combinations is created. Normally, the constant α in the equation (4) includes an initial value (1.5)
Is set.

The co-occurrence network generation processing (220)
The co-occurrence network corresponding to each document is created with reference to the table created by the subject suggestion independent word group selection process (210). The created co-occurrence network is displayed on the CRT screen. This co-occurrence network is based on FIG.
As shown in FIG. 2, the character string is a word indicating an independent word and a co-occurrence line connecting the character strings indicating the independent words in a co-occurrence relationship. Co-occurrence lines have different line types, colors,
Different shades, lengths, or thicknesses. When determining the line type, color, shading, length, or thickness of a link, refer to the co-occurrence probability in the co-occurrence probability table, and select the line type, color, and length of the link according to the magnitude of the co-occurrence probability. Or determine the thickness. As a method for differentiating the strength of co-occurrence,
A method in which two independent words having a co-occurrence relationship and a co-occurrence line connecting these independent words are displayed in a three-dimensional expression, and the expression form of the three-dimensional expression is changed depending on the magnitude of the co-occurrence probability can also be exemplified. Further, when the co-occurrence line is represented by shading according to the strength of the co-occurrence relation, it is possible to make only those whose co-occurrence relation is stronger than a certain degree by the user's selection substantially visible to the user. Alternatively, even weaker co-occurrence relationships can be made substantially visible to the user.

In the co-occurrence network generation process (220), when a plurality of documents are specified by the user,
A co-occurrence network is generated for each document. When the co-occurrence networks are displayed on the screen, the user can select between displaying all documents arranged side by side on one screen and displaying each document superimposed on one screen. Has become. It is also possible to change the expression level (co-occurrence level) of the co-occurrence network by changing the value of the constant α in equation (4) and re-selecting the subject suggestion independent word group.

The document search process (230) searches for one or more co-occurrence networks generated in the co-occurrence network generation process (220) when a user inputs a keyword from, for example, a keyboard or the like. , A co-occurrence network including the keyword input by the user is extracted. Here, when there are a plurality of co-occurrence networks including a keyword, these co-occurrence networks are displayed as a whole document or one document at a time depending on the selection of the user.

The original text reference process (240) reads out the original text of the document specified by the user from the document data in the mass storage device and displays it on the screen. As a procedure for designating the original text, the display coordinates of the co-occurrence network of the document desired by the user among one or two or more co-occurrence networks displayed on the screen by the document search process (230) by the position designation means. You can specify it. The position designation means
For example, it is a pointing device such as a mouse.
If the user specifies the display position of one or more arbitrary independent words on the co-occurrence network using a pointing device before designating the original sentence, the independent words are used as the second keyword of the present invention. Will be recognized as And
When displaying the original sentence, a sentence including the second keyword and sentences before and after the sentence are picked up and displayed from the original sentence. At this time, the second keyword or a sentence including the second keyword is emphasized or distinguished by a method such as hatching or reverse display. Further, when there are a plurality of sentences including the keyword in the original sentence, scroll or jump to a portion or a sentence before the currently displayed sentence among the sentences including the keyword, and You can scroll or jump to a part or sentence.

Next, when a plurality of documents are targeted, it was examined how the co-occurrence network changes depending on the logical sum (OR). (Experiment 1) 18 articles on "Escherichia coli O157" were extracted from the July and August 1996 editions of the Mainichi Shimbun, and a co-occurrence network in which all articles were connected to one document to extract themes and a co-occurrence network for each document were logically analyzed. The sum (OR) merged co-occurrence networks were compared. (Experiment 2) From an article on "O157" in Experiment 1 and an article on "Garuda Indonesia aircraft crash" extracted from the Mainichi Newspaper September and October 1997,
When multiple topics existed in a document group, the co-occurrence network was changed according to the ratio of the number of documents.

The respective results are shown in FIGS.
FIG. 4 shows a co-occurrence table created by connecting all documents of Experiment 1 and displayed in a co-occurrence network format. FIG. 5 shows a co-occurrence table created for each document in Experiment 1 and the result of merging these co-occurrence tables displayed in a co-occurrence network format. FIG. 6 shows the number of vocabulary words, the total number of independent words, and the total number of independent words. The number of documents is shown in a table format.

As a result, in Experiment 1, the same co-occurrence network was generated between the network connecting all the documents and the network merged by OR. At the same time, by ORing the co-occurrence networks,
It can be confirmed that word groups such as “O” and “157” and “food poisoning” and “Escherichia coli” representing the features of the document group of “157” are extracted.

FIGS. 7 to 9 show, in the co-occurrence network format, an article group divided into two topics in Experiment 2 with its ratio changed. FIG. 10 shows the document ratio.

In Experiment 2, it can be confirmed that by changing the ratio of the number of documents belonging to any one of the two topics, a word group representing each topic is extracted almost linearly. Thus, even if a plurality of topics exist in the document group, it can be understood that what kind of topics are present can be read from the co-occurrence network.

Next, the logical product (AN) of the co-occurrence network
Consider whether you can obtain useful information by taking D).

Assuming that the AND of the co-occurrence network is obtained from the co-occurrence table as in the case of ORing the co-occurrence network, a co-occurrence pair included in all documents can be simply obtained. These co-occurrence pairs are likely to be at the center of the documents. However, in some documents, there are co-occurrence pairs that represent the subject and co-occurrence pairs that do not represent the subject of other documents. Even if they are included in these two documents, the co-occurrence pair is the center of the two documents. I don't think it's a topic.

Therefore, it is considered whether or not an index indicating how central a co-occurrence pair obtained by taking an AND represents a central topic in a document group can be obtained.

The co-occurrence probability can be defined by the above equation (1) as the strength of the connection between two independent words having a co-occurrence relationship.

Here, attention is paid to the fact that the difference between two events can be represented by the amount of information by obtaining the mutual information from the probability between the two events.

Next, for the co-occurrence probability of the co-occurrence pair existing simultaneously between the two documents A and B, the mutual information amount can be defined as in equation (5) shown in FIG. In addition, P _A (ci / w
i) is the co-occurrence probability between the independent word wi and the co-occurring word cj in the document A, and P _B (ci / wi) is the independent word wi in the document B.
And P _AB (ci / wi) are the co-occurrence probabilities of the independent word wi and the co-occurring word cj when the document A and the document B are OR-merged.

By using the mutual information of equation (5), 2
For co-occurrence pairs existing simultaneously between documents, it is possible to indicate the strength of connection of the co-occurrence pairs between the two documents.

Therefore, the association of co-occurrence pairs that exist simultaneously between a large number of documents can be regarded as the average of mutual information of two combinations of existing documents, and this value is shown in FIG. 12 (6). It can be obtained by the formula.

It can be said that the smaller the average of the mutual information amount is, the more the co-occurrence pair indicates a more central topic.

By using the mutual information, how many co-occurrence pairs forming the AND of the co-occurrence network, that is, the co-occurrence pairs simultaneously included in the document group, represent the central topic in the document group. Index can be obtained.

As described above, the AN of a plurality of co-occurrence networks
D and OR can now be found. Since AND and OR are obtained by bottom-up processing from a document (a search processing in which a more specific object is assembled in order from an abstract object), it can be automatically calculated by a computer. AND represents the central topic of the document group, and OR must include AND. Therefore, the following topic is defined (see FIG. 13). “Central topic” is obtained by taking the AND of a plurality of co-occurrence networks, the constituent co-occurrence pairs are included in all documents, and these co-occurrence pairs are defined as “central topic”. The strength of association of co-occurrence pairs can be determined by the average of mutual information. By "ORing" a plurality of co-occurrence networks of "peripheral topics", it is possible to indicate what topics are present in the document group, and among them, AND, that is, "center topic"
Should be included, so that it is possible to know what topics other than the “center topic” exist in the document group. These topics are defined as “surrounding topics”. Further, the co-occurrence pair constituting the "peripheral topic" can be presented with the number of documents indicating how much is included in the document group.

Since the “central topic” and the “peripheral topic” have an inclusive relation, they can be displayed in the same co-occurrence network.

Focusing on the “peripheral topic”, the co-occurrence pairs are extracted from the documents belonging to the search space, and include fewer documents than the total number of documents existing in the search space. Therefore, by using "surrounding topics" as the search condition, it is possible to further narrow down as shown in FIG.

When the searcher selects the “surrounding topic” according to the required information, the “central topic” of the original search space is selected.
A new search space can be generated in which the “topic” selected by the searcher and the “surrounding topic” are set as a new “center topic”. This narrows the search space and narrows down the search space. At this time, if the searcher selects another search condition, it is possible to narrow down to another search space.

Since the co-occurrence network of the search space is obtained by the bottom-up process, the co-occurrence pair forming the "peripheral topic" always exists in the document belonging to the search space. Also,
The number of documents in which “peripheral topics” exist is certainly smaller than the number of documents belonging to the search space, and the search space can be reliably narrowed.

Further, even if a plurality of topics exist in the search space, the narrowing down of the topics can be achieved by narrowing down the topics instead of specifying the topics.

Making the "surrounding topic" selected by the searcher is as follows.
Key words in the search space are presented from the search system side, and it is also possible to reduce search omissions due to synonyms.

Further, at the time of narrowing down, matching is performed using "peripheral topics" representing the characteristics of the group of writers as keys, so that matching with words that do not show the characteristics of the document can be suppressed.

Although it has been described that the narrowing can be performed by using "peripheral topics" as a search condition, the following information may be additionally displayed during the narrowing process. “Total number of documents existing in search space” is the total number of documents presented in the conventional search, and can be used as an index of how far the searcher should further narrow down. Specifically, it may be displayed directly on the co-occurrence network screen by a number, or a box window may be opened on the screen and displayed here. As described above, it is possible to present the number of documents because it has information on which document contains the co-occurrence pair forming the “number of documents of“ peripheral topic ”” and “peripheral topic”. The number of documents in the "nearby topics" can be an index of how large the search space will be when narrowing down by selecting "nearby topics", and it is possible to know the risk (danger) as a search condition It becomes possible. This may also be displayed directly on the co-occurrence network screen by a number, or a box window may be opened on the screen and displayed here. By looking at the average of the mutual information of the "center topic" in the process of narrowing down the "average of the mutual information of the" center topic "", it is possible to know how the state of the "center topic" transitions. The searcher can determine whether the search space is intended. Further, when the “surrounding topic” newly selected as a search condition becomes the “center topic”, it is possible to determine whether the selected search condition is effective for the searcher.

As described above, it is possible to solve one of the problems in the search, that is, "there is little information that the searcher can know", and it is possible to determine whether the search space intended by the searcher is obtained. become able to. This may also be displayed directly on the co-occurrence network screen by a number, or a box window may be opened on the screen and displayed here.

Next, a search flow according to this embodiment will be described with reference to FIG.

First, documents are narrowed down from the entire search space (step 1501).

Next, a co-occurrence network of a search space newly constructed by the documents searched out is generated.
At the same time, the “central topic” and the “peripheral topic” are calculated, and the average of the mutual information amount and the number of documents are presented for each co-occurrence pair as additional information (1502).

Next, according to the information required by the searcher,
Select a co-occurrence pair that constitutes "neighborhood topics".

Here, the documents in which the co-occurring pairs selected from the search space exist are narrowed down (1503). That is, the process 2 is performed on the narrowed documents. Next, it is determined whether the narrowed search space is satisfactory or not (1504).

If the search result is not satisfied in step 1504, a backtrack occurs, the process returns to the search space before narrowing down, and the processes in step 1502 and thereafter are repeated.

Whether the searcher gives a signal to end the narrowing down,
Steps 1502 to 1504 are repeated until one document is obtained. At the end of the narrowing-down or when one document is obtained, a co-occurrence network is presented for each document belonging to the search space (1505).

[0077]

[Experimental example] A search system for specifically realizing the present invention was implemented. The functions realized in the implementation are as follows. (1) Back function and forward function These are a function to return the search processing one step before and a function to advance the search processing one step ahead. This can be specifically realized by temporarily storing a search log in a memory.

That is, when the searcher is not satisfied with the search space, the back track operation is performed, so a back function for returning to the previous search space is required. Conversely, when the backtracking is performed, the searcher may determine that the search conditions are still correct, so a forward function is also required. (2) Relocation function of autonomous node in co-occurrence network Search results are shown by displaying a visible co-occurrence network on the screen, but a function to relocate autonomous node when this network is difficult to see It is necessary to have. (3) Co-occurrence Probability Threshold Change Function Since co-occurrence pairs forming a co-occurrence network are linked by strength, the amount of information of the entire network can be changed by changing the threshold. Therefore, a function for changing the network threshold is required. (4) Others Present threshold of current search space and total number of documents, color-code "central topic" and "peripheral topic", and divide each co-occurrence pair into two equal points, average mutual information, number of documents And other functions.

FIG. 16 shows an example of a display screen provided with these functions and mounted.

Note that these implementation programs are UNI
Written in C language on X workstation.

FIGS. 21 to 25 show display screens for displaying a co-occurrence network. The screen configuration is
A window 2112 for displaying the co-occurrence network is open, and various buttons and display frames are arranged on the right side of the window 2112. Each of the buttons displayed here can be operated by coordinate indicating means such as a mouse.

The mode display unit 2102 displays a search mode (Searc
h Mode) and read mode (Read Mode). A back button 2103 (Back) is a button for returning to the previous search display, and a forward button 1204 (Forward) is a button for advancing the display screen returned by the back button and returning to the original display screen.

The rearrange button 2105 (Replace) is
A button for rearranging the position of the independent word of the co-occurrence network displayed in 112. Change button 210
6 (Change) is a button for changing the threshold value.
As shown at the right end of FIG. 2, the threshold value is changed so that the numerical value of the threshold value can be changed.

The threshold value display section 2107 is a small window for displaying the current threshold value. In FIG. 14, 3.5 is displayed as the threshold value.

The total number of documents display section 2108 is a small window for displaying the total number of documents in the search space. In the figure, 27 is displayed as the number of documents (number of files).

The mode switching button 2110 is a button for switching between a search mode and a reading mode.

Since a particular topic exists in the document group for each time period, an editorial of a television broadcast in 1995
A record of the broadcast from January to December 1997 was prepared, and a search experiment was performed.

First, the first search space was generated and narrowed down by two types of methods, namely, keyword matching and time-series co-occurrence network.

Hereinafter, the thick line segment of the co-occurrence network on the drawing indicates “center topic”, the numerical value on the line segment of the co-occurrence pair indicates the average of mutual information, and the thin line segment indicates “surrounding topic”.
And the numerical value on the line segment indicates the number of documents to be narrowed down when the “surrounding topic” is selected.

In the search space based on the above-mentioned editorial news manuscript, as a result of matching the co-occurrence pairs using the keywords “Okinawa” and “base”, 27 documents were found. FIG. 17A shows the co-occurrence network.

When different search conditions were selected for the obtained search space, the evaluations were “Okinawa” and “Problem”.
FIG. 17B shows a co-occurrence network in the case where is given. FIG. 17C shows a co-occurrence network when “day” and “rice” are given to (a).

FIG. 22 shows an interface for setting / changing the narrowing-down conditions. In the figure, the window 2112
A narrow window 2201 for narrowing down is displayed inside, and words for narrowing down are displayed.

FIGS. 18 and 19 show experimental results of retrieval from a time-series co-occurrence network. FIG.
The 1996 editorial news manuscript was categorized into four chronological series every three months from January, and a co-occurrence network for April to June was selected from among them, and further narrowed down by "democracy" and "ism" The result. FIG. 19 shows, as an evaluation of whether or not a search can be performed from a global time series, the time series is classified into four every three months in a two-year time series of 1996 and 1997. It is the result of choosing and narrowing down by "democracy" and "ism".

In each experiment, the search was narrowed down, but the "center topic" and the "surrounding topic" selected in the search space before the search were narrowed down.
However, it can be seen that the search space narrowed down to the "center topic". As a result, a new co-occurrence pair link is established like “problem” and “base” and “US Army” and “base” in FIG. 17B, and “safety” and “safety” in FIG. Results such as "surroundings" appearing in the search space before narrowing down, such as "security", were obtained. As described above, since the state of the search space is visualized by the co-occurrence network display, it is easy to determine whether the search space is intended by the searcher.

In FIG. 17, the search space generated by the selected “surrounding topic” is different, and it can be seen that the search space related to the selected “surrounding topic” is narrowed down. At that time, the average of the mutual information amounts of “Okinawa” and “base”, which are the original “center topics”, has changed as shown in FIG. 20, indicating that the search direction is not wrong. Further, the number of documents is actually narrowed down to the value presented on the line segment of "peripheral topic", so that the risk can be known when selecting a search condition.

The same can be seen in the technique of narrowing down from the time series.

Further, in a search from a group of documents having a time series, by utilizing the fact that a topic biased for each time period, which is a feature of the search, a synonym which is a problem in the search is used. A search with reduced search omission can be performed.

From the above, it can be said that this search method is efficient and effective for performing a search from a state where the search space is narrowed to some extent.

FIG. 24 shows an example of document display in the reading mode. FIG. 25 shows a state in which the co-occurrence network for each retrieved document is sequentially displayed in the reading mode.

[0100]

According to the present invention, in a search for a plurality of documents, a peripheral topic extracted under an OR condition and an AN
By displaying the central topic extracted under the D condition,
A heuristic search, that is, a flexible search including acquisition of an analysis result that the searcher did not intend in advance can be performed on a set of a plurality of documents.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining integration (merging) of co-occurrence tables according to the present invention;

FIG. 2 is an explanatory diagram showing a simple co-occurrence network

FIG. 3 is a block diagram of a search support system according to the present invention;

FIG. 4 is an explanatory view showing a co-occurrence network by combining two documents.

FIG. 5 is an explanatory diagram in which a co-occurrence network is configured by merging co-occurrence tables of two documents.

FIG. 6 is a comparison table between a case where two documents are connected and the subject is extracted and a case where the subject is extracted by merging.

FIG. 7 shows a co-occurrence network from a news editorial article (i).

FIG. 8 is a diagram showing a co-occurrence network from a news editorial article (ii).

FIG. 9 is a diagram showing a co-occurrence network from a news editorial article (iii).

FIG. 10 is a comparison table showing a merge ratio of a co-occurrence network.

FIG. 11 is an equation (5) that defines mutual information with respect to the co-occurrence probability of a co-occurrence pair existing simultaneously in two documents.

FIG. 12 is an equation (6) showing an average of mutual information of two total combinations of existing documents to show the association of co-occurrence pairs.

FIG. 13 shows a central topic (AN) from a co-occurrence network.
Explanatory diagram showing the concept of extraction of D) and surrounding topics (OR)

FIG. 14 is an explanatory diagram showing a concept of narrowing down using peripheral topics.

FIG. 15 is an explanatory diagram showing a process of narrowing down from a search space.

FIG. 16 is a diagram illustrating an example of a display screen of a co-occurrence network.

FIG. 17 is a diagram showing a co-occurrence network search from keyword matching

FIG. 18 is a diagram (1) showing a search from a time-series co-occurrence network;

FIG. 19 is a diagram illustrating a search from a time-series co-occurrence network (2).

FIG. 20 is a table showing a change in an average of mutual information of extracted words in an experimental example;

FIG. 21 is a diagram showing a display screen according to the embodiment (1).

FIG. 22 is a diagram (2) showing a display screen of the present embodiment.

FIG. 23 is a view showing a display screen according to the embodiment (3).

FIG. 24 is a diagram (4) showing a display screen of the embodiment.

FIG. 25 is a view showing a display screen according to the embodiment (5).

[Explanation of symbols]

 2101 window 2102 mode display section 2103 back button 2104 forward button 2105 rearrange button 2106 change button 2107 threshold display section 2108 total number of documents display section 2110 mode switching button 2111 end button 2112 network display section

Claims (3)

[Claims] An independent word is extracted from a given first plurality of documents, and a co-occurrence table for recording co-occurring words and the number of co-occurrences for the independent words generated for each document is created. A peripheral topic generating step of generating an OR network from the generated co-occurrence table for each document; a central topic generating step of generating an AND network from the generated co-occurrence table for each document; Displaying a logical sum network including a network and prompting input of a word to be narrowed down; narrowing down to a second document group including the input word; and defining the second document group as the first document group A document database search support method that repeats creation of a co-occurrence table and generation of peripheral topics and central topics. 2. The method according to claim 1, wherein the central topic generating step is a step of calculating the strength of a co-occurrence pair existing simultaneously in a plurality of documents by using a mutual information amount between the plurality of documents. 1. A search support method for a document database according to 1. 3. A step of extracting an independent word from a given first plurality of documents, and creating a co-occurrence table for recording co-occurring words and the number of co-occurrences for the independent words generated for each document. A peripheral topic generating step of generating an OR network from the generated co-occurrence table for each document; a central topic generating step of generating an AND network from the generated co-occurrence table for each document; Displaying a logical sum network including a network and prompting input of a word to be narrowed down; narrowing down to a second document group including the input word; and defining the second document group as the first document group A memory storing a search support program for a document database, comprising a step of repeating co-occurrence table creation, peripheral topic generation and central topic generation Body.