JP2012128509A - Conception processing apparatus and program - Google Patents

Abstract

An object of the present invention is to obtain an appropriate component of how an explanation should be configured when explaining contents related to an event.
An extraction unit 3 of a conceptual processing device 1 extracts item name data indicating a table of contents from article data explaining a certain type of event, and the number of items that each extracted item name data appears in article data. Count. The generalization processing unit 4 compares the extracted item name data, and if part of the item name is duplicated or has a higher concept, the item name data is integrated to obtain news element candidate data. Further, the number of appearances of each news element candidate data is calculated by the total number of appearances of the item name data integrated with the news element candidate data. The ontology generation unit 5 selects news element candidate data having a greater number of appearances than a predetermined number as news element data, generates an ontology with the event type data as the upper hierarchy, and the news element data as the lower hierarchy, and the ontology storage unit Write to 8.
[Selection] Figure 1

Description

  The present invention relates to a conceptual processing device and a program.

  With the appearance of news sites on the Internet, users can search and browse news articles about various events such as incidents and accidents (hereinafter referred to as “news events”). However, when a user wants to get an overview of a news event, the user searches for news articles related to the news event using an appropriate keyword, and browses the news articles obtained as a result of the search one by one. So, I needed to reorganize the contents of the article and get an overview.

  On the other hand, there are no commentary on words on sites that accumulate “collective intelligence” such as Wikipedia (registered trademark) (http://www.wikipedia.org/, Wikimedia Foundation), which has been expanding in recent years. Not only that, it also includes commentary on news events. Users have been able to get an overview of news events by reading the commentary contained on such sites. For example, if an article provided by the collective intelligence storage site regarding the “Niigata Chuetsu-oki Earthquake” is read, the user can obtain an overall picture of the earthquake.

  However, even such sites do not exhaustively describe all news events, and only important news events that are likely to attract many users are included as articles. Therefore, regarding news events not included in the site, the user can search for news articles and the like by himself / herself and know the outline as before. Therefore, if a “summary article” regarding a certain news event can be automatically generated from a huge amount of news articles, the burden on the user is expected to be greatly reduced. Non-Patent Document 1 proposes the concept of news ontology in order to realize such a technique.

  FIG. 10 is a diagram showing the news ontology. In this technology, it is assumed that a group of news articles that convey information related to a certain news event has a specific role. For example, articles F1 and F2 are text information belonging to a news event of the type “earthquake disaster” (hereinafter, the type of news event is described as “news event class”), and this news event class “earthquake disaster”. Suppose that it has a role to show the contents of “damage”. Such a role is referred to as a “news element”, and features of various types of news events are represented by a set of the news elements.

  Each news element is expressed by a set of words (or superordinate concepts of the words) included in the news element. That is, for each news event class, a news element group constituting the news event class is described, and a characteristic word (concept) group corresponding to the news element is described as an ontology. For example, as shown in FIG. 10, the news event class includes “corruption case”, “earthquake disaster”, “aircraft accident”, and the like, and the news event class “earthquake disaster” includes the news elements “occurrence overview”, “ It consists of “damage”, “response”, “cause”, and “after”. The news element “Outbreak Summary” includes the word “seismic intensity”, the news element “damage” includes the word “collapse”, and the news element “cause” includes the word “plate”. It will be structured. A word included in each news element is a word that frequently appears in an article associated with the news element. Patent Document 1 describes a technique for extracting an important keyword in a document.

  By creating such a news ontology, it can be used for services such as extracting only articles that correspond to a specific news element, or combining multiple articles for each news element to create a collective article. It is.

JP 2010-204866 A

Masaru Miyazaki et al., “Examination of News Ontology Construction Method Using External Knowledge”, 2009 Video Information Media Society Winter Conference Proceedings, Presentation Nos. 1-4, 2009

  When explaining news events such as incidents and accidents to people, for example, when explaining “earthquake disasters”, “Outline of occurrence”, “damage”, “response”, “cause”, “after” shown as news elements It is easier to understand if the contents are divided into categories such as However, the news element group of the news event class “earthquake disaster” does not match the news elements explaining the news event class such as “corruption case” and “aircraft accident”. In this way, the news elements differ depending on the individual news event class. In the technique of Non-Patent Document 1 described above, a news element group has to be manually determined for each of a huge number of news event classes. Therefore, it requires a tremendous amount of time and labor.

  The news elements constituting the news event class can be considered as important keywords included in the document described about the news event class. Many methods for acquiring important keywords in a specific document have been examined. For example, a group of words in a document considered to be important is extracted by an important keyword extraction method using word frequency information such as TF-IDF as an index. It is easy to do. However, the importance of the word obtained by this does not necessarily accurately reflect the intention of the document creator in the explanation. In other words, the importance of a word represents structured knowledge that describes what kind of information should be presented and how it should be presented when communicating a news event. is not.

  The present invention has been made in consideration of such circumstances, and a concept processing apparatus capable of obtaining appropriate components of how to explain the contents related to an event, Provide a computer program.

[1] One aspect of the present invention is an article data storage unit that stores article data including heading data, an extraction unit that reads out the article data from the article data storage unit, and extracts the heading data, and the extraction unit When the characters appearing in the plurality of heading data extracted by have a common appearance pattern, the combined heading data is generated based on the common appearance pattern, and the plurality of heading data extracted by the extraction unit are common A generalization processing unit that outputs, as component data, heading data as a result of performing generalization processing for generating heading data after integration based on the common superordinate concept when having a concept It is a conceptual processing device.
According to this aspect, the conceptual processing device can be used as a table of contents or a headline such as a news title or a lead sentence from a plurality of article data in which contents related to an event of the same type are described. Data is acquired, and integration is performed when the headlines indicated by the acquired headline data are superficially matched or have the same superordinate concept. The conceptual processing device outputs the heading data as a result of the integration as data indicating the constituent elements for explaining the event.
Based on knowledge of what kind of structure is used to convey an event using article data on an event provided by a collective intelligence storage site or news article data. You can get the component. Further, by using the acquired component elements for classification of articles and the like, it is possible to use them for generating a summary article about the event.

[2] One aspect of the present invention is the conceptual processing device described above, wherein the heading data integrated with the component data is added to a lower layer of the component data obtained by the generalization processing unit. Ontology or ontology data obtained by adding lower-layer heading data of the constituent element data included in the hierarchical heading data indicated by the article data to the lower-layer of the constituent element data obtained by the generalization processing unit It further comprises an ontology generation unit for generating.
According to this aspect, the conceptual processing device adds, to the lower layer of the component, the heading data that is deleted because it is integrated with the component data, or the content obtained from the table of contents of the hierarchical structure included in the article data. Layered ontology data is generated by adding heading data in the lower layer of the component.
Thereby, while being able to obtain a detailed component, the relationship between those components can be classified easily.

[3] One aspect of the present invention is the conceptual processing device described above, wherein the order of the component data obtained by the generalization processing unit is determined based on the appearance order of the heading data indicated by the article data. The apparatus further includes an ontology generation unit that generates ontology data in which information indicating the determined appearance order is added to the component element data.
According to this aspect, the conceptual processing device generates ontology data in which the component data is arranged in order based on the appearance order obtained from the article data.
Thereby, when explaining a certain event, it is possible to know in what order the explanation should be structured.

[4] One aspect of the present invention is the conceptual processing device described above, wherein the generalization processing unit is configured such that, in the generalization processing, the headings indicated by the plurality of heading data match backward, match forward, or partially If there is a match, or if the number of characters commonly appearing in the same order in the headings indicated by the plurality of heading data is greater than or equal to a predetermined number, the plurality of heading data is the shortest heading of the plurality of heading data. The heading data is integrated into the heading data, and the heading data having a common superordinate concept is integrated with the heading data indicating the superordinate concept, and meets a predetermined condition for determining that the heading data is inappropriate as a component. The heading data to be deleted is deleted.
According to this aspect, the conceptual processing device, when a heading of a certain heading data is included as a part of the heading of other heading data, or when the number of characters that appear in common in the heading of the heading data is large, Are integrated into the heading data of a short heading, and the heading data of the heading having a common superordinate concept is integrated into the heading data having the superordinate concept as a heading. In addition, the conceptual processing device deletes heading data of a headline that is highly likely to be specialized in the content of article data, such as a heading including a numerical value or a heading with a large number of characters.
As a result, it is possible to obtain a component expressed in general words that can be easily used as a keyword when searching for or classifying articles.

[5] In one embodiment of the present invention, an article data storage unit that stores article data including heading data is read out from the computer used as the conceptual processing device, and the article data is read from the article data storage unit, and the heading data is extracted. When the characters appearing in the plurality of heading data extracted by the extraction unit have a common appearance pattern, the integrated heading data is generated based on the common appearance pattern, and the extraction unit extracts When the heading data has a common superordinate concept, a generalization processing unit that outputs the heading data as a result of performing generalization processing for generating the heading data after integration based on the common superordinate concept, It is a program characterized by functioning.

  According to the present invention, when explaining contents related to an event in order to explain an event, it is possible to obtain an appropriate component as to how the explanation should be configured.

It is a functional block diagram of the conceptual processing apparatus by the 1st Embodiment of this invention. It is a figure which shows the example of the article data used for the embodiment. It is a figure which shows the processing flow of the conceptual processing apparatus by the embodiment. It is a figure which shows the integration process using the surface information by the same embodiment. It is a figure which shows the integration process using the semantic information by the embodiment. It is a figure which shows the deletion process by the embodiment. It is a figure which shows the example of multilayered ontology data by the 2nd Embodiment of this invention. It is a figure which shows the example of ontology in which the context between the news elements by the 4th Embodiment of this invention is included. It is a figure which shows the example of the article data used for the 5th Embodiment of this invention. It is a figure which shows the example of news ontology data in a prior art.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[1. First Embodiment]
FIG. 1 is a functional block diagram showing the configuration of the conceptual processing device 1 according to the first embodiment of the present invention. The conceptual processing device 1 can be realized by a computer device such as one or a plurality of servers. The conceptual processing device 1 includes an article data acquisition unit 2, an extraction unit 3, a generalization processing unit 4, an ontology generation unit 5, an article data storage unit 6, a thesaurus storage unit 7, and an ontology storage unit 8. .

  The article data acquisition unit 2 acquires article data obtained by converting a news article about an ontology generation target news event class into data, and writes the article data in the article data storage unit 6. A news event is an event such as an incident or accident, and a specific type of news event is called a news event class. An ontology is a correspondence between a news event class and a news element of the news event class. The news element is a constituent element of the explanation about the news event class, and the contents of explanation are classified into categories. The article data acquisition unit 2 may read article data from other servers connected via a public network such as the Internet or a private network such as an intranet, and may read article data from a computer-readable recording medium. You may read.

  The article data is semi-structured text information, and includes structured text information and unstructured text information. The structured text information includes text data describing items such as chapters and sections, and style data indicating the character size, font, display position, and the like of the text data. The item is a group of explanations about the events classified by contents, and the item includes an item name which is a heading attached so that the contents can be easily understood. The unstructured text information includes text data describing an explanation of items such as chapters and sections indicated by the structured text information, and style data of the text data. When the article data is HTML (Hypertext Markup Language) data, the style data corresponds to a tag. Note that the article data may include only structured text information, or may include other information such as image data.

  The extraction unit 3 extracts item name data indicating an item name as heading data from the structured text information in the article data stored in the article data storage unit 6, and extracts each item from the acquired article data The number of article data in which name data appears in structured text information is counted. The extraction unit 3 outputs the extracted item name data to the generalization processing unit 4 in association with the number of appearances counted for the item name data.

  It is preferable that the generalization processing unit 4 compares the item name data input from the extraction unit 3 and integrates item name data indicating duplicate item names or item names having similar meanings or uses them as news elements. Perform generalization processing such as deleting item name data for non-existing item names. When integrating the item name data, the generalization processing unit 4 adds up the number of appearances of the item name data to be integrated to obtain the number of appearances of the item name data after integration. The generalization processing unit 4 refers to the concept dictionary data stored in the thesaurus storage unit 7 when integrating item name data having the same meaning. The generalization processing unit 4 outputs the item name data remaining as a result of the integration processing and the deletion processing to the ontology generation unit 5 together with the number of appearances as news element data candidates.

  The ontology generation unit 5 extracts the news element data constituting the ontology from the news element data candidates input from the generalization processing unit 4 based on the number of appearances. The ontology generation unit 5 generates ontology data indicating the ontology of a tree structure with the news event class to be generated as an ontology being the upper hierarchy of the hierarchical structure and the news element indicated by the extracted news element data being the lower hierarchy, and the ontology storage unit Write to 8.

  The article data storage unit 6, the thesaurus storage unit 7, and the ontology storage unit 8 are realized by a hard disk device, a semiconductor memory, or the like. The article data storage unit 6 stores the article data acquired by the article data acquisition unit 2. The thesaurus storage unit 7 stores concept dictionary data indicating a correspondence between a higher concept word and a lower concept word in a tree structure or the like. The conceptual dictionary data may have a network structure. The ontology storage unit 8 stores ontology data generated by the ontology generation unit 5.

Subsequently, the operation of the conceptual processing device 1 of the present embodiment will be described. In the present embodiment, a news element group is extracted using article data including structured text information edited manually. In other words, items such as a table of contents used by the conceptual processing device 1 are manually edited, and therefore use knowledge of how humans compose explanations when explaining events in a news event class. And a news element group can be extracted.
As article data, HTML (Hypertext Markup Language) document data provided by a collective intelligence storage site, for example, Wikipedia (registered trademark) is used.

  FIG. 2 is a diagram showing an example of an article indicated by article data used by the concept processing device 1, and shows an excerpt of an article provided by Wikipedia (registered trademark) regarding the “Niigata Chuetsu-oki Earthquake”. FIG. 2A shows the display of items indicated by the structured text information included in the article data, and FIG. 2B shows the item names and unstructured text information included in the items indicated by the structured text information. This is a display (excerpt) of the text shown.

  As shown in FIG. 2A, the text data included in the structured text information is divided into a plurality of items such as chapters and sections and the contents thereof are described. The item is a description of the event as shown in FIG. 2B, which is classified for each group of contents, and includes an item name (title). These item names can be considered as indicating the news elements necessary for explaining the news event class to which the article data belongs. In other words, in Fig. 2 (a), the news event class "Earthquake" conveys its contents by item name items such as "Summary", "Cause", "Damage", "Government / Local Government Response", etc. These item names can be regarded as news element candidates.

  However, the content of all earthquake-related article data is not composed of such items. Furthermore, there are earthquakes that do not have article data with such content. Therefore, in the present embodiment, news elements are extracted from item names that appear in common in a plurality of articles. In this way, candidates corresponding to news elements are extracted from the structure of text information manually edited, and news elements are extracted by generalizing them.

  The article data is associated with a hierarchical category in advance, for example, the upper class “earthquake” and the lower class “Japan earthquake”. Therefore, in order to collect article data belonging to a specific news event class, it is possible to perform a keyword search or use category information associated with the article data.

FIG. 3 is a diagram illustrating a processing flow of the conceptual processing device 1 of the present embodiment.
First, the article data acquisition unit 2 of the conceptual processing device 1 acquires article data belonging to a specific news event class (step S10). For example, the article data acquisition unit 2 reads the article data A1, A2,... Associated with the news event class C1 “earthquake” from the server of the collective intelligence storage site connected via a network such as the Internet. There is no limit on the number of article data to be acquired. The article data acquisition unit 2 outputs the acquired article data to the extraction unit 3.

  The extraction unit 3 extracts item name data that is text data of an item name from the chapter, section, table of contents data, etc. indicated by the structured text information in each article data input from the article data acquisition unit 2 (step S15). ). Specifically, the following processing is performed.

  The structured text information in the article data includes style data, and text data corresponding to the style data is included in items such as chapters, sections, and table of contents based on the size of the characters indicated by the style data. It can be easily determined that the item name is item name data. When the article data is HTML document data, style data such as the character size is described in the tag. The extraction unit 3 identifies a tag indicated as item name data from the structured text information included in each article data A1, A2,... And extracts text data corresponding to the identified tag as item name data. To do. The extraction unit 3 extracts only item name data of items in the first layer when items are layered, such as section 1.1 in the lower layer of Chapter 1, but items of all layers are extracted. Item name data may be extracted.

  By the above process, the same item name data may be extracted from a plurality of article data. Therefore, when the extraction unit 3 extracts the item name data from all the article data, the same item name data that is completely matched is integrated to leave only one and eliminates duplication. For each item name data Bi (i is an integer equal to or greater than 1) obtained as a result of eliminating duplication, the extraction unit 3 calculates the number of appearing articles Ni, which is the number of article data including the item name data in the structured text information. obtain. Specifically, the number of appearing articles in the item name data without duplication is 1, and the number of appearing articles in the item name data with duplication is the number of the item name data before integration. The extraction unit 3 outputs a combination (B1, N1), (B2, N2),... Of the item name data and the number of appearance articles of the item name data to the generalization processing unit 4.

  The generalization processing unit 4 performs generalization processing on the item name data input from the extraction unit 3 (step S20). Generalization processing includes integration processing and deletion processing. In the integration process, the generalization processing unit 4 verifies whether or not a plurality of item name data is to be integrated, and if it is to be integrated, integrates it into one item name data. Further, in the integration process, the generalization processing unit 4 calculates the number of appearing articles obtained by summing up the number of appearing articles of each item name data to be integrated for each item name data remaining as a result of the integration. Further, in the deletion process, the generalization processing unit 4 verifies whether each item name data is a deletion target, and deletes the item name data if it is a target. The generalization processing unit 4 obtains a news element candidate data group that is a list of item name data remaining after the generalization processing (step S20). News element candidate data is data that is a candidate for news element data. Details of the integration process and the deletion process will be described later. The generalization processing unit 4 outputs the news element candidate data Dj (j is an integer of 1 or more) and the number of appearance articles Lj (D1, L1), (D2, L2),... To the ontology generation unit 5. .

  The ontology generation unit 5 sets the news element candidate data satisfying a predetermined condition from the news element candidate data D1, D2,... Input from the generalization processing unit 4 as news element data E1, E2,. select. For example, the predetermined condition may be news element candidate data with the number of appearing articles equal to or greater than a predetermined number, news element candidate data with a predetermined number or a predetermined ratio in descending order of the number of appearing articles, and the like. The ontology generation unit 5 sets the news event class data indicating the news event class C1 “earthquake” as the first hierarchy, and sets the selected news element data E1 “cause of occurrence”, news element data E2 “correspondence”,. The ontology structure thus created is created (step S25). The ontology generation unit 5 generates ontology data in which the created ontology is described using an ontology description language or the like, and writes the ontology data into the ontology storage unit 8 (step S30). The generated ontology has two layers as shown in FIG.

  Next, details of the generalization process executed by the generalization processing unit 4 in step S20 of FIG. 3 will be described with reference to FIGS.

FIG. 4 shows an integration process focusing on the surface features of the item names. In the integration process focusing on the surface features, when the characters appearing in a plurality of item name data have a common appearance pattern, the item name data are integrated. In addition, the numerical value described in parentheses after each item name data indicates the number of appearing articles. Hereinafter, item name data indicating the item name “X” is referred to as item name data “X”.
FIG. 4A shows an integration process based on backward matching. When it is determined that the item name indicated by one item name data matches the end of the item name indicated by the other item name data, the generalization processing unit 4 integrates these item name data, The shorter length is used as the item name data after integration. As shown in the figure, the item name data “correspondence” and the item name data “government correspondence” are in a backward match, and these two item name data are the item name data “correspondence” of the item name having a short length. Integrated into. The generalization processing unit 4 sets the number of appearance articles of the item name data “correspondence” after the integration, the number of appearance articles “6” of the item name data “correspondence” before the integration, and the appearance of the item name data “correspondence of the government” The total number of articles “2” is “8”.

  FIG. 4B shows an integration process based on front matching. When the generalization processing unit 4 determines that the item name indicated by one item name data matches the head of the item name indicated by the other item name data, the generalization processing unit 4 integrates these item name data, The shorter length is used as the item name data after integration. As shown in the figure, the item name data “damage” and the item name data “damage status” are in front of each other, and these two item name data are the item name data “damage” of the item name having a short length. Integrated into. The generalization processing unit 4 sets the number of appearing articles of the item name data “damage” after the integration, the number of appearing articles “12” of the item name data “damage” before the unification, and the appearing articles of the item name data “damage status”. The sum of the number “4” is “16”.

  FIG. 4C shows an integration process by partial matching. When the generalization processing unit 4 determines that the item name of one item name data is included other than the head or the end of the item name of the other item name data, these item name data are integrated, The item name data of the item name included in the item name is used as the integrated item name data. As shown in the figure, the item name data “Damage status in each location” and the item name data “Damage” are partially matched, and these two item name data are included in the item name “Damage status in each location”. It is integrated into the item name data of the item name “damage”. The generalization processing unit 4 sets the number of appearing articles of the item name data “damage” after integration, the number of appearing articles “3” of the item name data “damage status of each place” before the integration, and the item name data “damage”. The total number of appearance articles “12” is “15”.

  FIG. 4D shows integration processing by LCS (longest common subsequence). The generalization processing unit 4 extracts the number of characters that appear in the same order in common with the item name indicated by one item name data and the item name indicated by the other item name data, and the number of extracted characters is equal to or greater than a predetermined number If it is determined that the item name data is integrated, these item name data are integrated, and the item name with the shorter item name length is used as the integrated item name data. As shown in the figure, in the case of item name data “Cause of occurrence” and item name data “Cause of occurrence”, “Occurrence”, “Raw”, and “Cause” appear in the same order in common. The generalization processing unit 4 determines that the extracted number of characters “3” is equal to or greater than the threshold “3” for determining that they should be integrated, and uses these two item name data as item name data having a shorter length. Integrate into “Cause”. The generalization processing unit 4 sets the number of appearance articles of the item name data “occurrence cause” after the integration, the number of appearance articles “3” of the item name data “occurrence cause” before the integration, and the item name data “cause of occurrence”. The total number of appearing articles “6” is “9”.

  As described above, the generalization processing unit 4 determines whether or not the item name data is to be integrated based on the surface layer matching of the item names. When the item name data is to be integrated, the number of characters in the item name data to be integrated Item name data with less item names is used as item name data after integration.

FIG. 5 shows an integration process based on the semantic characteristics of the item names. In the integration process based on semantic features, when the meanings associated with a plurality of item names have a common concept, the item name data of those item names are integrated. The generalization processing unit 4 performs integration processing using a concept dictionary such as a thesaurus.
The concept dictionary data stored in the thesaurus storage unit 7 shows a plurality of subordinate concepts having a common superordinate concept. Therefore, the generalization processing unit 4 integrates a plurality of item name data indicating item names of lower concepts having a common upper concept into item name data having item names of the upper concepts common to these item names.
As shown in the figure, the concept dictionary data indicates that “bus” and “train” are subordinate concepts having a common superordinate concept “transportation”. The generalization processing unit 4 integrates the item name data “bus” and “train” into the new item name data “transportation”. The generalization processing unit 4 sets the number of appearance articles of the item name data “transportation” after the integration, the number of appearance articles “4” of the item name data “train” before the integration, and the appearance article of the item name data “bus”. The number “1” is added to “5”.

FIG. 6 shows a deletion process for deleting item name data inappropriate as a news element.
FIG. 6A shows a deletion process by character number determination. The generalization processing unit 4 deletes item name data indicating an item name having a predetermined number of characters. This is because an item name that is too long is considered to be an item name specific to the content handled by the article data that contains the item name, and is unlikely to be an item common to the entire news event class. It is.

  FIG. 6B shows a deletion process when a number is included. The generalization processing unit 4 deletes item name data indicating an item name including a number. Each article data often describes a specific event. For example, even the article data associated with the news event class “earthquake” includes an article explaining the Great Kanto Earthquake and an article explaining the recovery status from the recent earthquake damage. Numbers often represent cases such as years and numbers related only to specific events, and item names containing numbers are unlikely to be common to all news event classes, so they are subject to deletion.

  FIG. 6C shows a process for deleting item name data indicating the same item name as the news event class. The generalization processing unit 4 deletes item name data indicating the same item name as the news event class.

The generalization processing unit 4 performs generalization processing such as integration processing and deletion processing described above, and finally the remaining item name data becomes news element candidate data.
Note that all of the integration processes and deletion processes described above may be executed, or only a part thereof may be executed. Further, the execution order of each integration process and each deletion process is arbitrary. Since the results obtained as news element candidate data differ depending on the integration processing and deletion processing to be executed and the order of execution, the system designer can determine a combination of them flexibly according to the purpose.

  As a characteristic of Japanese, important content is often placed behind. Moreover, general words are often used as concepts in concept dictionaries. Therefore, for example, after performing integration processing by backward matching first, and then performing forward matching, partial matching, integration processing by LCS, and deletion processing, after becoming short general item name data of the number of characters, semantic features It is conceivable to perform integration processing based on the above.

A detailed processing example of each integration processing will be described.
First, a detailed processing example of integration processing by backward matching will be described. The generalization processing unit 4 pays attention to one item name data in the processing target item name data group, the item name indicated by the focused item name data, and other items included in the processing target item name data group Each of the item names indicated by the item name data is compared to determine whether or not there is a backward match. When there is item name data that matches backward, the generalization processing unit 4 has the largest number of characters in the item name data that combines the item name data of interest and the item name data determined to match backward. The item name data with fewer item names is used as the integrated item name data. The generalization processing unit 4 calculates the number of appearing articles of the item name data after the integration by adding the number of appearing articles of the item name data of interest and the number of appearing articles of each item name data determined to match backward. To do. The generalization processing unit 4 excludes item name data that did not remain as a result of integration from the item name data group to be processed.
Subsequently, the generalization processing unit 4 pays attention to one of the item name data not yet focused on other than the item name data remaining as a result of the integration from the processing target item name data group, and performs the same processing. repeat. The generalization processing unit 4 ends the integration process by backward matching when there is no item name data that has not yet been focused on in the item name data group to be processed. The item name data after integration and the item name data that did not match backward with other items are the item name data remaining as a result of the integration processing by backward matching.

The processing procedure of the integration process based on front matching is the same as that of the integration processing based on backward matching.
The integration process based on partial matching differs from the integration process based on backward matching in the following points. The generalization processing unit 4 determines whether or not the item name indicated by the item name data of interest is included as a part of the item name indicated by the other item name data. The generalization processing unit 4 uses the item name data of interest as item name data after integration, and only associates information indicating that the item name data has been integrated with other item name data, and does not exclude it from the item name data group to be processed. . The generalization processing unit 4 repeats the same processing by paying attention to one item name data to which information indicating integration is not added from the item name data group to be processed. At the end of processing, the item name data to which information indicating integration is not added is item name data after integration, or item name data that did not partially match other items, and results of integration processing by partial matching It is the remaining item name data.

  In the case of integration processing based on semantic features, the following points are different from integration processing based on backward matching. The generalization processing unit 4 determines whether or not the superordinate concept is the same for the item name indicated by the item name data of interest and the item name indicated by the other item name data. If there is item name data of a name, those item name data are excluded from the item name data group to be processed, and item name data indicating the item name of the superordinate concept is generated. The item name data of the item name indicating the superordinate concept and the item name data whose superordinate concept does not match the other item name data are the item name data remaining as a result of the integration processing based on the semantic features.

  Although the first embodiment has been described above, the article data of the unstructured text information may be included in the article data read by the article data acquisition unit 2 in step S10 of FIG. The text data in the unstructured text information describes only the explanation and does not describe items. In step S15, the extraction unit 3 extracts a keyword from the text data in the unstructured text information by an arbitrary existing keyword extraction process, and similarly performs the process using the data indicating the extracted keyword as item name data. .

  Further, in step S20 of FIG. 3, the generalization processing unit 4 may perform the integration process only for the combination of item name data extracted from different article data. In this case, the extraction unit 3 adds the extraction source information that identifies the article data of the extraction source to each item name data and outputs it to the generalization processing unit 4. The generalization processing unit 4 does not perform the integration process for the combination of item name data to which extraction source information indicating that reading is performed from the same article data.

[2. Second Embodiment]
In the first embodiment described above, an ontology having two layers is generated, but in the present embodiment, an ontology having three or more layers is generated. Hereinafter, differences from the first embodiment will be described.

  In each integration process shown in FIG. 4 and FIG. 5, the generalization processing unit 4 includes deleted item name information indicating item name data that did not remain as a result of integration, and upper layer information indicating which item name data was integrated. To generate integrated information associated with. For example, in the case of backward matching shown in FIG. 4A, integrated information in which the deleted item name information in which the item name data “government correspondence” is set and the upper hierarchy information in which the item name data “correspondence” is set are associated Is generated. Further, in the case of the integration process based on the semantic features shown in FIG. 5, the item name data “train” and the item name data “bus” and the item name data “transportation” are set. Integrated information in which upper layer information is associated is generated.

  In step S <b> 25 of FIG. 3, the generalization processing unit 4 outputs integrated information to the ontology generation unit 5 together with the set of news element candidate data and the number of appearing articles. In step S30, the ontology generation unit 5 first creates a tree-structured news ontology having the news event class as the first hierarchy and the news element data selected based on the number of appearing articles as the second hierarchy. Subsequently, the ontology generation unit 5 identifies, for each news element in the second hierarchy, integrated information in which the upper hierarchy information matches the news element data in the second hierarchy, and the item name from the deleted item name information of the identified integrated information Read data. The ontology generation unit 5 adds item name data read from the deleted item name of the integrated information corresponding to each news element data in the second hierarchy as news element data subordinate to the news element data, and hierarchy up to the third hierarchy Ontology data is created in which the structured ontology news ontology is described using an ontology description language.

  The ontology generation unit 5 may perform the same processing for the third and lower layers to create a tree-structured news ontology having four or more layers. The ontology generation unit 5 generates ontology data in which the created ontology is described using an ontology description language or the like, and writes the ontology data into the ontology storage unit 8.

  FIG. 7 is a diagram illustrating news ontology data generated according to the present embodiment. This figure shows a tree-structured news ontology having a category C1 that is a news event class as the highest hierarchy.

[3. Third Embodiment]
In the second embodiment described above, an ontology having a tree structure is generated using information when items are integrated. In this embodiment, when the table of contents included in the article data is hierarchized, A tree-structured ontology is generated using the hierarchical structure. Hereinafter, differences from the first embodiment will be described.

  In step S15 of FIG. 3, the extraction unit 3 reads item name data and the hierarchical relationship between the item name data from the structured text information included in each article data. For example, in the case of the structured text information shown in FIG. 2A, the item name data in the lower hierarchy of the item name data “Summary” are the item name “main shock”, “main shock (city where the seismic intensity is 4 or higher), ... The item name data in the lower hierarchy of the item name data “damage” is the item name data of the item name “transportation”,... And the item name data in the lower hierarchy of the item name data “transportation” Is read out as item name data of item names “highway”, “general road”,.

In step S30 in FIG. 3, the ontology generation unit 5 first creates a tree-structured news ontology having the news event class data as the first hierarchy and the news element data selected based on the number of appearing articles as the second hierarchy. . The ontology generation unit 5 outputs the second level news element data to the extraction unit 3, and the extraction unit 3 returns the item name data in the lower hierarchy of the item name data that matches the second level news element data. The ontology generation unit 5 adds item name data in a lower hierarchy of the news element data in the second hierarchy to each news element in the second hierarchy, and creates a tree-structured news ontology structure hierarchized up to the third hierarchy. Generate. The ontology generation unit 5 creates ontology data in which the ontology of the generated structure is described using an ontology description language or the like.
The ontology generation unit 5 may perform the same processing for the third and lower layers, generate a tree-structured news ontology structure of four or more layers, and generate ontology data indicating the ontology of the generated structure.

  When the news element data is item name data obtained by integration processing based on semantic features, the ontology generation unit 5 extracts lower-level item name data corresponding to the item name data before integration. It may be further acquired from the section 3 and added to the lower hierarchy of the news element data. For example, as in the second embodiment, the generalization processing unit 4 generates integrated information and outputs it to the ontology generation unit 5. The ontology generation unit 5 identifies, for each of the news element data in the second layer, integrated information whose upper layer information matches the news element data in the second layer, and extracts item name data from the deleted item name information of the specified integrated information Read out and output to the extraction unit 3. The extraction unit 3 returns the item name data in the lower hierarchy of the item name data input from the ontology generation unit 5, and the ontology generation unit 5 converts the returned item name data in the lower hierarchy to the lower hierarchy of the news element data. Append.

[4. Fourth Embodiment]
In the first embodiment described above, the context between news element data is not considered. However, when describing an event, the news elements are usually arranged in an easy-to-understand order. For example, for the news event class “earthquake”, even if the news elements “damage”, “after”, etc. are extracted, it is not possible to explain the contents about the news element “after” before the contents about the news element “damage” Absent. Therefore, in this embodiment, the context between news element data is also extracted. Accordingly, when generating a summary article in which a plurality of articles and information are grouped by news element data, it can be arranged in an appropriate order. Hereinafter, differences from the first embodiment will be described.

  In step S15 of FIG. 3, the extraction unit 3 reads out the context between the item name data from the structured text information included in each article data. For example, in the case of the structured text information shown in FIG. 2A, it is read that the item name data is in the order of “summary”, “occurrence factor”, “damage”, “government / local government response”,. The extraction unit 3 outputs order information indicating the context between the item name data to the ontology generation unit 5.

  In step S30 in FIG. 3, the ontology generation unit 5 selects news element data based on the number of appearing articles. The ontology generation unit 5 reads out the context obtained from the article data including two or more of these news element data as item name data from the order information. Thus, when the ontology generation unit 5 obtains statistical information about the context between news element data, the ontology generation unit 5 determines the arrangement order of news element data that appears most frequently based on the statistical information. The ontology generation unit 5 creates an ontology structure with the news event class data as the first hierarchy and the news element data arranged according to the determined arrangement order as the second hierarchy. The ontology generation unit 5 generates ontology data in which the created ontology is described using an ontology description language or the like, and writes the ontology data into the ontology storage unit 8.

  When the news element data is item name data obtained by integration processing based on semantic features, the ontology generation unit 5 uses the item name data before being integrated into the news element data, and uses the order name information. Statistical information about the context of news element data may be obtained.

  FIG. 8 is a diagram illustrating a news ontology generated according to the present embodiment. The figure shows a news ontology in which the news elements are arranged in the order of “Outbreak Occurrence”, “Damage”, “Response”, “Cause”, and “Subsequently” below the news event class “Earthquake”. .

[5. Fifth Embodiment]
In the embodiment described above, a news element data group constituting a news event class is extracted using article data including semi-structured text information. In the present embodiment, a news ontology is constructed from news articles by the same processing.
A general news article used in daily life or broadcasting is composed of a news title and a text representing the content. Therefore, an ontology can be constructed by performing the same processing as in the first embodiment using the news title instead of the item name as the headline. Hereinafter, differences from the first embodiment will be described.

  FIG. 9 is a diagram showing article data used in the present embodiment. As shown in the figure, article data A'1, A'2,... Are text data indicating the title of the news and the content of the news.

  2, the article data acquisition unit 2 of the conceptual processing device 1 acquires article data A′1, A′2,... Belonging to the specific news event class shown in FIG. In step S15, the extraction unit 3 reads the title of the article from the article data A′1, A′2,... Acquired by the article data acquisition unit 2. The conceptual processing device 1 uses the title read by the extraction unit 3 as heading data, and performs the same processing using the read heading data instead of the item data in the first embodiment. Thus, a news ontology can be constructed by obtaining a news element data group from headline data subjected to generalization processing.

If the article data includes news date information, the extraction unit 3 reads the title and date information in association with each other in step S15. In step S30 in FIG. 3, the ontology generation unit 5 acquires a date corresponding to the selected news element data from the extraction unit 3. An ontology structure is created with news event class data as the first hierarchy and news element data arranged in date order as the second hierarchy. When a plurality of dates are read for the same news element data, the date may be the earliest date or the date that appears most frequently. Thus, for example, for the news event class “corruption case”, an ontology that describes the knowledge that news element data such as “arrest”, “denial”, “transmission”, “judgment”, “appeal” appears in this order. Can be built.
By using this ontology, it is also possible to provide an advanced news search / presentation service.

  Note that the first sentence of the news content portion is called a lead sentence, and includes important contents indicating an outline of the news. Therefore, in step S15, the extraction unit 3 reads out the first sentence of the news body from the article data acquired by the article data acquisition unit 2 as heading data, and uses the read heading data as the item name data in the first embodiment. It can be used instead.

[6. effect]
According to the embodiment of the present invention described above, it is possible to obtain a news element group necessary for conveying information related to a certain news event. In order to obtain such a news element group, in the present embodiment, knowledge of what configuration is used when a person conveys information about a news event is used. As this knowledge, structural information edited manually, for example, article data provided by Wikipedia (registered trademark) can be used. Since the article data applied by Wikipedia (registered trademark) is provided with category information, the article data belonging to the news event class can be easily read using this information. Then, after extracting items constituting a particular type of news event class such as an earthquake disaster, a corruption incident, an aircraft accident, etc. from the structure information, a news element group is automatically extracted by performing a generalization process. As a result, it is possible to automatically acquire the structure of various types of news event classes, and it is possible to improve the construction of the news ontology.
In addition, if the structure indicated by the extracted ontology is used to reconstruct the news items according to the user's request, a summary article about an event in which multiple article contents are integrated can be created. It can be used for processing to search only articles corresponding to specific elements (for example, “cause” of the event).
In addition, a news manuscript or the like can be used as article data, and a news title or a lead sentence can be used as a headline.

[7. Others]
The above-described conceptual processing apparatus 1 has a computer system inside. The operation process of each unit of the conceptual processing device 1 is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by the computer system reading and executing this program. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage unit such as a hard disk built in the computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

DESCRIPTION OF SYMBOLS 1 ... Conceptual processing apparatus 2 ... Article data acquisition part 3 ... Extraction part 4 ... Generalization processing part 5 ... Ontology generation part 6 ... Article data storage part 7 ... Thesaurus storage part 8 ... Ontology storage part

Claims (5)

An article data storage unit for storing article data including heading data;
An extraction unit that reads out the article data from the article data storage unit and extracts the heading data;
When characters appearing in the plurality of heading data extracted by the extraction unit have a common appearance pattern, integrated heading data is generated based on the common appearance pattern, and the plurality of heading data extracted by the extraction unit is A generalization processing unit that outputs, as component data, heading data as a result of performing generalization processing for generating heading data after integration based on the common superordinate concept when having a common superordinate concept;
A conceptual processing apparatus comprising: Ontology in which the heading data integrated with the component data is added to the lower layer of the component data obtained by the generalization processing unit, or the lower layer of the component data obtained by the generalization processing unit And further comprising an ontology generation unit that generates ontology data to which heading data below the component data included in the hierarchical heading data indicated by the article data is added.
The conceptual processing device according to claim 1, wherein: Ontology data obtained by determining the order of the component data obtained by the generalization processing unit based on the appearance order of the heading data indicated by the article data, and adding information indicating the determined appearance order to the component data Further comprising an ontology generation unit for generating,
The conceptual processing device according to claim 1, wherein: In the generalization process, the generalization processing unit is configured such that, in the generalization process, the headings indicated by the plurality of heading data match backward, forward, or partially match, or in the same order as the headings indicated by the plurality of heading data. When the number of characters that appear in common is greater than or equal to a predetermined number, the plurality of heading data is integrated into the heading data of the shortest heading data among the plurality of heading data, the heading data having a common superordinate concept, Integrating headline data indicating a superordinate concept, and deleting headline data that matches a predetermined condition for determining that the headline data is inappropriate as a component,
The conceptual processing device according to claim 1, wherein: A computer used as a conceptual processing device,
An article data storage unit for storing article data including heading data;
An extraction unit that reads out the article data from the article data storage unit and extracts the heading data;
When characters appearing in the plurality of heading data extracted by the extraction unit have a common appearance pattern, integrated heading data is generated based on the common appearance pattern, and the plurality of heading data extracted by the extraction unit is A generalization processing unit that outputs, as component data, heading data as a result of performing generalization processing for generating heading data after integration based on the common superordinate concept when having a common superordinate concept;
A program characterized by functioning as

Images