CN1822000A - A Method for Automatically Detecting News Events - Google Patents

Abstract

This invention puts forward a practical news item test method, which introduces events, sequences, merges and adjusts them, eliminates news reports and describes news events to obviously increase the test result of news events, strengthen its practicability and can be widely used in intelligent information process.

Description

A Method for Automatically Detecting News Events

technical field

The invention belongs to intelligent information processing technology, and in particular relates to a method for automatically detecting news events.

Background technique

With the rapid development of the Internet, news information has shown explosive growth. How to timely obtain new hot news event information from the emerging news reports, and keep track of the news events you are interested in has become a research hotspot in recent years. Topic detection and tracking technology is an attempt to solve this problem.

Topic Detection and Tracking (TDT) research began in 1996, when the research initiator and participant James Allan et al. defined the specific tasks and performance evaluation indicators of TDT in the "Topic Detection and Tracking (TDT) Pilot Study Final Report". And some experimental results at that time are given. The three main tasks of TDT are:

(1) News report segmentation task: segment the voice or text records of continuous radio and TV news programs into different reports;

(2) Event detection task: identify events unknown to the system, and identify related reports;

(3) Event tracking task: monitor news report information flow to discover new reports related to a known event.

In addition, the paper mentioned that the current research focus in TDT is the detection and tracking of events, in which a topic is a concept broader than an event, and a topic can include multiple related events.

In essence, event detection is to cluster news reports according to different events, and it is necessary to classify reports discussing an event into one category (James Allan, 2002). Compared with the usual text clustering, the particularity of event detection is mainly manifested in two aspects: First, the processing object of event detection is the flow of news reports that appear in chronological order, which changes dynamically over time, rather than a static closed Text collection; secondly, event detection clusters based on the events discussed in the report rather than the topic category, and the granularity of the information based on it is relatively small, so there should be more classes obtained from event detection. Nevertheless, text clustering techniques are still the basis of event detection techniques.

Event detection can be subdivided according to specific detection scenarios: backtracking detection and online detection. The purpose of backtracking detection is to discover previously unidentified news topics from the existing news report collection, and the system is required to output news topic information, which can explain the relationship between news reports and topics. The focus of online detection is to timely identify new topics from the real-time news report flow, that is, to identify the news topic at the moment when a report expressing a new topic appears.

In the past few years, event detection researchers have tried many different text clustering methods, such as single-pass clustering, k-means clustering, hierarchical agglomerative clustering, probabilistic models, etc. Here are a few main existing event detection methods:

(1) Method of CMU

CMU researchers (Yiming Yang et al.) mainly use single-pass clustering algorithms with time windows in event detection. CMU researchers represent each report and each event as a vector in space. The calculation of the similarity between the report vector and the event vector mainly uses the cosine value of the angle between the vectors, but it needs to be adjusted according to the time factor using an event window. , two strategies can be adopted. The first strategy only considers events that occur within the time window, and the second strategy considers that as the number of reports between the current report s and event c increases, the similarity value between the two should be reduced.

In addition, on SIGKDD in 2002, Yiming Yang et al. proposed a topic-based event detection method in the article "Topic-conditioned novelty detection": firstly, a supervised learning algorithm is used to divide the online document stream into a predefined comparison. broad topic categories, and then combine the characteristics of each topic to perform new event detection on the document stream.

(2) The method of the University of Massachusetts

Researchers at the University of Massachusetts (James Alan, etc.) use a vector model to represent news reports, and the core algorithm still uses a single-pass clustering algorithm. A time-based threshold model is used to calculate the similarity between reports and events, and a linear function is used to adjust the clustering threshold, making it more difficult for news reports that are farther away from an event in time to join the event. When determining the closest event to the current report, in addition to the original centroid comparison strategy, a nearest neighbor comparison strategy is added.

In the centroid comparison strategy, two thresholds θmatch and θcertain are set. If the centroid similarity between the current report and an event is higher than θmatch, the report is classified into this event. But only when the similarity value between them is higher than θcertain, the current report is used to adjust the centroid of the event, that is, the vector representation of the event. When the nearest neighbor comparison strategy is used for identification, it first finds the k reports that are most similar to the current report among the existing reports, and determines the event that the current report should belong to based on the k reports and the preset threshold. If it cannot be assigned to any of the known events, create a new event for it as the first report of a new event.

In addition, James Alan et al. mentioned that the most frequently occurring words in the event were used as the description of the event.

(3) IBM's method

A relatively successful incident detection system developed by IBM uses a two-level clustering strategy, using the symmetric Okapi formula to compare the similarity of two reports. In the first processing of the system, reports are temporarily classified into different micro-events (microcluster), and in the second processing, these micro-events are used as processing objects to form a larger class, which is classified into the final event (Dharanipragada etc., 2002). Each of the above processes adopts a single-pass clustering algorithm, and the only difference lies in the different processing objects and the selection of different thresholds.

To sum up, in the prior art, during the event detection process, common steps can be summarized as follows:

1) Read a report from the data source, including content, time and other relevant information; there may be multiple data sources, and there may be no clear boundaries between reports, and preprocessing such as segmentation between reports is required;

2) Using the centroid comparison or nearest neighbor comparison strategy, calculate the similarity between reports and events, or between reports and reports, and determine the most similar event to the current report;

3) If the report is classified as an event, adjust the event; if the report cannot be classified as an existing event, list it as a newly detected event;

4) Output the detected event, and use several feature words with the highest weight in the event, or a representative report title as the event description.

Since the existing event detection technology only considers the false detection rate and missed detection rate on a fixed small data set, there are several defects as follows:

(1) Event sequencing problem

People's attention has become a scarce resource, and people often don't have time to check a large number of news events, so the most popular news events should be sorted higher, so that such a system can better meet people's needs. The prior art does not consider this problem, and simply outputs detected events.

(2) Event similarity problem

Since the news that reports on different aspects of the same news event may have less similarity, the same news event is divided into multiple small events at the beginning of the event, and as the situation continues to develop, the similarity of these events may increase. It will become larger and larger, which may cause confusion and inconvenience to users' browsing. The prior art does not consider this problem either.

(3) The elimination of news reports

In the actual application environment, event detection is a long-term continuous process. With the dynamic evolution of the event, the relevance of some news in the event to the event is gradually decreasing. In addition, events with a long period of time may also expand with the accumulation of time, and the content of the entire event is too broad. The existing technology overcomes the problem of event dynamic evolution by introducing time window strategy and dynamically adjusting events, but does not consider the problem of news report elimination.

(4) Event description problem

There are two ways to describe a news event: some of the most important feature words in the event, or select a news title in the event. Due to the immature natural language processing technology, the extracted feature words are difficult to effectively describe events, and even the most important feature words such as names of people, places, institutions, and time in news events may not be extracted, such as the Eleventh Five-Year Plan, Shenzhou VI wait. However, if a certain report title in the event is used as the description, for some comprehensive events, the report may only be one aspect of the event, and the description of the event is not comprehensive enough.

Contents of the invention

Aiming at the defects existing in the prior art, the purpose of the present invention is to utilize the characteristics of the news event itself, by solving problems such as event sorting, event merging and adjustment, news report elimination, and news event description, etc., to realize the dynamic, Efficient event detection.

In order to achieve the above object, the technical solution adopted in the present invention is: a method for automatically detecting news events, comprising the following steps:

1) Read a report from the data source and preprocess the report;

2) Calculate the similarity between the report and the detected event, or between the report and the report, determine the events related to the current report, and classify them into related events;

3) If the report is classified into an existing event, adjust the event; if the report cannot be classified into an existing event, list it as a newly detected event;

4) Perform a pairwise comparison of the detected events, merge related events, and re-adjust the events, as well as the similarity between reports and events;

5) Eliminate the reports that do not meet the restriction conditions in each event, and adjust the event;

6) Compare the current number of events with the size of the time window, if the number of events is greater than the size of the event window, sort and eliminate events; otherwise, go to step 7;

7) Output the detection result.

Further, in order to make the present invention obtain a better inventive effect, in step 1), if the similarity between the new report and the previously processed news report is greater than the preset threshold θd, that is, the repetition threshold, it is considered to be a repeated news report, which requires Deduplication processing is performed on news reports, and the value range of θd is 0<θd≤1. The deduplication processing is carried out according to the content of news reports by using the similarity calculation method in text retrieval and text mining.

In step 1), an automatic classification method is first used to classify news reports according to preset categories.

When adopting the method of automatic classification to classify news reports in step 1), the method of combining rule classification based on source and automatic classification based on content is adopted, and automatic classification based on content is the text classification technology adopted. A method for automatically detecting news events according to claim 4, characterized in that: said text classification technology is a support vector machine algorithm based on a vector space model.

Further, in order to make the present invention obtain a better inventive effect, centroid comparison or nearest neighbor comparison strategy is adopted when determining events related to the current report in step 2), and the similarity calculation method can adopt existing text mining technology, The document model is based on the vector space model, probability model, or language model; the similarity formula adopts the cosine angle or the Hellinger distance formula, etc.; the similarity calculation considers the time characteristics of the report and the event.

In step 2), when calculating the similarity, a higher weight is given to the title in the report, or a higher weight is given to the report with higher authority, and the authority of the report adopts the authority of the news source.

Further, in order to make the present invention obtain better invention effect, the measurement of the similarity between events described in step 4) is to adopt the clustering similarity value calculated in the traditional clustering algorithm; if the similarity of two events is greater than The merge threshold θu is regarded as two events are related and merged, and the value range of θu is 0<θu≤1. At the same time, other strategies can also be used for event merging. For example, if several feature words in the internal representations of two events are the same, it is considered that the similarity is high, and the two events are merged.

Further, in order to make the present invention achieve better inventive effects, the restriction conditions described in step 5) can be a similarity threshold or a time limit, or external restrictions such as report attention, user click times, etc.

Further, in step 4) or/and 5), after each processing of new reports determined by the user, or after a period of time determined by the user, or after the detected event is determined by the number of new users , and then carry out the operation of step 4) or/and 5).

Further, in order to obtain a better inventive effect of the present invention, when calculating the sorting of events in step 6), it is necessary to combine the time characteristics and quantity characteristics of news events, for example, within a certain recent time range (for example, 12 hours) within the event The number of newly added reports is used as the event score value; in addition, in the sorting, multiple different sortings can be considered at the same time, for example, considering the last 12 hours, 1 day, 3 days, 7 days, 30 days, etc., only when the event The event is eliminated only when it is not within the event window in any sorting; in this way, multiple sorting can provide users with information references of different granularities.

When calculating the ranking of events in step 6), multiple ranking results in step 6) can be combined to output a ranking that meets the user's requirements, or multiple rankings can be output at the same time. For example, the user can request to view the most Hot events.

Further, in order to make the present invention obtain a better inventive effect, when outputting the detection results in step 7), for all current events, calculate the description; at the same time, combine the time characteristics and quantity characteristics, calculate the event score and sort the events, and select the score Higher news events are regarded as important news events, and the event description and the list of news reports included are output. The generation process of the event description is as follows:

a) Select the number of feature words determined by the user with the highest internal weight in the event;

b) According to the news report selection strategy, select the title of the most representative news report in the event;

c) Combine a) and b), and output a description of the event.

The representative news report selection strategy in step 7) is a threshold strategy that combines the authority of the news source, report click-through rate, report time and other relevant information, and the threshold strategy is a preset event threshold θe, The value range of θe is 0<θe≤1; for example, among the news reports within the event whose similarity with the event is greater than the threshold, the title of the latest news report is selected. Or output the most relevant news stories according to the ratio determined by the user.

The effect of the present invention is that: on the basis of fully considering the characteristics of news events and people's cognitive rules, the present invention aims at sorting events in practical applications, merging and adjusting events, eliminating news reports, and describing news events. There are practical solutions. Experiments show that the method of the present invention can significantly improve the detection effect of news events, thereby greatly enhancing its practicability.

Why the present invention has above-mentioned invention effect, because the present invention has following characteristics:

(1) In terms of event ranking, a mechanism for calculating importance scores for events at a certain moment is introduced. This mechanism comprehensively considers the time characteristics and quantity characteristics of news events, and then gives each event a comparative score at a certain moment. Reasonable score value, used for event sorting.

(2) In terms of event similarity, a mechanism of event merging and adjustment is introduced to overcome the phenomenon that the same news event is mistakenly divided into multiple small events. Every time a fixed number of news reports are processed, the events are compared in pairs. If the similarity of the two events is judged to be high according to the comparison strategy, the events are merged and adjusted.

(3) In terms of news reports, a mechanism for eliminating news reports within events is introduced to overcome the phenomenon that the content of news events is too broad. Every time a fixed number of news reports are processed, the news reports in each event are eliminated.

(4) In terms of event description, a method of combining feature words and news report titles is proposed to overcome the defects of both. First, select several feature words with the highest weight within the event as part of the event description; at the same time, according to the report selection strategy, select the most representative news report in the event, and use the title of the report as part of the event description.

Description of drawings

Fig. 1 is a flow chart of the present invention;

Figure 2 is a schematic diagram of the results of detecting news events from July 22, 2005 to August 9, 2005 using existing methods;

Fig. 3 is a schematic diagram of the results of detecting news events between July 22, 2005 and August 9, 2005 using the method of the present invention;

Figure 4 is a screenshot of the important news on Sina.com on August 9, 2005;

Figure 5 is a schematic diagram of the results of detecting news events from July 22, 2005 to October 9, 2005 using existing methods;

Fig. 6 is a schematic diagram of the results of detecting news events during the period from July 22, 2005 to October 9, 2005 using the method of the present invention;

Figure 7 is a screenshot of the news from Sina.com on October 9, 2005.

Detailed ways

Below in conjunction with accompanying drawing and embodiment the present invention will be further described:

As shown in Figure 1, a method for automatically detecting news events includes the following steps:

1) Read a report from the data source, continuously detect multiple news network data sources (such as Sina.com, Xinhuanet, People.cn, etc.), automatically grab news reports from the network, and analyze the content of the news reports. Time, title and text information, etc. If the time is not found from the report, the crawling time shall prevail;

Due to the considerable duplication among multiple data sources, newly captured news reports are deduplicated according to the text content of the reports; if the duplication degree between the new report and the previously processed news reports is greater than the duplication threshold θd, it is considered as For repeated news reports, the repetition threshold θd set in this embodiment is 0.9;

Because the scope of news reports is too broad, the method of combining source-based rule classification and content-based automatic classification is used to classify news reports (categories are pre-set, for example, refer to the channels of Sina.com, which can be divided into news, technology, finance, sports, etc.). The classification rules are based on news sources and authors. For example, content from Sina’s “Domestic News” channel is classified into the “Domestic News” category, and content from Xinhuanet’s “Technology” channel is classified into the “Technology” category. Content-based automatic classification adopts vector space model and support vector machine algorithm to automatically classify news reports according to report content and title; and perform step 2)-step 7) processing according to category c;

2) Using the centroid comparison strategy, compare the report with the existing detected news events in the category c, and consider the time characteristics and content characteristics at the same time, calculate the similarity between the report and the event, and record the maximum similarity S _max and the similarity The event Es with the highest degree is used to determine the event closest to the current report; the event itself is expressed by several feature words with the highest comprehensive weight in all news inside the event; the similarity between news reports and events is based on the vector space model, through two The cosine value of the included angle (cosine) of the reporter is used to calculate, and the title of the news report is given a higher weight.

3) According to the maximum similarity S _max calculated in step 2) and the event Es with the largest similarity, take the following measures for the current report:

a) If S _max is less than the innovation threshold θn (0.25 in this embodiment): create a new event in the category to which the report belongs;

b) If S _max is greater than θn but less than the clustering threshold θc (0.30 in this embodiment): no processing, return to step 1);

c) If S _max is greater than θc but less than the contribution threshold θt (0.35 in this embodiment): classified into the current event;

d) If S _max is greater than θt: classify event Es, and adjust Es;

The value ranges of the aforementioned S _max , θn, θc, and θt are all greater than 0 and less than or equal to 1.

4) After a class processes a fixed number of new reports determined by the user (the number determined in this embodiment is 20), compare news events in the class; if the similarity of the two events is greater than the merging threshold θu (e.g. 0.20), then merge them. The formula for calculating the similarity between events can use the method of calculating the similarity of two clusters in traditional clustering algorithms, for example, based on the vector space model, comprehensively considering the pairwise similarity between all news reports in the two events, as follows formula:

$\mathrm{<span\; class="notranslate">\; Sim</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; sim</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span>}$

Among them, E ₁ , E ₂ are two detected news events, d _i , d _j are news reports in E ₁ , E ₂ respectively, sim(d _i , d _j ) is the similarity between two news reports degree, |E ₁ |, |E ₂ | are the number of news reports contained in the two events respectively;

5) After a class processes a fixed number of new reports determined by the user (the number determined in this embodiment is 20), the news reports in each event are eliminated: the similarity between the news report and the event is recalculated, Eliminate news reports whose similarity is lower than the clustering threshold θc, or that do not meet the constraints (such as whether the report is within the past 30 days); and then recalculate the internal representation of the event and its weight;

6) If the number of events in the current category exceeds the size of the event window, sort all news events in the category: combine the time characteristics and quantity characteristics of news events, calculate the score value of news events and sort them; when calculating the score value, simultaneously Consider multiple different sorts, taking into account the last 12 hours, 1 day, 3 days, 7 days, 30 days, etc., and only knock out an event if it is not within the event window in any sort; thus, multiple Sorting provides users with information references of different granularities. The system eliminates news events that are not in the event window to improve the efficiency of system processing;

7) According to user requirements, externally output the detection results: for all current events in the category, calculate their description; at the same time, combine the time characteristics of the event and the number of news reports in the event, select several highest scoring ones from all categories News event, as the hottest news event in this category, outputs an event description and a list of included news reports. Among them, the generation process of the event description is as follows:

a) Read several feature words with the highest internal weight in the event;

b) Among the news reports within the event whose similarity with the event is greater than the event threshold θe (0.6 in this embodiment), select the title of the latest news report; the event threshold can also be taken in a proportional (20%) manner .

c) Combine a) and b), and output a description of the event.

In order to verify the effectiveness of the present invention, we used 100,000 articles captured from some channels (news, science and technology, sports, etc.) The news corpus was tested, and 100,000 news corpora were divided into 3 categories: news, technology, and sports. The evaluation index adopts the detection rate of major news events (because the Sina.com News Channel Highlights column is edited manually, so the Sina.com News Channel Highlights column of the same time period is used as the expert results for comparison). Let's take the "news" category as an example to illustrate the test results, which are shown in Figure 2 to Figure 7.

Fig. 2 to Fig. 7 all compare the top 10 major news events (wherein the parentheses are the number of relevant news detected) of the sorting top 10 major news events detected by the method of the present invention and the traditional method at news detection suspension time, and Sina news channel highlights Column at 21 o'clock of the day for a list of major news events. Among them, the news detection time in Figure 2 to Figure 4 is from July 22, 2005 to August 9, 2005, and the news detection suspension time in Figure 5 to Figure 7 is from July 22, 2005 to October 9, 2005 . Among them, the traditional method is the single-pass clustering algorithm adopted by Yiming Yang et al.: event sorting directly uses the reverse order of event detection (that is, the latest detected event is listed at the top), and event elimination uses the method of event window (every sorting Events beyond the event window are eliminated), and the event description adopts the keyword description method proposed by James Allan et al.

As can be seen from Fig. 2 to Fig. 7, the method that the present invention proposes is better than traditional method, comprises:

1. The sequence of events is more reasonable; as can be seen from Fig. 2 to Fig. 7, in the method proposed by the present invention, the detection rates of the main topic of Sina in the first ten events reached 62.5% and 57% respectively;

2. Reduced the situation that the same event is mistakenly divided into multiple small events; the 3rd-6 events in Fig. 2 all commemorate the 60th anniversary of the victory of the War of Resistance Against Japan, and are divided into multiple events in traditional methods, but in the present invention The method is unified as the fourth event in Figure 3;

3. The description of news events is more accurate and comprehensive; for example, the "Shenzhou 6" incident, through the description of the third event in Figure 5, will be more accurate and comprehensive than pure keywords or pure representative news headlines.

In addition, since the method proposed by the present invention introduces a news report elimination mechanism in the news event, the content of the news event is more concentrated.

Experiments show that because the traditional method only considers the false detection rate and missed detection rate on a fixed small data set, there are many defects in the actual application environment. The method proposed by the present invention fully considers the characteristics of news events and people's cognition rules, so that the detection effect of news events is significantly improved, and its practicability is greatly enhanced.

In practical application, content-based automatic classification can also use other text classification techniques, such as language model-based KNN algorithm; in step 2), when determining the most similar event to the current report, a centroid comparison strategy can also be used. Therefore, the method described in the present invention is not limited to the examples described in the specific implementation, as long as those skilled in the art obtain other implementations according to the technical solution of the present invention, they also belong to the technical innovation scope of the present invention.

Claims (15)

1. A method for automatically detecting news events, comprising the following steps: 1) Read a report from the data source and preprocess the report; 2) Calculate the similarity between the report and the detected event, or between the report and the report, determine the events related to the current report, and classify them into related events; 3) If the report is classified into an existing event, adjust the event; if the report cannot be classified into an existing event, list it as a newly detected event; 4) Perform a pairwise comparison of the detected events, merge related events, and re-adjust the events, as well as the similarity between reports and events; 5) Eliminate the reports that do not meet the restriction conditions in each event, and adjust the event; 6) Compare the current number of events with the size of the time window, if the number of events is greater than the size of the event window, sort and eliminate events; otherwise, go to step 7; 7) Output the detection result. 2. a kind of method for automatically detecting news event as claimed in claim 1, it is characterized in that: in step 1), if new report and the previously processed news report similarity are greater than preset threshold θd i.e. repetition threshold, It is considered to be a repeated news report, and the news report needs to be deduplicated. The value range of θd is 0<θd≤1. The deduplication process is based on the content of the news report using text retrieval and text mining. The similarity calculation method is carried out. 3. A kind of method for automatically detecting news events as claimed in claim 1 or 2, is characterized in that: in step 1), first adopt the method for automatic classification to classify news reports by preset categories. 4. A kind of method for automatically detecting news event as claimed in claim 3, is characterized in that: When adopting the method of automatic classification to classify news reports in step 1), the method of combining rule classification based on source and automatic classification based on content is adopted, and automatic classification based on content is the text classification technology adopted. 5. A method for automatically detecting news events as claimed in claim 4, characterized in that: said text classification technology is a support vector machine algorithm based on a vector space model. 6. A kind of method for automatically detecting news event as claimed in claim 1, is characterized in that: adopt centroid comparison or nearest neighbor comparison strategy when determining the event relevant to current report in step 2), similarity calculation method is to adopt text For the mining technology, the document model is based on the vector space model, probability model, or language model; the similarity formula uses the cosine angle or the Hellinger distance formula; the similarity calculation also considers the time characteristics of reports and events. 7. A method for automatically detecting news events as claimed in claim 6, characterized in that: In step 2), when calculating the similarity, a higher weight is given to the title in the report, or a higher weight is given to the report with higher authority, and the authority of the report adopts the authority of the news source. 8. a kind of method for automatically detecting news events as claimed in claim 1, is characterized in that: step 4) the measurement of similarity between events described in, is to adopt the clustering similarity value calculated in the traditional clustering algorithm ; If the similarity of the two events is greater than the merge threshold θu, the two events are considered to be related and merged, and the value range of θu is 0<θu≤1; or, if the internal representation of the two events If several feature words are the same, it is considered that the similarity is high, and the two events are merged. 9. A method for automatically detecting news events according to claim 1, characterized in that: the restriction condition described in step 5) is a similarity threshold, a time restriction or an external restriction. 10. A kind of method for automatically detecting news event as claimed in claim 8 or 9, it is characterized in that: in step 4) or/and 5), after every newly added report of the determined quantity of processing user, or every running section After the time determined by the user, or after the number of detected events is increased by the number determined by the user, the operation of step 4) or/and 5) is performed. 11. a kind of method for automatically detecting news events as claimed in claim 1, is characterized in that: in step 6), in conjunction with the time characteristics and quantity characteristics of news events, calculate the scoring value of news events and sort; System only A fixed number of news events are saved, and news events ranked lower are eliminated. 12. a kind of method for automatically detecting news event as claimed in claim 11 is characterized in that: when calculating event sorting in step 6), need to combine the time characteristic and quantity characteristic of news event; In sorting, consider simultaneously many sorts by different time periods, and events are only eliminated if they are not within the event window in any sort. 13. A kind of method for automatically detecting news event as claimed in claim 11 or 12, it is characterized in that: when step 6) calculates event sorting, in conjunction with step 6) in a plurality of sorting results, output meets certain user requirement sort, or output multiple sorts at the same time. 14. A kind of method for automatically detecting news event as claimed in claim 13, is characterized in that: when step 7) output detection result, for current all events, calculate event description; Simultaneously, in conjunction with the time characteristic and quantity characteristic of event, Sort the events and select news events with higher scores as important news events, and output event descriptions and a list of news reports included, where the generation process of event descriptions is as follows: a) Select the number of feature words determined by the user with the highest internal weight in the event; b) According to the news report selection strategy, select the title of the most representative news report in the event; c) Combine a) and b), and output a description of the event. 15. A kind of method for automatically detecting news events as claimed in claim 14, is characterized in that: the news report selection strategy described in step b) is the threshold value of the authoritativeness of combining news source, report click-through rate, report time strategy, the threshold strategy is a preset event threshold θe, and the value range of θe is 0<θe≤1.

Images