CN114756673A - Method, device, electronic device and storage medium for generating policy text abstract - Google Patents

Abstract

The embodiment of the present invention discloses a method, device, electronic device and storage medium for generating a policy text abstract. The policy text abstract generating method obtains the candidate title of the candidate policy text, classifies the candidate title, and obtains the policy corresponding to the candidate policy text Category, determine the text keywords from the candidate policy texts, determine the first frequency of the text keywords appearing in the preset keyword database, determine the text importance of the candidate policy texts according to the first frequency, and use the text importance in each policy. Determine the target policy text from the candidate policy text of the category, extract the target key sentence from the target policy text, and generate the policy summary of the policy category corresponding to the target policy text according to the target key sentence. Compared with manual sorting, the policy can be generated automatically and quickly. Abstract, which can improve the efficiency of generating policy text summaries and the accuracy of generating policy summaries.

Description

Method, device, electronic device and storage medium for generating policy text abstract

technical field

The present invention relates to the technical field of natural language processing, and in particular, to a method, device, electronic device and storage medium for generating policy text abstracts.

Background technique

With the advent of the information society, the electronic version of the policy news has replaced the paper version of the policy news. Among them, the Policy Morning Post is a form of policy news, and the Policy Morning Post is a collection of main policy summaries generated based on the daily updated policies in the policy database, in order to facilitate readers to understand the important policy and current affairs of the day in a short time. Therefore, for people who are always concerned about the direction of the policy and lack sufficient reading time, the Policy Morning Post is the first choice for reading.

In the related technology, the policy morning report is generated by manual daily regular sorting. It is necessary to understand the policy text based on the experience of the organizer, and then extract the policy summary and organize it into a policy morning report. However, using manual extraction of policy summaries not only consumes manpower, but also reduces the efficiency of generating policy summaries.

SUMMARY OF THE INVENTION

The following is an overview of the topics detailed in this article. This summary is not intended to limit the scope of protection of the claims.

The embodiments of the present invention provide a method, apparatus, electronic device and storage medium for generating a policy text abstract, which can improve the efficiency of generating a policy text abstract.

In a first aspect, an embodiment of the present invention provides a method for generating a policy text summary, including:

Obtain the candidate title of the candidate policy text, classify the candidate title, and obtain the policy category corresponding to the candidate policy text;

Determine text keywords from the candidate policy texts, determine the first frequency of the text keywords appearing in a preset keyword database, and determine the text importance of the candidate policy texts according to the first frequency;

determining a target policy text among the candidate policy texts of each of the policy categories according to the text importance;

A target key sentence is extracted from the target policy text, and a policy summary of the policy category corresponding to the target policy text is generated according to the target key sentence.

Further, classifying the candidate titles to obtain a policy category corresponding to the candidate policy text, including:

Perform word segmentation processing on the candidate title to obtain title keywords;

determining the second frequency of the title keyword appearing in the candidate policy text, and calculating the keyword vector of the title keyword according to the second frequency;

Calculate the title vector of the candidate title according to the keyword vector;

Classify the candidate title according to the title vector to obtain the policy category corresponding to the candidate policy text.

Further, the determining text keywords from the candidate policy texts includes:

Perform word segmentation processing on the candidate policy text to obtain text candidate words;

Calculate the keyword score of the text candidate word in the candidate policy text;

Sort the text candidates according to the keyword scores in descending order, and determine the text candidates whose ranking is before the first threshold as text keywords; or according to the keyword scores by The text candidate words are sorted in order from small to large, and the text candidate words ranked after the second threshold are determined as text keywords.

Further, the determining the text importance of the candidate policy text according to the first frequency includes:

Obtain the word importance of the text keyword according to the product of the first frequency and the keyword score corresponding to the text keyword;

According to the sum of the word importances of all the text keywords in the candidate policy text, the text importance of the candidate policy text is obtained.

Further, determining a target policy text from the candidate policy texts of each of the policy categories according to the text importance includes:

In each of the policy categories, the candidate policy texts are sorted in descending order of the importance of the texts, and the candidate policy texts ranked before the third threshold are determined as the corresponding policy categories target policy text in;

Alternatively, in each of the policy categories, the candidate policy texts are sorted in descending order of the importance of the texts, and the candidate policy texts ranked after the fourth threshold are determined as the corresponding The target policy text in the policy category.

Further, the extraction of target key sentences from the target policy text includes:

Perform sentence segmentation processing on the target policy text to obtain target candidate sentences;

Perform vectorization processing on the target candidate sentence to obtain a candidate sentence vector;

Calculate the similarity value between each two of the target candidate sentences according to the candidate sentence vector;

Calculate the candidate sentence score corresponding to the target candidate sentence according to the similarity value;

Sort the target candidate sentences according to the scores of the candidate sentences in descending order, and determine the target candidate sentences ranked before the fifth threshold as the target key sentences; or according to the score of the candidate sentences by The target candidate sentences are sorted in order from small to large, and the target candidate sentences ranked after the sixth threshold are determined as target key sentences.

Further, calculating the candidate sentence score corresponding to the target candidate sentence according to the similarity value includes:

The target candidate sentence is used as a sentence node, and the similarity value is used as a connection edge between two corresponding sentence nodes, and a candidate sentence graph is constructed according to the sentence node and the connection edge;

determining an initial score and a probability coefficient of the sentence node, where the probability coefficient is used to represent the probability that the sentence node points to any other node in the candidate sentence graph;

The initial score is propagated and updated in the candidate sentence graph according to the probability coefficient and the similarity value until convergence, and the initial score at the time of convergence is used as the candidate sentence score corresponding to the target candidate sentence.

In a second aspect, an embodiment of the present invention further provides an apparatus for generating a policy text summary, including:

a policy text classification module, configured to obtain candidate titles of candidate policy texts, classify the candidate titles, and obtain policy categories corresponding to the candidate policy texts;

A text importance determination module, configured to determine text keywords from the candidate policy text, determine the first frequency of the text keywords appearing in a preset keyword database, and determine the candidate according to the first frequency The textual importance of the policy text;

a target policy text determination module, configured to determine target policy texts in the candidate policy texts of each of the policy categories according to the text importance;

A policy summary generating module is configured to extract a target key sentence from the target policy text, and generate a policy summary of the policy category corresponding to the target policy text according to the target key sentence.

In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the policy text summary of the first aspect when executing the computer program Generate method.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the storage medium stores a program, and the program is executed by a processor to implement the method for generating a policy text summary according to the first aspect.

The embodiments of the present invention at least include the following beneficial effects:

A method for generating a policy text abstract proposed by an embodiment of the present invention obtains the candidate titles of the candidate policy texts, classifies the candidate titles, obtains the policy categories corresponding to the candidate policy texts, determines the text keywords from the candidate policy texts, and determines the texts. The first frequency of the keyword appearing in the preset keyword database, the text importance of the candidate policy text is determined according to the first frequency, and the target policy text is determined in the candidate policy text of each policy category according to the text importance, and the target policy text is determined from the target policy text. The target key sentence is extracted from the text, and the policy summary of the policy category corresponding to the target policy text is generated according to the target key sentence. Compared with manual sorting, the policy summary can be automatically and quickly generated, and the efficiency of generating the policy text summary is improved; and based on the candidate title Determine the policy category of the candidate policy text, so that the generated policy summary corresponds to the policy category of the target policy text, which can achieve the classification effect of different policy summaries, which is convenient for readers to read; in addition, the candidate policy is determined by the first frequency of text keywords. The text importance of the text can then determine the more important target policy texts in different policy categories, and then generate policy summaries through the target policy texts, which is beneficial to improve the accuracy of generating policy summaries.

Other features and advantages of the present invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the description, claims and drawings.

Description of drawings

The accompanying drawings are used to provide a further understanding of the technical solutions of the present invention, and constitute a part of the description. They are used to explain the technical solutions of the present invention together with the embodiments of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention;

2 is a schematic flowchart of a method for generating a policy text summary provided by an embodiment of the present invention;

3 is a schematic diagram of a specific flow for classifying candidate titles provided by an embodiment of the present invention;

4 is a schematic diagram of a classification example of candidate policy texts provided by an embodiment of the present invention;

5 is a schematic diagram of a specific flow of determining text keywords provided by an embodiment of the present invention;

6 is a schematic diagram of a specific flow of determining the importance of a text provided by an embodiment of the present invention;

7 is a schematic diagram of a specific flow of extracting target key sentences provided by an embodiment of the present invention;

8 is a schematic diagram of a specific flow of calculating candidate sentence scores according to an embodiment of the present invention;

9 is a schematic structural diagram of an apparatus for generating a policy text summary provided by an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

It should be understood that, in the description of the embodiments of the present invention, the meaning of several is more than one, the meaning of multiple (or multiple) is more than two, and greater than, less than, exceeding, etc. are understood as not including this number, above, below, Within, etc., are understood to include this number. If there is a description of "first", "second", etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance, or implicitly indicating the number of indicated technical features or implicitly indicating the indicated The sequence of technical characteristics.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein are for the purpose of describing the embodiments of the present invention only, and are not intended to limit the present invention.

First, some terms involved in this application are analyzed:

Artificial intelligence (AI): It is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science, artificial intelligence. Intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, language recognition, image recognition, natural language processing, and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

The embodiments of the present application may acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. .

With the advent of the information society, the electronic version of the policy news has replaced the paper version of the policy news. Among them, the Policy Morning Post is a form of policy news, and the Policy Morning Post is a collection of main policy summaries generated based on the daily updated policies in the policy database, in order to facilitate readers to understand the important policy and current affairs of the day in a short time. Therefore, for people who are always concerned about the direction of the policy and lack sufficient reading time, the Policy Morning Post is the first choice for reading.

In the related technology, the policy morning report is generated by manual daily regular sorting. It is necessary to understand the policy text based on the experience of the organizer, and then extract the policy summary and organize it into a policy morning report. However, using manual extraction of policy summaries not only consumes manpower, but also reduces the efficiency of generating policy summaries.

Based on this, embodiments of the present invention provide a method, device, electronic device, and storage medium for generating a policy text summary, which can improve the efficiency of generating a policy text summary.

The method, device, electronic device, and storage medium for generating a policy text abstract provided by the embodiments of the present invention are specifically described by the following embodiments. First, the method for generating a policy text abstract in the embodiment of the present disclosure is described.

Referring to FIG. 1 , FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. The implementation environment includes a terminal 101 and a server 102 , wherein the terminal 101 and the server 102 are connected through a communication network 103 .

The server 102 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or may provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, Cloud servers for basic cloud computing services such as middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

In addition, the server 102 may also be a node server in the blockchain network.

The terminal 101 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, etc., but is not limited thereto. The terminal 101 and the server 102 may be directly or indirectly connected through wired or wireless communication, which is not limited in this embodiment of the present invention.

Based on the implementation environment shown in FIG. 1 , an embodiment of the present invention provides a method for generating policy text summaries, and the method for generating policy text summaries can be executed by the server 102 shown in FIG. 1 , or by the terminal 101 shown in FIG. 1 . Executed, or executed cooperatively by the terminal 101 and the server 102 shown in FIG. 1 , the embodiment of the present invention is described by taking the method for generating a policy text summary executed by the server 102 shown in FIG. 1 as an example.

Referring to FIG. 2 , FIG. 2 is a schematic flowchart of a method for generating a policy text summary provided by an embodiment of the present invention. The method for generating a policy text summary includes but is not limited to the following steps 201 to 203 .

Step 201: Obtain the candidate title of the candidate policy text, classify the candidate title, and obtain the policy category corresponding to the candidate policy text;

Step 202: Determine the text keywords from the candidate policy texts, determine the first frequency of the text keywords appearing in the preset keyword database, and determine the text importance of the candidate policy texts according to the first frequency;

Step 203: Determine the target policy text from the candidate policy texts of each policy category according to the text importance;

Step 204 : extracting the target key sentence from the target policy text, and generating a policy summary of the policy category corresponding to the target policy text according to the target key sentence.

The number of candidate policy texts may be multiple, and the candidate policy texts may be current affairs policies, including but not limited to different policy categories such as economic policies, industry policies, and local policies. In a possible implementation manner, the news texts of which the news category published by the website on the current day is a current affairs policy may be obtained, so as to obtain candidate policy texts. Specifically, it is possible to obtain news texts published on the current day from a preset website, and obtain news texts whose news category is current affairs policy from multiple news texts, so as to obtain candidate policy texts, wherein the preset websites include news websites, One or more kinds of consulting websites, video websites, etc. In actual application, the websites to be obtained can be set according to the requirements. Among them, obtaining the news text released on the current day mainly obtains the release date of the news text, and then obtains the date of the current day to obtain the news text whose release date is the same as the date of the current day, and obtains the news category of the news text to obtain the news whose news category is the current affairs policy. text gets the candidate policy text.

Among them, the candidate title is used to summarize the content of the candidate policy text. For example, the candidate title can be "Interpretation of the digital economy and industrial policy system in the L region". By classifying the candidate title, the policy category corresponding to the candidate policy text can be obtained. For example, according to the above-mentioned candidate title "Interpretation of Digital Economy Industrial Policy System in Region L", it can be determined that the policy category of the corresponding candidate policy text is local policy or economic policy.

Among them, there are multiple text words in the candidate policy text, and not all the text words need to be calculated in the preset keyword database. It is only necessary to obtain the key text words as the text keywords of the candidate policy text, and then according to the text The frequency of occurrence of the keyword in the preset keyword database determines the importance of the candidate policy.

The keyword database can be preset. In a possible implementation, the text keywords of all candidate policy texts within a preset historical time range can be counted, and then the text keywords of all candidate policy texts can be placed in the The preset keyword database is stored, and the preset key database stores the text keywords of each candidate policy text within the preset historical time range. For example, the preset historical time range is the previous month, that is, the text keywords of all candidate policy texts within one month before the current time are obtained and stored in the preset keyword database. The first frequency of the text keywords of the current candidate policy texts is calculated, and the text can be obtained directly according to whether the text keywords of the current candidate policy texts are consistent with the text keywords of the candidate policy texts stored in the preset keyword database. The first frequency that the keyword appears. For example, if the text keyword of 100 candidate policy texts stored in the preset keyword database contains "carbon neutrality", the first frequency of the text keyword "carbon neutrality" in the current candidate policy text is 100 . The higher the first frequency of text keywords in the preset keyword database, the higher the text importance of the candidate policy text.

Among them, in the embodiment of the present invention, classification is performed according to the policy category of the candidate policy text, and there are corresponding candidate policy texts in different policy categories, and then the candidate policy text with the highest text importance can be determined as the corresponding policy category. Target policy text. For example, the candidate policy texts include: A, B, C, D, E, F, G, H, and the candidate policy texts that divide the multiple candidate policy texts into the first policy category according to the policy category include: A, C, E; The candidate policy texts of the second policy category include: B, D, G; the candidate policy texts of the third policy category include: F, H; the candidate policy text with the highest text importance in the first policy category is C, and the second policy category The candidate policy text with the highest Chinese text importance is D, and the candidate policy text with the highest text importance in the third policy category is F. Therefore, the target policy text of the first policy category is determined as the candidate policy text C, and the target of the second policy category is determined as the candidate policy text C. The policy text is candidate policy text D, and the target policy text of the third policy category is candidate policy text F. Therefore, target policy texts of different policy categories are determined, and corresponding policy morning reports are formed, so as to save the time for users to view policy information. It can be understood that when classifying candidate policy texts, the same candidate policy text can be classified into different policy classes. For example, the above candidate policy text A can be classified into the first policy class or into the second policy class. .

In the method for generating a policy text abstract provided by the embodiment of the present invention, the candidate titles of the candidate policy texts are obtained, the candidate titles are classified, the policy categories corresponding to the candidate policy texts are obtained, the text keywords are determined from the candidate policy texts, and the text keywords are determined. The first frequency that appears in the preset keyword database, the text importance of the candidate policy text is determined according to the first frequency, and the target policy text is determined in the candidate policy texts of each policy category according to the text importance, and the target policy text is determined from the target policy text. The target key sentence is extracted, and the policy summary of the policy category corresponding to the target policy text is generated according to the target key sentence. Compared with manual sorting, the policy summary can be automatically and quickly generated, and the efficiency of generating the policy text summary is improved; and the candidate title is determined based on the candidate title. The policy category of the policy text makes the generated policy summary correspond to the policy category of the target policy text, which can achieve the classification effect of different policy summaries and facilitate readers to read; in addition, the first frequency of the text keywords to determine the candidate policy text. The importance of the text can be used to determine the more important target policy texts in different policy categories, and then generate policy summaries through the target policy texts, which is beneficial to improve the accuracy of generating policy summaries.

In a possible implementation, referring to FIG. 3, FIG. 3 is a schematic diagram of a specific flow of classifying candidate titles provided by an embodiment of the present invention. In the above step 201, the candidate titles are classified to obtain the policy corresponding to the candidate policy text The category may specifically include the following steps 301 to 304 .

Step 301: perform word segmentation processing on candidate titles to obtain title keywords;

Step 302: Determine the second frequency of the title keyword appearing in the candidate policy text, and calculate the keyword vector of the title keyword according to the second frequency;

Step 303: Calculate the title vector of the candidate title according to the keyword vector;

Step 304: Classify the candidate titles according to the title vector to obtain the policy category corresponding to the candidate policy text.

Among them, the word segmentation processing of the candidate title can be realized by using analysis software. The stop words in the candidate titles are removed, and then the candidate titles with the stop words removed are segmented by word segmentation software to obtain title keywords. Stop words include: punctuation marks, numbers, English letters, etc., and the analysis software used can be jieba software. Among them, the principle of jieba software for word segmentation mainly includes full mode, precise mode and search engine mode. The full mode is to scan all possible words in the sentence; the precise mode is to try to cut the sentence most accurately. , suitable for text segmentation; the search engine mode is based on the precise mode, and the long words (words > 2) are segmented again to improve the recall rate, which is suitable for search engine segmentation. In this embodiment, the full mode word segmentation is used to separate out the words that may become words in the candidate title. For example, if the candidate title is "Interpretation of the "1+N+S" digital economy industrial policy system in L region", remove the stop words in the candidate title to obtain "Interpretation of the digital economy industrial policy system in L region", and then use The jieba software performs word segmentation on the candidate titles, and obtains the title keywords as "L region, numbers, economy, industry, policy, system, interpretation".

After dividing the candidate title into multiple title keywords, it is necessary to calculate the second frequency of each title keyword appearing in the candidate policy, and then determine the title keyword according to the second frequency of each title keyword appearing in the candidate policy. vector. In a possible implementation, the second frequency of each title keyword appearing in the candidate policy can be directly used as the vector of the title keyword. For example, if the second frequency is 3, the vector of the title keyword is "3" . In addition, TF-IDF (Term Frequency-Inverse Document Frequency, term frequency-inverse document frequency) can also be used to calculate the vector of title keywords; the main idea of TF-IDF is: if a certain word or phrase is in an article If the frequency of occurrence TF is high, and it rarely appears in other articles, it is considered that the word or phrase has a good ability to distinguish between categories and is suitable for classification. TF-IDF is actually: TF*IDF, TF term frequency (Term Frequency), IDF inverse document frequency (InverseDocument Frequency). TF represents the second frequency of the term appearing in document d. The main idea of IDF is: if there are fewer documents containing term t, that is, the smaller n is, and the larger the IDF is, it means that term t has a good ability to distinguish categories. Therefore, the obtained title keywords are calculated using the tfidf method to calculate the probability of each title keyword appearing in the candidate policy text to obtain the title keyword vector, which is used as the vector input for subsequent model training. For example, the title keywords obtained by word segmentation processing are "L region, number, economy, industry, policy, system, interpretation", and then the vector of title keywords calculated by the tfidf method is "0.023, 0.021, 0, 0, 0.002... …”.

After calculating the vector value of each title keyword, the title vector of the candidate title can be obtained by splicing the vector values of all the title keywords. For example, based on the above example, the vectors of title keywords are respectively [0.023, 0.021, 0, 0, 0.002...].

Next, the title vector is fed into the SVM model to classify candidate titles. Among them, the SVM model is also called the support vector machine model, and the hard-bound SVM is used in this example. This method is an algorithm for solving the maximum-margin hyperplane in a linearly separable problem. The constraints are The distance from the sample point to the decision boundary is greater than or equal to 1. The hard boundary SVM can be transformed into an equivalent quadratic convex optimization problem to solve. By processing the title vector through the SVM model, the classification value of the title vector can be obtained, and this classification value can be used to represent the policy category to which the corresponding candidate policy text belongs.

Among them, the classification value also represents the classification result of the title keyword in each section, and then the policy category of the candidate policy text is determined according to the classification value of the title keyword. For example, the classification value of the above title vector is [0 0 1 0 0 10], where there are 7 elements in the classification value, each element in turn represents seven different policy categories, and an element of "1" indicates that the title vector belongs to The policy category, an element of "0" indicates that the title vector does not belong to this policy category. Therefore, according to the above classification values, it is obtained that the candidate policies belong to the third policy category and the sixth policy category.

For example, referring to FIG. 4, FIG. 4 is a schematic diagram of a classification example of the candidate policy text provided by the embodiment of the present invention. If the title text of the input candidate policy text is "L region "1+N+S" Digital Economy Industrial Policy System Interpretation" , the title text is divided into words, and the title keywords are "L region, number, economy, industry, policy, system, interpretation". Then the TF-IDF method is used to calculate the vector of title keywords as "0.023, 0.021, 0, 0, 0.002...", and then the vector value of each title keyword is classified according to the SVM model to obtain the classification value of the policy is [0 0 1 0 0 1 0], assuming that the third and sixth policy categories are local policies and economic policies, so the policy categories that get the candidate policy text are local policies and economic policies.

In a possible implementation manner, referring to FIG. 5, FIG. 5 is a schematic diagram of a specific flow of determining text keywords provided by an embodiment of the present invention. In the above step 202, determining text keywords from candidate policy texts may specifically include the following Steps 501 to 503 .

Step 501: Perform word segmentation processing on the candidate policy text to obtain text candidate words;

Step 502: Calculate the keyword scores of the text candidate words in the candidate policy text;

Step 503: Sort the text candidates according to the order of the keyword scores from large to small, and determine the text candidate words ranked before the first threshold as the text keywords; or according to the order of the keyword scores from small to large The text candidate words are sorted, and the text candidate words ranked after the second threshold are determined as text keywords.

Among them, the word segmentation processing of the candidate policy text can also be realized by using analysis software. Calculating the keyword scores of text candidate words in candidate policy texts can be calculated using the above TF-IDF. Among them, when calculating the keyword scores of text candidate words in candidate policy texts, IDF is based on all historical policies. Text to calculate, so that the keyword score is more reasonable and accurate.

After obtaining the keyword scores of multiple text candidate words, the text keywords can be sorted according to the keyword scores to obtain a sorting order, and the previous preset number of text candidate words are obtained as text keywords according to the sorting order, that is, Get a few words that represent candidate policies. The sorting method may be in descending order of keyword scores or in descending order of keyword scores, which is not limited in the embodiment of the present invention, and the first threshold and the second threshold may be based on actual conditions. The setting, for example, may be 50, 70, 100, etc., which is not limited in the embodiment of the present invention. Take the sorting of text candidates according to the order of keyword scores from large to small as an example, the text candidates include: K1, K2, K3, K4, K5, K6, K7, K8, assuming that the first threshold is 3, the text The ranking results of the candidate words are K3, K4, K5, K1, K2, K6, K7, K8, so the text keywords are determined to be K3, K4, K5.

Calculate the keyword score of the candidate text word in the candidate policy text. In addition to determining the text keyword, the keyword score corresponding to the text keyword and the first frequency of the text keyword appearing in the preset keyword database are also calculated. combined to determine the text importance of candidate policy texts. Referring to FIG. 6 , FIG. 6 is a schematic diagram of a specific flow of determining text importance provided by an embodiment of the present invention. In the above step 202 , determining the text importance of candidate policy texts according to the first frequency may specifically include the following steps 601 to 602 .

Step 601: Obtain the word importance of the text keyword according to the product of the first frequency and the keyword score corresponding to the text keyword;

Step 602: Obtain the text importance of the candidate policy text according to the sum of the word importances of all text keywords in the candidate policy text.

Specifically, the importance of the text keywords is obtained by multiplying the keyword scores of the text keywords of the candidate policy text and the first frequency of the text keywords in the previous preset time period, and then all the text keywords in the candidate policy text are The importance of the candidate policy text is obtained by summing the importance of the word, so that the policy text can be screened by the importance of the candidate policy text, so as to obtain the candidate policy text with the highest text importance as the target policy text of the policy category.

Among them, the key scores of text candidates in the full text of the policy can be calculated by TF-IDF. The main idea of TF-IDF is: if a word or phrase appears frequently in one article TF, and in other If it rarely appears in the article, it is considered that the word or phrase has a good ability to distinguish between categories and is suitable for classification. Therefore, the key score of each text candidate word is determined according to the frequency of the text candidate word in the full text of the candidate policy text, so as to judge the importance of the text candidate word to the candidate policy text by the key score.

Among them, the formula for calculating the text importance of candidate policy texts is as follows:

In the formula, s _j represents the importance of the text, tfidf _i represents the keyword score of the text keywords of the current candidate policy text, kf _i represents the first frequency of text keywords, and i, j are positive integers.

The importance of the text keywords can be obtained by multiplying the keyword scores of the text keywords of the candidate policy text and the first frequency of the text keywords in the previous preset time period, and the keyword scores can be correlated with the first frequency. Combined, the text importance of the candidate policy text can better reflect the importance of the candidate policy text, making the text importance of the candidate policy text more reasonable and accurate.

In a possible implementation manner, in the above step 203, the target policy text is determined from the candidate policy texts of each policy category according to the text importance. Specifically, in each policy category, the text importance may be in descending order. Sort the candidate policy texts, and determine the candidate policy texts ranked before the third threshold as the target policy texts in the corresponding policy category;

Alternatively, in each policy category, the candidate policy texts are sorted in ascending order of text importance, and the candidate policy text ranked after the fourth threshold is determined as the target policy text in the corresponding policy category.

The third threshold and the fourth threshold may be set according to actual conditions, for example, may be 1, 2, 3, etc., which are not limited in this embodiment of the present invention. For example, taking the example of sorting candidate policy texts in descending order of text importance, the third threshold is 1, and the candidate policy texts include: A, B, C, D, E, F, G, H, according to The policy category divides multiple candidate policy texts into the candidate policy texts of the first policy category including: A, C, E; the candidate policy texts of the second policy category include: B, D, G; the candidate policy texts of the third policy category Including: F, H; the ranking results of text importance in the first policy category are C, A, E, the ranking results of text importance in the second policy category are D, G, B, and the text importance in the third policy category The sorting results are F and H, so the target policy text of the first policy category is determined as the candidate policy text C, the target policy text of the second policy category is the candidate policy text D, and the target policy text of the third policy category is the candidate policy text. F.

In a possible implementation, referring to FIG. 7 , FIG. 7 is a schematic diagram of a specific flow of extracting target key sentences provided by an embodiment of the present invention. In the above step 204, the target key sentences are extracted from the target policy text, which may specifically include The following steps 701 to 705.

Step 701: Perform sentence segmentation processing on the target policy text to obtain target candidate sentences;

Step 702: Perform vectorization processing on the target candidate sentence to obtain a candidate sentence vector;

Step 703: Calculate the similarity value between each two target candidate sentences according to the candidate sentence vector;

Step 704: Calculate the candidate sentence score corresponding to the target candidate sentence according to the similarity value;

Step 705: Sort the target candidate sentences in descending order of the candidate sentence scores, and determine the target candidate sentence ranked before the fifth threshold as the target key sentence; or in the order of the candidate sentence scores from small to large The target candidate sentences are sorted, and the target candidate sentences ranked after the sixth threshold are determined as target key sentences.

Among them, analysis software can be used to process the target policy text into sentences to obtain target candidate sentences, and the basis for the sentence processing can be punctuation marks. Then, the Word2vec model can be used to convert the target candidate sentence into a candidate sentence vector. Word2vec is a word vector model, which converts the words in each target candidate sentence in the target policy text into the corresponding word vector, and then splices the word vectors of the words in the target candidate sentence to convert the target candidate sentence into a candidate sentence vector .

Among them, the similarity value between each two target candidate sentences is calculated according to the candidate sentence vectors, the cosine similarity between the two candidate sentence vectors can be calculated, or the Euclidean distance between the two candidate sentence vectors can also be calculated, This embodiment of the present invention is not limited. It should be added that, in the embodiment of the present invention, the similarity value between each two target candidate sentences is calculated. For example, the target candidate sentences include S1, S2, S3 and S4, and the target candidate sentence S1 needs to be Calculate the similarity value between the target candidate sentence S1 and the target candidate sentence S2, calculate the similarity value between the target candidate sentence S1 and the target candidate sentence S3, and calculate the similarity between the target candidate sentence S1 and the target candidate sentence S4 value.

Then, after calculating the candidate sentence score corresponding to the target candidate sentence according to the similarity value, the target candidate sentence is sorted according to the candidate sentence score, and then the target key sentence is determined. The fifth threshold and the sixth threshold may be set according to actual conditions, for example, may be 1, 2, 3, etc., which are not limited in this embodiment of the present invention. For example, take the example of sorting the target candidate sentences in descending order of the candidate sentence scores, the fifth threshold is 1, and the target candidate sentences include: S1, S2, S3, S4, S5, S6, S7, S8, Assuming that the fifth threshold is 2, the ranking results of the target candidate sentences are S3, S4, S5, S1, S2, S6, S7, and S8, so the target key sentences are determined to be S3 and S4.

In a possible implementation, referring to FIG. 8 , FIG. 8 is a schematic diagram of a specific flow of calculating a candidate sentence score provided by an embodiment of the present invention. In the above step 704, the candidate sentence score corresponding to the target candidate sentence is calculated according to the similarity value. value, which may specifically include the following steps 801 to 803 .

Step 801: The target candidate sentence is used as a sentence node, and the similarity value is used as a connection edge between the corresponding two sentence nodes, and a candidate sentence graph is constructed according to the sentence node and the connection edge;

Step 802: Determine the initial score and probability coefficient of the sentence node;

Step 803: Propagating and updating the initial score in the candidate sentence graph according to the probability coefficient and the similarity value until convergence, and using the initial score at the time of convergence as the candidate sentence score corresponding to the target candidate sentence.

Among them, a sentence node corresponds to a target candidate sentence, and the candidate sentence graph can be expressed as G=(V, E), where G is the candidate sentence graph, V is the sentence node set, and E is the connecting edge set. The weight of the connection edge between any two sentence nodes V _i and V _j in the candidate sentence graph is W _ji . In the embodiment of the present invention, the similarity value is used as the weight. For a given sentence node V _i , In(V _i ) is the set of sentence nodes pointing to the sentence node, and Out(V _i ) is the set of sentence nodes pointed to by the sentence node V _i .

The candidate sentence score can be calculated using the following formula:

Among them, WS(V _i ) represents the candidate sentence score of the sentence node V _i , WS(V _j ) represents the candidate sentence score of the sentence node V _j , d represents the probability coefficient, and the probability coefficient is used to indicate that the sentence node points to the candidate sentence graph the _probability of _{any other node in} One of the sentence nodes of , V _k is one of the sentence nodes of the sentence node set Out(V _i ), W _ji is the similarity value between the sentence nodes V _i and V _j , W _jk is the sentence nodes V _k , V _j similarity value between. The value range of the probability coefficient is 0 to 1, which represents the probability of pointing from a certain point in the figure to any other point. In this embodiment of the present invention, the value of the probability coefficient is 0.85. It can be understood that the probability coefficient can be determined according to The actual setting is not limited in this embodiment of the present invention.

Among them, each sentence node in the candidate sentence graph is given an initial score, and then the initial score of each sentence node is propagated and updated in the candidate sentence graph according to the probability coefficient and similarity value until convergence, and according to the above candidate sentence scores The formula is iteratively calculated, and the initial score at the time of convergence is used as the candidate sentence score corresponding to the target candidate sentence. By constructing the candidate sentence graph, and calculating the candidate sentence score according to the probability coefficient and similarity value in the candidate sentence graph, the candidate sentence score can be efficiently and accurately calculated, and then the target key sentence can be determined according to the candidate sentence score.

The number of target key sentences can be one or more, and when multiple target key sentences are obtained, the multiple target key sentences can be combined to obtain a policy summary. And, the finally generated policy summary serves as the policy summary in the policy category it belongs to. Since the policy morning paper is to allow readers to quickly obtain the important policy content of the day, the reading time should not be too long, so the policy summary extraction is to obtain the summary information in the policy text, so that readers do not need to spend time reading the full text, but also can understand to the central idea that this policy needs to express.

It can be understood that, although the steps in the above-mentioned flowcharts are displayed in sequence according to the arrows, these steps are not necessarily executed in the sequence indicated by the arrows. Unless explicitly stated in this embodiment, the execution of these steps is not strictly limited in sequence, and these steps may be executed in other sequences. Moreover, at least a part of the steps in the above flow chart may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution sequence of these steps or stages It is also not necessarily performed sequentially, but may be performed alternately or alternately with other steps or at least a portion of a step or phase within the other steps.

In addition, referring to FIG. 9, FIG. 9 is a schematic structural diagram of an apparatus for generating a policy text summary provided by an embodiment of the present invention. An embodiment of the present invention further provides a policy text summary generating apparatus, and the policy text summary generating apparatus 900 includes:

The policy text classification module 901 is used to obtain candidate titles of candidate policy texts, classify the candidate titles, and obtain policy categories corresponding to the candidate policy texts;

A text importance determination module 902, configured to determine text keywords from the candidate policy texts, determine the first frequency of the text keywords appearing in the preset keyword database, and determine the text importance of the candidate policy texts according to the first frequency;

a target policy text determination module 903, configured to determine target policy texts from candidate policy texts of each policy category according to the text importance;

The policy summary generating module 904 is used for extracting target key sentences from the target policy text, and generating a policy summary of the policy category corresponding to the target policy text according to the target key sentences.

The policy text summary generating apparatus 900 proposed in the embodiment of the present invention is based on the same inventive concept as the policy text summarizing method. The policy text summary generating apparatus 900 obtains candidate titles of candidate policy texts, classifies the candidate titles, and obtains candidate titles. The policy category corresponding to the policy text, determine the text keywords from the candidate policy texts, determine the first frequency of the text keywords appearing in the preset keyword database, and determine the text importance of the candidate policy texts according to the first frequency. The importance determines the target policy text in the candidate policy texts of each policy category, extracts the target key sentence from the target policy text, and generates the policy summary of the policy category corresponding to the target policy text according to the target key sentence. And quickly generate policy summaries, improve the efficiency of generating policy text summaries; and determine the policy categories of candidate policy texts based on candidate titles, so that the generated policy summaries correspond to the policy categories of the target policy texts, and can achieve the classification effect of different policy summaries, which is convenient for Readers read it; in addition, the text importance of candidate policy texts is determined by the first frequency of text keywords, and then the more important target policy texts in different policy categories can be determined, and subsequent policy summaries are generated through the target policy texts. It is beneficial to improve the accuracy of generating policy summaries.

It can be understood that, each of the foregoing semantic recognition apparatuses may also be specifically configured to execute various processes described in the foregoing semantic recognition method embodiments.

Referring to FIG. 10 , FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device 1000 includes: a memory 1001, a processor 1002, and a computer program stored in the memory 1001 and running on the processor 1002, and the computer program is used to execute the above-mentioned semantic recognition method when running.

The processor 1002 and the memory 1001 may be connected by a bus or other means.

As a non-transitory computer-readable storage medium, the memory 1001 can be used to store non-transitory software programs and non-transitory computer-executable programs, such as the semantic recognition method described in the embodiments of the present invention. The processor 1002 implements the above-mentioned semantic recognition method by running the non-transitory software programs and instructions stored in the memory 1001 .

The memory 1001 may include a storage program area and a storage data area, wherein the storage program area may store an operating system and an application program required by at least one function; the storage data area may store and execute the above-mentioned semantic recognition method. Additionally, memory 1001 may include high-speed random access memory 1001, and may also include non-transitory memory 1001, such as at least one storage device storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1001 may optionally include memory 1001 located remotely from the processor 1002, and these remote memories 1001 may be connected to the electronic device 1000 through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The non-transitory software programs and instructions required to implement the above-mentioned semantic recognition method are stored in the memory 1001 , and when executed by one or more processors 1002 , execute the above-mentioned semantic recognition method.

Embodiments of the present invention further provide a computer-readable storage medium storing computer-executable instructions, where the computer-executable instructions are used to execute the foregoing semantic recognition method.

In one embodiment, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more control processors to implement the above-mentioned semantic recognition method.

The apparatus embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

Those of ordinary skill in the art can understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit . Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules or other data flexible, removable and non-removable media. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tapes, storage device storage or other magnetic storage devices, or Any other medium that can be used to store the desired information and that can be accessed by a computer. In addition, communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery media, as is well known to those of ordinary skill in the art .

It should also be understood that various implementation manners provided in the embodiments of the present invention may be arbitrarily combined to achieve different technical effects.

The preferred implementation of the present invention has been specifically described above, but the present invention is not limited to the above-mentioned embodiments. Those skilled in the art can also make various equivalent deformations or replacements on the premise of not violating the spirit of the present invention. These Equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims (10)

1. A policy text summary generation method is characterized by comprising the following steps:

obtaining candidate titles of candidate policy texts, and classifying the candidate titles to obtain policy categories corresponding to the candidate policy texts;

determining text keywords from the candidate policy text, determining first frequency of the text keywords appearing in a preset keyword database, and determining text importance of the candidate policy text according to the first frequency;

determining a target policy text in the candidate policy texts of the policy categories according to the text importance;

and extracting a target key sentence from the target policy text, and generating a policy abstract of the policy category corresponding to the target policy text according to the target key sentence.

2. The method of claim 1, wherein the classifying the candidate titles to obtain the policy categories corresponding to the candidate policy texts comprises:

performing word segmentation processing on the candidate titles to obtain title keywords;

determining a second frequency of the title keywords appearing in the candidate policy text, and calculating keyword vectors of the title keywords according to the second frequency;

calculating title vectors of the candidate titles according to the keyword vectors;

and classifying the candidate titles according to the title vectors to obtain policy categories corresponding to the candidate policy texts.

3. The method of claim 1 wherein the determining text keywords from the candidate policy text comprises:

performing word segmentation processing on the candidate policy text to obtain text candidate words;

calculating a keyword score of the text candidate word in the candidate policy text;

sorting the text candidate words according to the descending order of the keyword scores, and determining the text candidate words with the ranking before a first threshold value as text keywords; or sorting the text candidate words according to the sequence of the scores of the keywords from small to large, and determining the text candidate words ranked after a second threshold value as text keywords.

4. The method of claim 3, wherein the determining the text importance of the candidate policy text according to the first frequency comprises:

obtaining word importance of the text keywords according to the product of the first frequency and the keyword scores corresponding to the text keywords;

and obtaining the text importance of the candidate policy text according to the sum of the word importance of all the text keywords in the candidate policy text.

5. The method of claim 1, wherein the determining a target policy text among the candidate policy texts in each policy category according to the text importance comprises:

in each policy category, sorting the candidate policy texts according to the sequence of the text importance degrees from large to small, and determining the candidate policy texts with the ranking before a third threshold value as target policy texts in the corresponding policy category;

or in each policy category, the candidate policy texts are sorted according to the order of the text importance degrees from small to large, and the candidate policy texts ranked after the fourth threshold are determined as the target policy texts in the corresponding policy category.

6. The method of claim 1, wherein the extracting target key sentences from the target policy text comprises:

performing sentence division processing on the target policy text to obtain a target candidate sentence;

vectorizing the target candidate sentence to obtain a candidate sentence vector;

calculating the similarity value between every two target candidate sentences according to the candidate sentence vectors;

calculating a candidate sentence score corresponding to the target candidate sentence according to the similarity value;

sequencing the target candidate sentences according to the sequence of the candidate sentence values from large to small, and determining the target candidate sentences ranked before a fifth threshold value as target key sentences; or sequencing the target candidate sentences according to the sequence of the candidate sentence values from small to large, and determining the target candidate sentences ranked after a sixth threshold value as target key sentences.

7. The method of claim 6, wherein the calculating a candidate sentence score corresponding to the target candidate sentence according to the similarity value comprises:

taking the target candidate sentence as a sentence node, taking the similarity value as a connecting edge between two corresponding sentence nodes, and constructing a candidate sentence graph according to the sentence node and the connecting edge;

determining an initial score and a probability coefficient of the sentence node, wherein the probability coefficient is used for representing the probability that the sentence node points to any other node in the candidate sentence graph;

and propagating and updating the initial score in the candidate sentence graph until convergence according to the probability coefficient and the similarity value, and taking the initial score during convergence as a candidate sentence score corresponding to the target candidate sentence.

8. A policy text summary generation apparatus, comprising:

the policy text classification module is used for acquiring candidate titles of the candidate policy texts, classifying the candidate titles and obtaining policy categories corresponding to the candidate policy texts;

the text importance determining module is used for determining text keywords from the candidate policy text, determining the first frequency of the text keywords appearing in a preset keyword database, and determining the text importance of the candidate policy text according to the first frequency;

a target policy text determining module, configured to determine a target policy text from the candidate policy texts in each policy category according to the text importance;

and the policy abstract generating module is used for extracting a target key sentence from the target policy text and generating a policy abstract of the policy category corresponding to the target policy text according to the target key sentence.

9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the policy text summary generating method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that the storage medium stores a program, the program being executed by a processor to implement the policy text digest generation method according to any one of claims 1 to 7.

Images