TWI748749B - Method for automatic extraction of text classification and text keyword and device using same - Google Patents

Abstract

The present invention provides a method for automatic extraction of text classification and text keyword. The method includes inputting a plurality of texts; preprocessing the plurality of texts to generate a plurality of preprocessed texts; and classifying the plurality of preprocessed texts into a plurality of classifications with different topics according to a text topic analysis model, and outputting the plurality of preprocessed texts and a plurality of corresponding classification indexes; extracting a plurality of sets of corresponding keywords of the plurality of classifications according to the plurality of pre-processed texts, the plurality of classification indexes, an open source word vector pre-training set and a pairing algorithm; and establishing a word vector text classifier according to the plurality of classifications and the plurality of sets of keywords.

Description

Short text automatic extraction classification and keyword method and device adopting the method

The present invention refers to a method for automatic extraction and classification of short texts and keywords, in particular to a method for automatic extraction and classification of short texts and keywords that can perform word vector classification training through keywords in a small sample size.

The training data sets of the current short text classification models are mostly news databases, and most of them are used in news gathering or Internet public opinion analysis. Its characteristics include sufficient amount of text, consistent article topics, clear syntactic structure, etc., so it is suitable for use. The training word vector and the classifier method of the known technology are used for classification.

However, it is not easy to perform general clustering and classification of natural language processing ( Natural Language Processing (NLP) model, conventional technology training word vectors and classifiers for classification are only suitable for large sample numbers. In addition, the classifiers trained by conventional technology by machine learning are more time-consuming and resource-intensive, and after establishment It cannot be fine-tuned according to demand. If you need to add keywords such as product names or function names in response to business needs, you can only retrain and lack flexibility.

In view of this, it is necessary to improve the conventional technology.

Therefore, the main purpose of the method and device for automatic extraction and classification of essays and keywords of the present invention is to select the best candidate keywords as keywords corresponding to a specific category by using a suitable algorithm, so that the set of keywords are from words The angle of vector analysis is sufficient to represent the key points of most essays, so that word vectors can be classified by keywords in a small sample size.

Another main purpose of the method and device for automatic extraction and classification of short essays and keywords of the present invention is to solve the problem of poor analysis efficiency in small sample numbers by using word vector training in the prior art by introducing open source word vector pre-training word sets. .

Another main purpose of the method and device for automatic extraction and classification of short texts and keywords of the present invention is that the keyword extraction module can extract and update the keywords of the merged category again, and is used to correct the representativeness of the keywords in the conventional technology category. The problem of clarification.

Another main purpose of the method and device for automatic extraction and classification of short texts and keywords of the present invention is to achieve the purpose of saving system resources without training word vectors and training the classifier through machine learning when performing classification compared with conventional techniques. .

Another main purpose of the method and device for automatic extraction and classification of short essays and keywords of the present invention is to extract the best complex array of keywords by using the open source word vector pre-training set and matching with a moderate algorithm, and combine the word vector similarity classification model Groups are classified, so that when keywords need to be adjusted in the future (for example, new product names or function names should be added in response to business needs), there is no need to retrain in order to achieve more agile and flexible goals.

The present invention discloses a method for automatic extraction and classification of short essays and keywords, which includes inputting plural essays; preprocessing the plural essays to generate plural preprocessing essays; and preprocessing the plural essays according to a text topic analysis model The essays are divided into multiple categories of different topics, and output the multiple preprocessed essays and corresponding multiple category labels; according to the multiple preprocessed essays, the multiple category labels, an open source word vector pre-training set and a pair Moderate algorithm, extract the plural classes Other corresponding complex group keywords; and according to the plural categories and the complex group keywords, a word vector essay classifier is established.

10: Short text automatic extraction classification and keyword device

102: Pre-processing module

104: Short essay topic classification module

106: Keyword Extraction Module

108: Keyword merge module

110: Similarity classification module

12: Input device

14: output device

T1~Tm: short text

T1’~Tm’: Preprocessed short text

C1~Cn: category

TI1~TIm: category label

KW1~KWn: Keywords

60: Process

600~612: steps

Figure 1 is a schematic diagram of a short text automatic extraction and classification and keyword device according to an embodiment of the present invention.

Figures 2 to 5 are schematic diagrams of the operation of the short text automatic extraction and classification and keyword device shown in Figure 1 of the embodiment of the present invention.

Figure 6 is a schematic diagram of a process of automatic extraction and classification of short texts and keywords according to an embodiment of the present invention.

Please refer to FIG. 1. FIG. 1 is a schematic diagram of a short text automatic extraction and classification and keyword device 10 according to an embodiment of the present invention. As shown in Figure 1, the automatic short text extraction and classification and keyword device 10 includes a pre-processing module 102, a short text topic classification module 104, a keyword extraction module 106, a keyword merging module 108, and a similar Degree classification module 110. To put it simply, an input device 12 inputs the short text T1~Tm to the pre-processing module 102, and the pre-processing module 102 performs word segmentation, stop words, capitalization, part-of-speech restoration, and verb form processing on the short text T1~Tm. , To generate pre-processed short texts T1'~Tm'; the short text topic classification module 104 can divide the pre-processed short texts T1'~Tm' into categories C1~Cn of different topics according to a text topic analysis model, and output the pre-processed short texts T1'~Tm' and corresponding category labels TI1~Tim; the keyword extraction module 106 is based on the pre-processed short text T1'~Tm', category labels TI1~TIm, an open source word vector pre-training set and a matching appropriate algorithm, Extract the corresponding keywords KW1~KWn of the categories C1~Cn; the keyword merging module 108 determines the vector distance between the keywords KW1~KWn, and the vector distance of the corresponding keywords in the categories C1~Cn is less than a threshold The categories are merged; the keyword extraction module 106 updates the re-extracted keywords from the merged categories in the categories C1~Cn to modify the representativeness of the merged category keywords The problem of insignificance. Finally, the similarity classification module 110 can calculate the distance based on the keywords KW1~KWn and their word vectors, build a word vector essay classifier and provide it to an output device 14 to improve the keyword and classification recognition accuracy of subsequent short essays with a small sample number The accuracy rate (such as calculating which group of keywords in the subsequent specific essay and keywords KW1~KWn have the highest similarity, then it is determined that the specific essay belongs to the corresponding category). In this way, the present invention can classify word vectors through keywords in a small sample size and is more flexible, and does not need to train word vectors and classifiers, which saves resources.

Specifically, please refer to FIG. 2 to FIG. 5. FIG. 2 to FIG. 5 are schematic diagrams of the operation of the short text automatic extraction and classification and keyword device 10 according to the embodiment of the present invention. As shown in Figure 2, the short texts T1~T6 are preprocessed by the preprocessing module 102 for word segmentation, stop word processing, capitalization processing, part-of-speech restoration processing, and verb type processing. The preprocessed short texts T1'~T6' suitable for processing are provided to the preprocessed short text topic classification module 104. The preprocessing is well known to those skilled in the art and will not be repeated here. Then, as shown in Figure 3, the short text topic classification module 104 divides the preprocessed short text T1'~T6' into categories C1~C4, and determines the corresponding category labels TI1~TI6 to be categories C1, C2, C2, C3, respectively , C1, C4, and short texts can be classified into the main body and short texts with similar topics are placed in the same category, and then the keyword extraction module 106 extracts the corresponding keywords KW1~KW4 of the categories C1~C4 to represent each category. Then, as shown in Figure 4, the keyword merging module 108 determines that the vector distance of keywords KW2 and KW3 is less than the threshold and merges category C3 into category C2 to merge the close categories, and then the keyword extraction module 106 The keywords are extracted for the merged category C2, that is, the keyword KW2 is updated as shown in Figure 5 (the original category label TI4 is also updated to category C2).

In detail, the text topic analysis model of the short text topic classification module 104 may include a Gibbs Sampling Dirichlet Multinomial Mixture Model (GSDMM) and a Dirichlet Multinomial Mixture Model (Dirichlet). Mixture Model, DMM), a GPU-improved Dirichlet Multinomial Mixture (GPU-DMM), a Latent Dirichlet Allocation (LDA), a topic representative word discovery At least one of (Topic Representative Term Discovery, TRTD), a Biterm Topic Model (BTM), a Latent Semantic Indexing (LSI), and a Latent Semantic Analysis (LSA) . For example, GSDMM randomly groups the preprocessed essays T1'~Tm' into groups, and then moves each essay until the nature of the essays in the categories C1~Cn of different topics are similar. GSDMM and other methods are well known in the art , I won’t repeat it here for brevity.

On the other hand, the open source word vector pre-training set in the keyword extraction module 106 may include pre-training sets wiki-news-300d-1M.vec, wiki-news-300d-1M-subword, crawl-300d-2M.vec, and At least one of crawl-300d-2M-subword, for small sample size, word vector training cannot have good results, so by introducing open source word vector pre-training word set to improve analysis efficiency, open source word vector pre-training The collection is also well-known in the art, so I won't repeat it here for the sake of brevity.

In this case, the keyword extraction module 106 executes an appropriate algorithm based on the sum of multiple similarities of multiple candidate keywords corresponding to a specific category in the categories C1~Cn or at least one of the multiple number of passages passing the threshold. , Determine the keyword corresponding to the specific category. In detail, the keyword extraction module 106 calculates the similarity between each group of candidate keywords and all pre-processed essays in a specific category according to the open source word vector pre-training set, and then adds up the similarity or sets a threshold (such as 0.3) Then calculate the proportion of passages that pass the threshold (for example, if the number of passages with a similarity greater than 0.3 is 70, and the number of all passages in a particular category is 100, the proportion of passages that pass the threshold is 0.7), and then add the highest similarity in the particular category to the sum or one One of the best candidate keywords with the highest proportion of short passages through the threshold is used as the keyword corresponding to a specific category. Among them, the calculation of the similarity between keywords and essays is to map them to the vector space, and then calculate different vectors The degree of similarity is calculated by the size of the difference. For example, ["hello","world"]/["hi","world"] can calculate the similarity as 0.76. Similarity calculation is also well-known in the art, so I won’t Repeat it again for brevity.

It is worth noting that the above-mentioned embodiment is mainly to perform an appropriate matching algorithm to extract the candidate keywords with the highest similarity summation or the highest proportion of passages in the specific category as the keywords corresponding to the specific category. The corresponding keywords KW1~KWn of the categories C1~Cn, and then a word vector essay classifier is established to classify the subsequent essays. Those with ordinary knowledge in the field can modify or change according to it, but it is not limited to this. For example, in the above embodiment, after extracting the corresponding keywords KW1~KWn of the categories C1~Cn for the first time, the categories whose vector distances of the corresponding keywords in the categories C1~Cn are less than a threshold are merged to modify The problem of insufficient representativeness, but in other embodiments, it may not be combined. In addition, in the above-mentioned embodiment, a word vector essay classifier is established based on the corresponding keywords KW1~KWn of the categories C1~Cn to classify subsequent essays. Therefore, in practical applications, the similarity classification module 110 can also adjust the categories C1~ Cn and keywords KW1~KWn create word vector short text classifiers (such as increasing or decreasing categories), which can have better flexibility.

On the other hand, in the above-mentioned embodiment, the best candidate keyword with the highest similarity summation or the highest proportion of passages among the candidate keywords in the complex array is used as the keyword corresponding to a specific category. In one embodiment, The complex array of candidate keywords can be generated according to a genetic algorithm, and the genetic algorithm is executed with a specified number of cycles, or the sum of the highest similarity of one of the best candidate keywords or the proportion of short passages with the highest passing threshold greater than one When matching the appropriate threshold, the matching algorithm uses the best candidate keyword as the keyword corresponding to the specific category.

For example, a genetic algorithm can encode a plurality of words in a specific category to generate a complex set of candidate keywords. For example, select 3 words from 5 words as a set of candidate keywords, and then generate 10-20 sets of parent candidate keywords, and then a moderate algorithm is used to calculate the total similarity of 10-20 sets of candidate keywords or the proportion of passages that pass the threshold, and keep the total similarity of the candidate keywords of the parent or the proportion of passages that pass the threshold greater than Retain candidate keywords with a specific value (such as the proportion of passages greater than 0.8), and then genetic algorithm will retain candidate keywords for mating. For example, a binary code can be used to indicate the presence or absence of 5 words in the following table. One word in the two groups of reserved candidate keywords originally has one group and one group does not have it. After mating, it becomes both groups (the first word), Neither of the two groups (the third word), or the original one has become absent and the original one has not become present (the fourth word):

In addition, the genetic algorithm can also perform mutations, with a certain probability (set to a low probability) to randomly change the presence or absence of specific words in the reserved candidate keywords. For example, the reserved candidate keyword does not originally have a third word and is changed to have a third word. word. In this case, the genetic algorithm can mate or mutate the reserved candidate keywords to form a loop to generate new candidate keywords, and then the fitness algorithm calculates the sum of the similarities of the new candidate keywords or passes the threshold Proportion of short essays. In this way, when the genetic algorithm executes a specified number of cycles, or the sum of the highest similarity of one of the best candidate keywords, or the proportion of short passages with the highest passing threshold is greater than a suitable threshold, the appropriate algorithm is used to The best candidate keyword is used as the keyword corresponding to a specific category, and the loop of genetic algorithm is ended.

Therefore, the operation of the short text automatic extraction classification and keyword device 10 can be summarized as a short text automatic extraction classification and keyword process 60, as shown in Figure 6, which includes the following steps:

Step 600: Start.

Step 602: Input a plurality of short essays.

Step 604: Perform a preprocessing on the plurality of short texts to generate a plurality of preprocessed short texts.

Step 606: According to a text topic analysis model, divide the plurality of preprocessed essays into a plurality of categories of different topics, and output the plurality of preprocessed essays and corresponding plural category labels

Step 608: According to the plurality of pre-processed essays, the plurality of category labels, an open source word vector pre-training set and a suitable algorithm, extract the corresponding complex array of keywords of the plurality of categories

Step 610: Create a word vector essay classifier based on the plural categories and the plural keywords

Step 612: End.

For the detailed operation of the automatic short-text extraction classification and keyword process 60, please refer to the related content of the short-text automatic extraction classification and keyword device 10, which will not be repeated here for the sake of brevity.

In addition, the short text automatic extraction and classification and keyword device 10 may include a processing device and a storage unit. The processing device can be a microprocessor or an application-specific integrated circuit (ASIC). The storage unit can be any data storage device, used to store a program code, and read and execute the program code through the processing device to execute the pre-processing module 102, the essay topic classification module 104, the keyword extraction module 106, The functions of the keyword merging module 108 and the similarity classification module 110 further complete the automatic extraction and classification of short texts and the steps of the keyword process 60. The storage unit can be a subscriber identity module (SIM), read-only memory (read-only memory, ROM), random-access memory (RAM), optical disc read-only memory ( CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. are not limited thereto.

In summary, the present invention can execute a suitable algorithm to use the open source word vector pre-training set, extract the corresponding keywords of each category in a small sample size, and then establish the word vector essay classification based on the keywords It is more flexible and does not need to train word vectors and classifiers to save resources.

The foregoing descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made in accordance with the scope of the patent application of the present invention shall fall within the scope of the present invention.

60: Process

600~612: steps

Claims (8)

A method for automatic extraction and classification of essays and keywords includes: inputting plural essays; preprocessing the plural essays to generate plural preprocessing essays; according to a text topic analysis model, the plural preprocessing essays are divided into Multiple categories of different topics, and output multiple category labels corresponding to the multiple preprocessed essays, and each category has multiple corresponding candidate keywords; according to the multiple preprocessed essays, the multiple Preprocess the plural category labels corresponding to the essay, an open source word vector pre-training set, the plural categories and the plural corresponding candidate keywords, and an appropriate algorithm to pluralize the plural categories in each category At least one of the sum of similarities or the proportion of multiple passages through the threshold determines the keywords corresponding to each category of the plurality of categories; then the appropriate matching algorithm is performed, and the corresponding categories of the plurality of categories are determined For the vector distance between keywords, merge the categories whose vector distance is less than a threshold, and extract the corresponding keywords from the merged categories again to update; and according to the plural categories and the corresponding categories of the plural categories Establish a word vector essay classifier to achieve the purpose of automatic extraction and classification of essays. The method for automatic extraction and classification of short texts and keywords according to claim 1, wherein the preprocessing includes at least one of word segmentation processing, stop word processing, capitalization processing, part-of-speech reduction processing, and verb type processing . According to claim 1, the method for automatic extraction and classification of short texts and keywords, wherein the text topic analysis model includes a Gibbs sampling algorithm of Dirichlet polynomial mixed model (Gibbs sampling algorithm) Sampling Dirichlet Multinomial Mixture Model (GSDMM), a Dirichlet Mixture Model (DMM), an application of GPU to improve the Dirichlet Multinomial Mixture (Generalized Polya Urn-Dirichlet Multinomial Mixture, GPU-DMM), a hidden Includes Dirichlet Allocation (LDA), Topic Representative Term Discovery (TRTD), Biterm Topic Model (BTM), and Latent Semantic Indexing , LSI) and at least one of a Latent Semantic Analysis (LSA); the open source word vector pre-training set includes pre-training sets wiki-news-300d-1M.vec, wiki-news-300d-1M-subword , At least one of crawl-300d-2M.vec and crawl-300d-2M-subword. The method for automatic extraction and classification of short texts and keywords as described in claim 1, which further includes: the optimal matching algorithm in the specific category has the highest degree of similarity or the highest proportion of short texts that have the highest passing threshold. The keyword is the keyword corresponding to the specific category. The method for automatic extraction and classification of short texts and keywords as described in claim 1, which further includes: generating the complex array of candidate keywords according to a genetic algorithm; and executing a specified number of cycles on the genetic algorithm, or a maximum When the sum of the highest similarity of one of the best candidate keywords or the proportion of a short passage with the highest passing threshold is greater than a suitable threshold, the matching algorithm uses the best candidate keyword as the keyword corresponding to the specific category. The method for automatic extraction and keyword extraction of short texts as described in claim 5, which is based on The step of generating the complex set of candidate keywords by the genetic algorithm includes: encoding a plurality of words in the specific category to generate a complex set of matrix candidate keywords in the complex set of candidate keywords. As described in claim 5, the method for automatic extraction and classification of short texts and keywords, which further includes: a plurality of similarities or a plurality of retained candidates whose ratios of passage threshold passages are greater than a specific value in the complex group of parent candidate keywords The keywords are mated or mutated to generate a new candidate keyword in the complex set of candidate keywords. A system for automatic extraction and classification of short texts and keywords includes: a processor for executing a program; and a storage unit coupled to the processor for storing the program; wherein the program is used to instruct the processor Carry out the automatic extraction and classification of short texts and keywords as described in Claims 1 to 7.

Images