CN100495408C - Text clustering element study method and device - Google Patents

Abstract

The invention discloses a text clustering meta-learning method, which comprises: performing soft clustering or soft classification processing on a text set by a text analysis method to obtain at least two clustering or classification results; said clustering or classification The results are expressed as processing result matrices, and the processing result matrices are spliced into a text vector matrix; meta-learning is performed on the text vector matrix to obtain a final clustering result. The invention also discloses a text clustering meta-learning device, which includes: a text analysis module, a matrix synthesis module and a meta-learning module. Since the text clustering meta-learning method and device provided by the present invention comprehensively consider the results of multiple clustering and classification, more accurate text clustering results can be obtained, and the deviation caused by a single clustering can be effectively reduced , which improves the accuracy and stability of the clustering results.

Description

A text clustering meta-learning method and device

technical field

The invention relates to a text clustering method, in particular to a text clustering meta-learning method and device.

Background technique

Text clustering method is a clustering analysis method, which is an application of clustering analysis technology in the field of text processing. The text clustering method can automatically discover several clusters in a text set, and divide all the texts in the text set into multiple clusters, so that the contents of the texts belonging to the same cluster have a high degree of similarity, while the texts belonging to different clusters have a high degree of similarity. The content of the texts of the clusters is quite different. Text clustering methods can be applied in many aspects, for example: the topic detection and tracking (TDT, Topic detection and tracking) project of the U.S. Department of Defense is trying to automatically discover hot topics in a news text flow through text clustering methods; in addition, Text clustering methods can be used to cluster the result pages returned by search engines, so that users can obtain more structured and understandable search results; ) such as the classification system of network texts, etc.

Current text clustering methods are usually based on vector space models. In the vector space model, each text is represented as a text vector in a multi-dimensional Euclidean space, each dimension in the space corresponds to a feature word, and the value of the text vector on each dimension is generally Defined as the number of occurrences of the feature word corresponding to the dimension in the text corresponding to the text vector. For any text set, a vector space model can be used to generate a feature word-based text vector matrix V(n*k), where n is the number of texts in the text set, k is the dimension of each text vector, and each row of the matrix Both correspond to a text vector. After obtaining the vector matrix of the text set, various classical clustering algorithms such as K-means (K-means) algorithm, hierarchical agglomerative clustering (HAC) algorithm, etc. can be used to cluster the vector matrix of the text set, thereby generating Text clustering results.

The existing clustering algorithms can be roughly divided into hierarchical clustering, partition clustering, density-based clustering, grid-based clustering and model-based clustering algorithms. Among them, partition clustering algorithm, especially K-means algorithm has been one of the most widely used clustering algorithms. In the K-means algorithm, the categories are divided by comparing the distance between the data samples and the center points of each class, and the data set is divided into K parts after repeated iterations. Among them, K is the number of desired clusters, which needs to be specified in advance. Specifically, the above-mentioned K-means algorithm includes three steps: the first step is to determine K initial class center points in the data set, representing K clusters respectively; the second step is to assign each data sample to the nearest The cluster represented by the center point of the class; the third step is to calculate the center point of each cluster currently formed, replace the original class center point, and return to the second step; execute the second and third steps in a loop until the result converges , that is, until the clusters to which all data samples belong no longer change, so as to achieve the purpose of clustering.

Apart from text clustering methods, text classification is another approach to text analysis. Different from the text clustering method, the text classification method requires manual training, that is, it needs to manually specify the categories, and provide certain training data for each category, and then judge the detected text according to the difference between the detected text and the training data. The category the text belongs to. Commonly used text classification methods include K-Nearest Neighbor (KNN, K-Nearest Neighbor) algorithm, etc.

At present, the general text clustering and classification methods divide the text in the text set into a specific cluster or category, while the soft clustering and soft classification methods are an extension of the above text clustering and text classification methods. These two methods do not divide the text in the text set into one cluster or category, but divide the text in the text set into multiple clusters and categories with different probabilities. Generally speaking, the classification results obtained by soft clustering and soft classification methods are more scientific.

One of the main problems of the current text clustering methods is the poor stability of text clustering methods, that is, for different text sets, the results of using a text clustering method to process them may be good or bad; For a text set, the classification results obtained by using text clustering method A are better than those obtained by using text clustering method B, and for another text set, the classification results obtained by using text clustering method A are better than those obtained by using text clustering method B bad question.

To solve this problem, some researchers proposed meta-learning methods for text clustering. This method improves the stability of the text clustering method by synthesizing the clustering results generated by various text clustering algorithms. The meta-learning mentioned above is the re-learning of the learning results, that is, the re-learning of the clustering results generated by the above-mentioned multiple text clustering algorithms. The above text clustering meta-learning method can be implemented through different strategies, and the most intuitive method is the voting (consensus) method, which is very easy to implement. In the voting method, a variety of text clustering algorithms are used, and for which cluster an object should belong to, each different text clustering method will give its own evaluation, and then take a voting method, The clustering result with the most votes will be the cluster to which this object should finally belong. In addition, Alexander Strehl et al. also proposed three voting-based meta-learning methods: the first method is called Cluster-based Similarity Partitioning Algorithm (CSPA, Cluster-based Similarity Partitioning Algorithm), which analyzes the same cluster The relationship between two objects in the network is used to evaluate the similarity between each two objects, and the clusters are re-divided according to the obtained similarity evaluation, so as to obtain better clustering results; the second method is called hypergraph Partitioning Algorithm (HGPA, HyperGraph Partitioning Algorithm), which regards the process of text clustering meta-learning as the process of hypergraph segmentation, and the edges of the hypergraph are the generated clusters; the third method is to use text clustering meta-learning The process is regarded as a process of cluster correspondence, that is, to correspond to similar clusters in two groups of different clustering results, and then synthesize the final cluster. Before the method is implemented, the problem of cluster correspondence must be solved first, but since there is no fixed category system in the text clustering method, it is very difficult to match the clusters in two groups of different clustering results one by one. Corresponding to the third method above, another looser method is to find clusters that appear in both groups of clustering results and ignore other clusters. But even so, when the number of text clustering is large, such repeated clusters will become very few; considering the above situation, the standard can be lowered, that is, as long as the clusters in different clustering results are similar. They can be connected, but in this way, how to determine the similarity threshold and other issues will bring unstable factors to the meta-learning itself, making it difficult to achieve the goal of using meta-learning technology to improve the stability of the algorithm.

In summary, in the text clustering meta-learning method in the prior art, although the clustering results generated by multiple text clustering algorithms are synthesized, the meta-learning method for the clustering results is The way of voting makes it impossible to accurately divide clusters based on similarity, resulting in low accuracy and stability of clustering results.

Contents of the invention

In view of this, the present invention provides a text clustering meta-learning method and device to improve the accuracy and stability of clustering results.

In order to achieve the above object, technical solution of the present invention is achieved in that way:

Embodiments of the present invention provide a text clustering meta-learning method, the method comprising the following steps:

A. Perform soft clustering or soft classification processing on the text set with a text analysis method, and obtain at least two clustering or classification results;

B. Representing the clustering or classification results as processing result matrices, and splicing the processing result matrices into a text vector matrix;

C. Performing meta-learning on the text vector matrix to obtain a final clustering result.

The embodiment of the present invention also provides a text clustering meta-learning device, which includes: a text analysis module, a matrix synthesis module and a meta-learning module;

The text analysis module is used to perform soft clustering or soft classification processing on the text set, and send the obtained clustering or classification results to the matrix synthesis module;

The matrix synthesis module is used to convert the received clustering or classification results into a matrix, and splice the converted matrix into a text vector matrix, and send the text vector matrix to the meta-learning module;

The meta-learning module is used to perform meta-learning on the received text vector matrix, and output the final clustering result.

It can be seen from the above-mentioned technical solution that the text clustering meta-learning method and device in the present invention synthesize a text vector matrix based on multiple text analysis results, and carry out meta-learning on the above text vector matrix, comprehensively considering multiple soft Clustering or soft classification analysis results, so more accurate text clustering results can be obtained, effectively reducing the deviation caused by a single clustering, and improving the accuracy and stability of the clustering results.

Description of drawings

FIG. 1 is a schematic diagram of a text clustering meta-learning method in an embodiment of the present invention.

Fig. 2 is a flow chart of the text clustering meta-learning method in the embodiment of the present invention.

Fig. 3 is a structural diagram of a text clustering meta-learning device in an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention more clearly, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

The present invention provides a meta-learning method for text clustering, which uses soft clustering and soft classification methods to process text sets multiple times to obtain processing results; then constructs text through multiple soft clustering and soft classification processing results Vector matrix, and finally use the clustering algorithm to perform meta-learning processing on the text vector matrix to obtain the final clustering result.

FIG. 1 is a schematic diagram of a text clustering meta-learning method in an embodiment of the present invention. As shown in Figure 1, in the text clustering meta-learning method in the embodiment of the present invention, firstly, a plurality of text soft clustering methods and text soft classification methods are used to process the text set, thereby obtaining multiple clustering results; Then, construct a text vector matrix based on the multiple clustering results obtained by the above-mentioned various text soft clustering methods and text soft classification; then, normalize the text vector matrix, and finally based on the element of the hard clustering algorithm The learning method performs meta-learning on the above text vector matrix, so as to obtain the final text clustering result.

Fig. 2 is a flow chart of the text clustering meta-learning method in the embodiment of the present invention. As shown in Figure 2, the text clustering meta-learning method in the embodiment of the present invention includes the following steps:

Step 201, use the text analysis method to process the text set m times.

The text analysis methods mentioned above include soft clustering methods and soft classification methods. The soft clustering method and the soft classification method are two different data mining methods, each having different application fields. The results obtained by using soft clustering methods and soft classification methods are several classes, and the probability that each text belongs to a specific class. For soft clustering methods, a class represents the clusters found by the soft clustering method, and for soft classification, a class represents a specific category. In the embodiment of the present invention, the above two types of methods are organically combined, so that a more accurate and stable clustering result can be obtained.

The above-mentioned processing of the text set by the text analysis method for m times may be performed by using one or more soft clustering methods to process the text set m times; The text set is processed m times; it is also possible to use one or more soft clustering methods and soft classification methods to process the text set m times. For the convenience of description, the following will use the soft clustering method and the soft classification method to process the text set m times as an example for detailed description.

In the above steps, some public soft clustering and soft classification methods can be used to process the text set, and the soft clustering and soft classification methods include algorithms such as soft K-means and soft KNN. In practical applications, n algorithms can be selected respectively from soft clustering and soft classification methods, and then each of the selected algorithms can be used to process the text set to be processed m times. Both n and m mentioned above are integers and can be set in advance, and said n and m satisfy: n≥1, m≥1. The reason why the same method is used for m processing is that even if the same method is used to process the same text set, the processing results obtained may be different. For example, for the soft K-means algorithm, even for the same text set A text set is processed with the same soft K-means algorithm every time, and the results obtained are different. In order to improve the accuracy of the obtained results, it is necessary to use the same method for m times of processing, so as to obtain m processing results.

In addition, before using the above soft clustering method and soft classification method, the text set needs to be preprocessed, and the text in the text set is converted into a text vector based on feature words, so that the soft clustering method and soft classification method can be used Process the text set. The preprocessing mainly includes three steps of word segmentation, feature selection, and text vectorization. Through these three steps, each text in the text set can be converted into a text vector in a multidimensional Euclidean space. The specific processing steps are as follows:

1) Word segmentation processing is first required.

Word segmentation is to divide the text in the text set into individual words, and count the number of times each word appears in the text, and use it as the basis for text vectorization.

2) Then, perform feature selection processing.

Since there are many words in a text set, and many words have no positive effect on the clustering distinction, but will affect the clustering effect, it is necessary to select the features of the words in the text set. The purpose of feature selection is to retain those words that are conducive to clustering and remove most of the words that are not conducive to clustering, so as to reduce the dimension of the text vector without affecting the clustering effect. Various feature selection methods can be used in the present invention, including methods such as removing blocked words, removing words that appear in too many or too few texts, and removing words that appear too few times in a single text. By using the above methods in combination, the number of feature words in the text set can be reduced from tens of thousands to about one thousand with little impact on the clustering effect.

3) Finally, perform text vectorization.

The purpose of text vectorization is to convert the text in the text set into a text vector in a multi-dimensional Euclidean space. Each dimension in the multi-dimensional Euclidean space corresponds to a feature word, and a text vector The value of each dimension in the multidimensional Euclidean space is the weight of the feature word corresponding to the dimension in the text corresponding to the text vector. Commonly used text vector weight calculation methods include Term Frequency (TF), Term Frequency Inverse Document Frequency (TFIDF), Term Frequency Control (tfc), and Length Term Frequency (1tc). Compared with other weight calculation methods, the 1tc method reduces the impact of different text length differences on text vectorization and clustering, so the 1tc method is used in the present invention. The formula for calculating the weight using the 1tc method is:

$\mathrm{<span\; class="notranslate">\; ltc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; ik</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1.0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; N</span>}}{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; )</span>}{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; jk</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1.0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; N</span>}}{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Among them, N is the number of texts in the text set, M is the number of selected feature words, f _ik is the number of times the i-th feature word appears in the k-th text, and n _i is the number of texts containing the i-th feature word quantity.

After the above-mentioned preprocessing is performed on the text set, the above-mentioned soft clustering method and soft classification method are used to process the text set multiple times to obtain a processing result.

The above is the method of using the soft clustering method and the soft classification method to process the text set m times at the same time, and the steps of using the soft clustering method or soft classification method alone to process the text set are the same as the above-mentioned using the soft clustering method at the same time The steps of the soft classification method are similar, so they will not be repeated here.

Step 202, constructing a text vector matrix based on the m times of processing results.

In this step, the construction method of the above text vector matrix is as follows:

是该文本集第i次软聚类或者软分类结果中的第j个簇或者类别。因此，根据上述的处理结果，可将第i次软聚类和软分类结果表示成一个矩阵：First, each of the above-mentioned processing results obtained by using soft clustering or soft classification methods is represented as a matrix. The rows of the matrix represent the clusters generated by the soft clustering method or the categories in the soft classification method, and the columns represent the text in the text set. , each object is the probability that a certain text belongs to a cluster or category. For example, for a text set Corpus containing n texts: (d ₁ , d ₂ , d ₃ , ..., d _n ), assuming that the number of times of soft clustering and soft classification is m, we can get The m division results of the text set can be expressed as Partition: (P ₁ , P ₂ , P ₃ , ..., P _m ), where P _i is the processing result of the i-th soft clustering or soft classification, It can be expressed as $P_i=\left\{\begin{array}{cccc}{C}_i^1& C_i^2& ...& C_i^{k_i}\end{array}\right\},$ Where k _i is the number of clusters or categories in the i-th soft clustering or soft classification result, which means that in this processing result P _i , the text set is divided into k _i clusters or categories,

is the jth cluster or category in the ith soft clustering or soft classification result of the text set. Therefore, according to the above processing results, the i-th soft clustering and soft classification results can be expressed as a matrix:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}\mathrm{<span\; class="notranslate">\; i</span>}}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; i</span>}}\\ <span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span>\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; ni</span>}}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Among them, v _li is the processing result of the i-th soft clustering or soft classification of the l-th text in the above-mentioned text set, which can be expressed as $v_{\mathrm{li}}=\left\{\begin{array}{cccc}{\mathrm{Prob}}_{\mathrm{li}}^1& {\mathrm{Prob}}_{\mathrm{li}}^2& ...& {\mathrm{Prob}}_{\mathrm{li}}^{k_i}\end{array}\right\},$ in is the probability that the l-th text belongs to the j-th cluster or category in the i-th soft clustering or soft classification results.

Through the above method, we can obtain all soft classification and soft clustering result matrices. Then, all the soft clustering and soft classification result matrices are concatenated into a block matrix M, as shown below, M is the text vector matrix.

M＝[M ₁ M ₂ ... M _m ] (3)

Among them, M _i is the processing result matrix obtained by using soft clustering or soft classification methods, and m is the number of times of soft clustering and soft classification.

Since the accuracy of different soft clustering and soft classification methods is different, the user can pre-set different weights k for the processing results obtained by using different soft clustering methods and soft classification methods according to the actual situation, and then After each obtained processing result, that is, the matrix is multiplied by the corresponding weight k, all the matrices are spliced into a text vector matrix. As follows:

M＝[k ₁ M ₁ k ₂ M ₂ ... k _m M _n ] (4)

Among them, _kj is the preset weight for different soft clustering methods and soft classification methods, m is the number of different soft clustering methods or soft classification methods used.

Step 203, performing meta-learning to obtain the final clustering result.

In this step, the embodiment of the present invention first performs normalization processing on the text vector matrix, and then uses a clustering method to perform meta-learning on the above text vector matrix to obtain the final text clustering result. The meta-learning mentioned above is the re-learning of the learning results of soft clustering and soft classification in the previous step. Some common clustering algorithms in this field can be used, such as K-means algorithm, hierarchical agglomerative clustering algorithm, density clustering algorithm, etc., to perform normalization processing and clustering element learning on the above-mentioned text vector matrix. The final clustering result is: P={C ₁ , C ₂ , . . . , C _k }, where K is the number of clusters generated, and C _i is the i-th cluster.

In summary, the present invention provides a meta-learning method for text clustering. This method first vectorizes the text set, uses soft clustering and soft classification algorithms to process the text set several times, and then according to the obtained soft clustering Construct a text vector matrix with the results of soft classification, each object in the vector matrix is the probability that a certain text belongs to a cluster generated by a certain soft clustering method or a classification in a certain soft classification method, and finally use the general clustering method The class algorithm performs meta-learning on the text vector matrix to obtain the final text clustering result. Because this method comprehensively considers the results of multiple clustering and classification, it can obtain more accurate text clustering results, effectively reduce the deviation caused by single clustering, and improve the accuracy and stability of clustering results. sex.

In addition, the embodiment of the present invention also provides a text clustering meta-learning device. FIG. 3 is a structural diagram of a text clustering meta-learning device in an embodiment of the present invention. As shown in FIG. 3 , the device includes: a preprocessing module 300 , a text analysis module 301 , a matrix synthesis module 302 and a meta-learning module 303 . Wherein, the preprocessing module 300 is used to carry out text vectorization to the text in the text set, and the text set after the text vectorization is sent to the text analysis module 301; The text analysis module 301 performs text analysis on the text set to be processed, and The obtained text analysis results are sent to the matrix synthesis module 302 . In the text analysis module 301, the text analysis methods used include soft clustering methods and soft classification methods; the matrix synthesis module 302 synthesizes a text vector matrix according to the received text analysis results, and sends the synthesized text vector matrix to the A meta-learning module 303; the meta-learning module 303 is configured to perform meta-learning on the received text vector matrix, and output a final clustering result.

In the above text clustering meta-learning device, the preprocessing module 300 is not an essential module, so it is represented by a dotted line. As shown in FIG. 3 , the preprocessing module 300 further includes: a word segmentation unit 304 , a feature selection unit 305 and a text vectorization unit 306 . The word segmentation unit 304 is used to divide the text in the text collection into individual words, and counts the number of times each word occurs in the text collection, and sends the division results and statistical results to the feature selection unit 305; the feature selection unit 305 receives the The division result and statistical result, select feature word from text collection, and send the selected feature word to text vectorization unit 306; Text vectorization unit 306 converts the text in text collection into text vector according to the received feature word, and Send the text set after text vectorization to the text analysis module 301 .

As shown in FIG. 3 , the matrix synthesis module 302 further includes: a matrix unit 307 and a synthesis unit 308 . The matrixing unit 307 converts the received text analysis results into a matrix, and sends the converted matrix to the synthesis unit 308; the synthesis unit 308 synthesizes all the converted matrices received into a text vector matrix, and synthesizes the synthesized text The vector matrix is sent to the meta-learning module 303.

As shown in FIG. 3 , the meta-learning module 303 further includes: a normalization unit 309 and a learning unit 310 . The normalization unit 309 is used to normalize the received text vector matrix, and sends the normalized text vector matrix to the learning unit 310; the learning unit 310 is used to normalize the received normalized The text vector matrix is meta-learned to output the final clustering result.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (9)

1, a kind of text clustering element study method is characterized in that, this method may further comprise the steps:

A, with the text analyzing method text set is carried out soft cluster or soft classification and handle, obtain at least two clusters or classification results;

B, described cluster or classification results are expressed as the result matrix, described result matrix is spliced into the text vector matrix;

C, described text vector matrix is carried out unit study, obtain final cluster result.

2, method according to claim 1 is characterized in that: the text analyzing method in the steps A is soft clustering method and/or soft sorting technique.

3, method according to claim 1, it is characterized in that, described result matrix be spliced into the text vector matrix comprise among the described step B: after described result matrix is multiplied by the weights that set in advance respectively, more all result matrixes are spliced into the text vector matrix.

4, method according to claim 1 is characterized in that, also comprises before the steps A: described text set is carried out pre-service; Described pre-service comprises: participle, Feature Selection and text vectorization.

5, a kind of text clustering element learning device is characterized in that, this device comprises: text analysis model, matrix synthesis module and first study module;

Described text analysis model is used for text set is carried out soft cluster or soft classification processing, and cluster or the classification results that obtains sent to described matrix synthesis module;

Described matrix synthesis module is used for received cluster or classification results are changed into matrix, and the matrix after will transforming is spliced into the text vector matrix, and described text vector matrix is sent to described first study module;

Described first study module is used for the text vector matrix that receives is carried out unit's study, exports final cluster result.

6, device according to claim 5 is characterized in that, described device also comprises: pretreatment module;

Described pretreatment module is used for the text of text set carry out text vectorization, and the text set after the text vectorization is sent to described text analysis model.

7, device according to claim 6 is characterized in that, described pretreatment module comprises: participle unit, Feature Selection unit and text vector unit;

Described participle unit is used for the text of text set is divided into single speech, and adds up the number of times that each speech occurs in text set, will divide result and statistics and send to described Feature Selection unit;

Described Feature Selection unit is used for according to the division result and the statistics that receive, and selected characteristic speech from text set sends to described text vector unit with the feature speech of choosing;

Described text vector unit is used for according to the feature speech that receives the text of text set being changed into text vector, and the text set after the text vectorization is sent to described text analysis model.

8, device according to claim 5 is characterized in that, described matrix synthesis module comprises: matrixing unit and synthesis unit;

Described matrixing unit, the cluster or the classification results that are used for receiving change into matrix, and the matrix after transforming is sent to described synthesis unit;

Described synthesis unit, the matrix after all conversions that are used for receiving is spliced into a text vector matrix, and described text vector matrix is sent to described first study module.

9, device according to claim 5 is characterized in that, described first study module comprises: normalization unit and unit;

Described normalization unit is used for the text vector matrix that receives is carried out normalized, and the text vector matrix after the normalization is sent to described unit;

Described unit is used for the text vector matrix after the normalization that receives is carried out unit's study, exports final cluster result.

Images