CN103761532B - Label space dimensionality reducing method and system based on feature-related implicit coding - Google Patents

Abstract

The present invention proposes a label space dimensionality reduction method based on feature-related implicit coding, which includes the following steps: providing a training data set; constructing a feature matrix and a labeling matrix according to the training data set; obtaining the optimal dimensionality reduction matrix and feature matrix according to the feature matrix Optimum correlation function, and obtain the optimal restoration error function of the dimension reduction matrix and the annotation matrix according to the labeling matrix; construct the objective function according to the optimal correlation function and the optimal restoration error function; apply the objective function to optimize the dimensionality reduction matrix, and according to the optimized The dimensionality reduction matrix is solved to obtain the decoding matrix; the optimized dimensionality reduction matrix is used to learn and train to obtain the prediction model; the characteristics of the test instance are extracted, and the prediction model is used to predict the representation of the test instance in the latent semantic space; and the decoding matrix is used to analyze the test instance in the latent semantic space The representation in the latent semantic space is decoded to obtain the classification result of the test instance in the original label space. The method of the invention has high compression rate, good stability and strong universality.

Description

Label space dimensionality reduction method and system based on feature-related implicit coding

technical field

The invention relates to computer software technology, in particular to a label space dimensionality reduction method and system based on feature-related implicit coding.

Background technique

Multi-label classification technology (Multi-label classification) is mainly used to divide an instance into one or more categories, so as to describe the characteristics of the instance more completely and in detail, and the category to which the instance belongs is also called its category. The corresponding label (Label). Multi-label classification technology has a very wide range of applications in reality, such as multi-label text classification, image semantic annotation, audio sentiment analysis and so on. In recent years, with the emergence and rapid development of network applications, multi-label classification applications have begun to face many challenges and difficulties brought about by the expansion of data volume, including the rapid growth of label space. For example, on the picture sharing website Flickr, when uploading pictures, users can choose several words to describe the pictures from millions or even more vocabularies. For multi-label classification applications using Flickr data, such as network image semantic annotation, these text words will be regarded as different labels, so such a large number of labels will bring cost to the underlying algorithm learning process of these applications Great improvement. For multi-label classification, the basic idea of a large number of current methods is still to decompose it into multiple binary classification problems, that is, to train a corresponding predictive model (Predictive model) for each label to judge whether an instance belongs to the label, and finally All tags to which the instance belongs are its corresponding multiple descriptions. When the label space expands rapidly, that is, when the number of labels is very large, the number of prediction models that these methods need to train also increases rapidly, resulting in a significant increase in their training costs.

The emergence of label space dimensionality reduction points out a feasible exploration direction for solving the multi-label classification problem with a large number of labels, and provides technical support. In recent years, it has gradually become a hot spot in the research community, and several excellent dimensionality reduction method. For example, using the sparsity of the original label space, the dimensionality reduction of the label space is carried out by means of compressed sensing (Compressed sensing), and the corresponding decoding algorithm is used to recover from the latent semantic space to the original label space. On the basis of this scheme, some researchers further unify the dimensionality reduction process and the learning process of the prediction model under the same probability model framework, and then improve the classification performance by optimizing the above two processes at the same time. In addition, some studies also apply the principal component analysis method (Principal component analysis) to label space dimensionality reduction, which is called the Principal label space transformation method. Furthermore, some researchers took into account the correlation between feature space and latent semantic space, and proposed the Feature-ware conditional principal labelspace transformation method, which achieved a more obvious performance improvement. Another researcher also proposed to use linear Gaussian random projection direction to map the original label space, and retain the mapped symbol value as the result of dimensionality reduction, while the decoding process uses a series of KL divergence (Kullback-Leibler divergence) ) for hypothesis testing. There are also researchers who directly perform Boolean matrix decomposition on the annotation matrix of the training data to obtain the dimensionality reduction matrix and the decoding matrix. A linear map to the original label space.

From the current research, the main solution is to presuppose an explicit encoding function, and it is usually taken as a linear function. However, due to the complexity of the high-dimensional space structure, the explicit encoding function may not be able to accurately describe the mapping relationship between the original label space and the optimal latent semantic space, thus affecting the final dimensionality reduction results. In addition, although there are a few works that can directly learn the dimensionality reduction results without assuming an explicit encoding function, these works do not take into account the correlation between the latent semantic space and the feature space, which may make the final dimensionality reduction results difficult. Described by a predictive model learned from the feature space, leading to poor final classification performance.

Contents of the invention

The present invention aims to solve one of the technical problems in the related art at least to a certain extent. Therefore, an object of the present invention is to propose a label space dimensionality reduction method based on feature-related implicit coding, which has sufficient information considerations, high classification performance retention, high label space compression rate, good stability, and strong universality. .

Another object of the present invention is to propose a label space dimensionality reduction system based on feature correlation implicit coding.

The embodiment of the first aspect of the present invention proposes a label space dimensionality reduction method based on feature-related implicit coding, including the following steps: providing a training data set; constructing a feature matrix and a labeling matrix according to the training data set; Matrix to obtain the optimal correlation function between the dimensionality reduction matrix and the feature matrix, and obtain the optimal recovery error function between the dimensionality reduction matrix and the annotation matrix according to the labeling matrix; according to the optimal correlation function and the constructing an objective function with an optimal recovery error function; optimizing the dimensionality reduction matrix by applying the objective function, and solving a decoding matrix according to the optimized dimensionality reduction matrix; using the optimized dimensionality reduction matrix to learn and train to obtain a prediction model; extracting test instance features, and using the predictive model to predict the representation of the test instance in the latent semantic space; and using the decoding matrix to decode the representation of the test instance in the latent semantic space to obtain the The classification results of the above test examples in the original label space.

According to the label space dimensionality reduction method based on feature-related implicit coding according to the embodiment of the present invention, in the process of learning the dimensionality reduction result, the recovery error with the labeling matrix and the correlation with the feature space are also fully considered. Through the optimization process It ensures that the dimensionality reduction results can be well restored to the label matrix, and can also be described by the prediction model learned in the feature space, so that better multi-label classification performance can be achieved at a lower training cost.

In some examples, the dimensions of the latent semantic space are mutually orthogonal.

In some examples, binary processing is performed on the classification result of the test instance in the original label space.

In some examples, the dimensionality of the latent semantic space is smaller than the dimensionality of the original label space.

The embodiment of the second aspect of the present invention proposes a label space dimensionality reduction system based on feature-related implicit coding, including: a training module, which is used to perform learning and training according to the training data set to obtain a prediction model; The above prediction model obtains the classification result of the test instance in the original label space.

According to the label space dimensionality reduction system based on feature-related implicit coding according to the embodiment of the present invention, in the process of learning dimensionality reduction results, the recovery error with the labeling matrix and the correlation with the feature space are also fully considered. Through the optimization process It ensures that the dimensionality reduction results can be well restored to the label matrix, and can also be described by the prediction model learned in the feature space, so that better multi-label classification performance can be achieved at a lower training cost.

In some examples, the training module specifically includes: a construction module, configured to construct a feature matrix and a label matrix according to training data; an optimization module, used to obtain the optimal dimensionality reduction matrix and the feature matrix according to the feature matrix A correlation function, and obtain the optimal restoration error function between the dimensionality reduction matrix and the annotation matrix according to the annotation matrix; a modeling module is used to construct an objective function according to the optimal correlation function and the optimal restoration error function , and after applying the objective function to optimize the dimensionality reduction matrix, use the optimized dimensionality reduction matrix to solve the decoding matrix; a learning module is used to use the optimized dimensionality reduction matrix to learn and train to obtain a prediction model.

In some examples, the dimensions of the latent semantic space are mutually orthogonal.

In some examples, binary processing is performed on the classification result of the test instance in the original label space.

In some examples, the dimensionality of the latent semantic space is smaller than the dimensionality of the original label space.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

Fig. 1 is a flow chart of a label space dimensionality reduction method based on feature correlation implicit coding according to an embodiment of the present invention;

Fig. 2 is a schematic diagram of a label space dimensionality reduction method based on feature-related implicit coding according to an embodiment of the present invention;

Fig. 3 is a structural block diagram of a label space dimensionality reduction system based on feature correlation implicit coding according to an embodiment of the present invention; and

Fig. 4 is a structural block diagram of a training module of an embodiment of the present invention.

detailed description

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

In fact, the main purpose of label space dimensionality reduction is to compress the high-dimensional original label space (Original labelspace), and encode it into a low-dimensional latent semantic space (Latentsemantic space) while maintaining acceptable algorithm performance. , so that the original model training process is decomposed from the learning process of the prediction model from the feature space (Feature space) to the original label space into the learning process of the prediction model from the feature space to the latent semantic space and from the latent semantic space to the original label Spatial decoding process. Through dimensionality reduction, the number of prediction models required to go from feature space to latent semantic space will be greatly reduced compared to the number required before dimensionality reduction. And, if the prediction model is accurate enough, and at the same time, the decoding process from the latent semantic space to the original label space is also accurate and efficient enough, then the final multi-label classification performance should still be acceptable theoretically, while training Costs are greatly reduced.

In the embodiment of one aspect of the present invention, a label space dimensionality reduction method based on feature-related implicit coding is proposed, which includes the following steps: providing a training data set; constructing a feature matrix and a labeling matrix according to the training data set; The optimal correlation function between the dimension matrix and the feature matrix, and the optimal restoration error function of the dimensionality reduction matrix and the annotation matrix are obtained according to the annotation matrix; the objective function is constructed according to the optimal correlation function and the optimal restoration error function; the optimal dimensionality reduction is applied to the objective function Matrix, and solve the decoding matrix according to the optimized dimensionality reduction matrix; use the optimized dimensionality reduction matrix to learn and train to obtain the prediction model; extract the characteristics of the test instance, and use the prediction model to predict the representation of the test instance in the latent semantic space; and The representation of the test instance in the latent semantic space is decoded by the decoding matrix to obtain the classification result of the test instance in the original label space.

Figure 1 is a flow chart of a label space dimensionality reduction method based on feature-related implicit coding according to an embodiment of the present invention, and Figure 2 is a schematic diagram of a feature-related implicit coding-based label space dimensionality reduction method according to an embodiment of the present invention . The label space dimensionality reduction method based on feature correlation implicit coding of the present invention is described in detail with reference to FIG. 1 and FIG. 2 .

Step S101: providing a training data set.

As shown in FIG. 2 , the principle frame diagram of the label space dimensionality reduction method based on feature-related implicit coding, the method of the present invention includes a training process and a prediction process. In the training process, a certain number of training data sets need to be given.

Step S102: Construct a feature matrix and a label matrix according to the training data set.

Specifically, for a given training data set containing m test instances, select the appropriate feature type according to the attributes of the data itself, and extract the corresponding feature vector x=[x ₁ ,x ₂ ,. .., x _d ], where x _i is the i-th dimension of the feature vector x. After obtaining the eigenvectors of all test instances, they can be concatenated into the required eigenmatrix X by rows in any order, where X is an m×d matrix, where d is the dimension of the eigenvectors.

At the same time, for the training data set containing m test instances, count the number k of different labels appearing in it, and construct the corresponding label vector y=[y ₁ ,y ₂ ,...,y _k ], where y _j indicates whether the instance belongs to the jth label. If yes, the value is 1, otherwise, the value is 0, and so on. Similarly, after obtaining the label vectors of all test instances, they can be concatenated into a label matrix Y by row, which is consistent with the concatenation sequence of the feature matrix, and Y is an m×k matrix.

Step S103: Obtain the optimal correlation function between the dimensionality reduction matrix and the feature matrix according to the feature matrix, and obtain the optimal restoration error function between the dimensionality reduction matrix and the labeling matrix according to the labeling matrix.

Specifically, on the one hand, the optimal correlation function between the dimensionality reduction matrix and the feature matrix is obtained according to the feature matrix.

In the actual operation process, combined with the implicit coding method, it is assumed that there is a dimensionality reduction matrix C. The correlation between the dimensionality reduction matrix C and the feature matrix X can be decomposed into the sum of the correlations between each column of the dimensionality reduction matrix C and the feature matrix X. For any column c in the dimensionality reduction matrix C, the correlation between it and the feature matrix X can be described by cosine correlation, expressed as a function as follows:

Among them, r is a linear map of X, which is used to project X to the space where c is located.

At the same time, in order to reduce the redundant information in the dimensionality reduction results, it is assumed that the columns of the dimensionality reduction matrix C are orthogonal to each other, that is, the dimensions of the latent semantic space described by the dimensionality reduction matrix C are orthogonal to each other, and the corresponding mathematical expression The formula is C ^T C = E. From C ^T C = E, it can be known that c ^T c = 1, and the linear scaling of r does not affect the value of the cosine correlation ρ, so the following optimization function can be constructed to solve the optimal linear map r, and then Get the optimal correlation between c and X.

ρ ^* =max _r (Xr) ^T c

Constraints: (Xr) ^T (Xr)=1

The optimal linear mapping can be obtained by the method of Lagrange multipliers Re-substituting into the above optimization function can get the optimal correlation as Among them, Δ=X(X ^T X) ^-1 X ^T .

Therefore, the optimal correlation between the dimensionality reduction matrix C and the feature matrix X can be expressed as the following function:

Among them, C _.,i represents the i-th column of the dimensionality reduction matrix C.

On the other hand, the optimal recovery error function of the dimensionality reduction matrix and the labeling matrix is obtained according to the labeling matrix.

Combined with the method of implicit coding, it is assumed that there is a dimensionality reduction matrix C, and C is an m×l matrix, where l is the dimension size after dimensionality reduction, so l<<k.

Assuming that the dimensionality reduction matrix C exists, the error restored from C to the label matrix can be expressed as the following function:

Among them, D is the linear decoding matrix introduced to ensure the decoding efficiency, Represents the square of the Frobenius normal form of the matrix. Given the dimensionality reduction matrix C, the optimal restoration error is the smallest ε. Therefore, the optimal decoding matrix D and the optimal recovery error can be obtained by minimizing ε. Optimize the function by constructing the following:

The optimal decoding matrix D ^* =(C ^T C) ^-1 C ^T Y of the decoding matrix D can be solved. Since C ^T C = E is an identity matrix, D ^* can be further simplified to D ^* = C ^T Y, and re Substituting the above optimization function, the optimal recovery error function is obtained: ε ^* =Tr[Y ^T YY ^T CC ^T Y], where Tr[ ] represents the trace of the matrix, that is, the sum of the diagonal elements.

Step S104: Construct an objective function according to the optimal correlation function and the optimal restoration error function.

Through step S103, the optimal restoration error function ε ^* =Tr[Y ^T YY ^T CC ^T Y] from the dimensionality reduction matrix C to the labeling matrix Y can be obtained, as well as its optimal correlation function with the feature matrix X Therefore, the optimal dimensionality reduction matrix should be able to minimize the above optimal restoration error function and maximize the above optimal correlation function at the same time.

According to the properties of the matrix trace, ε ^* =Tr[Y ^T YY ^T CC ^T Y]=Tr[Y ^T Y]-Tr[Y ^T CC ^T Y], since Tr[Y ^T Y] is a constant, then minimize ε ^* is equivalent to maximizing Tr[Y ^T CC ^T Y]. Furthermore, for the optimal correlation part, since maximizing is equivalent to maximizing Therefore, maximizing P ^* is equivalent to maximizing Therefore, in order to minimize ε ^* and maximize P ^* at the same time, the optimal dimensionality reduction matrix C can be solved by constructing the following equivalent objective function.

Ω=max _C Tr[Y ^T CC ^T Y]+αTr[C ^T ΔC]

Constraints: C ^T C = E

Among them, α is a weight parameter, which is used to adjust the weight relationship between the optimal recovery error and the optimal correlation. According to the properties of the matrix trace, the above objective function can be rewritten as follows:

Ω=max _C Tr[C ^T (YY ^T +αΔ)C]

Constraints: C ^T C = E

Step S105: Applying the objective function to optimize the dimensionality reduction matrix, and solving the decoding matrix according to the optimized dimensionality reduction matrix.

Specifically, the optimal dimensionality reduction matrix C can be obtained through the objective function Ω obtained in step S104. The solution of C can be decomposed into the optimization problem of each column. For the optimal solution of the i-th column C _.,i of the dimensionality reduction matrix C, the following optimization sub-problems can be constructed:

Restrictions:

Using the Lagrange multiplier method, the optimal C can be obtained _{, i} needs to meet the following optimality conditions:

Among them, λ _i is the introduced Lagrangian multiplier, and substituting it into the above optimization sub-problem can get the optimal Ω _i , namely λ _i .

It can be observed from the above optimality conditions that the optimal C _.,i is a unit eigenvector (Eigenvector) of the matrix (YY ^T +αΔ), and due to the orthogonality of the eigenvectors, the following orthogonal constraints can be naturally Satisfy. So far, it can be found that the dimensionality reduction matrix C is actually composed of unit eigenvectors corresponding to the largest l eigenvalues (Eigenvalue) in the matrix (YY ^T + αΔ) concatenated by columns. Therefore, the process of solving the dimensionality reduction matrix C is the process of performing eigenvalue decomposition on the matrix (YY ^T + αΔ), so that all eigenvalues of the matrix and the unit eigenvectors corresponding to each eigenvalue can be obtained. Since the complexity of eigenvalue decomposition is not greater than And because (YY ^T +αΔ) is a symmetric matrix, and in one embodiment of the present invention, only the unit eigenvectors corresponding to the largest l eigenvalues are required, so the complexity of the dimensionality reduction matrix C can be lower than This ensures that the process of solving the dimensionality reduction matrix C is sufficiently efficient.

In addition, after the dimensionality reduction matrix C is solved by the eigenvalue decomposition method, the optimal decoding matrix D can be obtained by minimizing the optimal restoration error from the dimensionality reduction matrix C to the original labeling matrix Y. It can be seen from the calculation process of step S103 that the optimal decoding matrix D ^* =C TY ^.

Step S106: learning and training by using the optimized dimensionality reduction matrix to obtain a prediction model.

According to the optimal dimensionality reduction matrix C obtained in step S105, corresponding prediction models are trained for each dimension of the latent semantic space described by it. Specifically, for the i-th dimension (1≤i≤l), the value of the j-th test instance in this dimension is C _j,i , so that the values of the training data of all test instances in this dimension constitute the The vector is the ith column C _.,i of C. According to the feature matrix X of the training data of all test instances and the value vector C _.,i in the i-th dimension of the latent semantic space, the corresponding prediction model h _i :X→C _.,i can be learned and trained to predict any The value of the instance on this dimension, the input is the feature vector of the test instance, and the output is the value of the test instance on the i-th dimension.

In actual operation, the choice of prediction model can be set according to the specific situation of the application, and commonly used ones include linear regression (Linear regression). After label space dimensionality reduction, the dimension l of the latent semantic space is often much smaller than the dimension k of the original label space, which greatly reduces the number of required prediction models and effectively reduces the training cost.

Step S107: Extract the features of the test instance, and use the predictive model to predict the representation of the test instance in the latent semantic space.

Specifically, when a test instance to be classified is given, it is necessary to extract the same features as the training process, and obtain the d-dimensional feature vector of the test instance

After getting the feature vector of the test instance Afterwards, use the l prediction models learned in step S106 to predict the values of the test instance in each dimension of the latent semantic space, so as to obtain its l-dimensional representation vector in the latent semantic space

Step S108: Use the decoding matrix to decode the representation of the test instance in the latent semantic space to obtain the classification result of the test instance in the original label space.

Utilize the optimal decoding matrix D ^* obtained in step S105 to decode the l-dimensional representation vector of the test instance in the latent semantic space, and restore the k-dimensional representation vector of the original label space that is

The vector obtained at this time The value is a real value, which needs to be binarized by setting a threshold (usually 0.5). Specifically, if the value of each dimension exceeds the set threshold, the value is 1, otherwise the value is 0, thus indicating the label attribution of the test instance in the original label space, that is, each dimension with a value of 1 The label corresponding to the dimension is the multi-label classification result of the test instance to be classified.

In one embodiment of the present invention, each dimension of the latent semantic space is orthogonal to each other, which minimizes the redundant information in the dimensionality reduction result, so that this method can encode more information in the original label space with a lower dimension. information, which ensures a greater compression rate for the original label space during dimensionality reduction.

In one embodiment of the present invention, the dimensionality of the latent semantic space is smaller than the dimensionality of the original label space, and there is no need to pre-set an explicit encoding function, thereby ensuring that the number of prediction models required by the method of the embodiment of the present invention is large The reduction in magnitude effectively reduces the training cost, so that the optimal dimension reduction result can be learned adaptively in different data scenarios, and the stability is better.

For example, through experiments on the standard data set delicious in the field of text classification, the effectiveness of the label space dimensionality reduction method based on feature-related implicit coding in the embodiment of the present invention is verified. Specifically, the label space dimension on the delicious data set is reduced to the original 10%, 20%, 30%, 40% and 50%, and the classification achieved by the method of the embodiment of the present invention is observed at different ratios Performance, measured by label-based average F1 score and instance-based average precision (both higher is better). Table 1 shows the statistical results of the classification performance of the method of the present invention, and also provides the classification performance that can be achieved by using a linear support vector machine (Support vector machine) with outstanding performance when the label space dimensionality reduction is not performed, so as to carry out Comparison before and after dimensionality reduction. It can be seen from the results in the table that the method of the embodiment of the present invention can maintain 68% of the F1 value before dimensionality reduction on the average F1 value while only retaining 10% of the original label space dimension, and at the same time, the average accuracy 85% of that before dimensionality reduction. It can be seen that the method of the embodiment of the present invention can effectively reduce the dimension of the original label space, and can well guarantee acceptable classification performance while greatly reducing the training cost.

Table 1 Experimental results of the method of the embodiment of the present invention on the delicious data set

Dimension ratio after dimensionality reduction 10% 20% 30% 40% 50% Before dimensionality reduction Tag-based average F1-score 0.054 0.059 0.060 0.060 0.059 0.079 Instance-based average precision 0.120 0.121 0.120 0.120 0.112 0.142

According to the label space dimensionality reduction method based on feature-related implicit coding according to the embodiment of the present invention, in the process of learning the dimensionality reduction result, the recovery error with the labeling matrix and the correlation with the feature space are also fully considered. Through the optimization process It ensures that the dimensionality reduction results can be well restored to the label matrix, and can also be described by the prediction model learned in the feature space, so that better multi-label classification performance can be achieved at a lower training cost.

Another embodiment of the present invention proposes a label space dimensionality reduction system based on feature-related implicit coding, including: a training module 100 and a prediction module 200 , as shown in FIG. 3 .

Wherein, the training module 100 is configured to perform learning and training according to the training data set to obtain a prediction model. The prediction module 200 is configured to obtain the classification result of the test instance in the original label space according to the prediction model obtained by the training module 100 .

Specifically, as shown in FIG. 4 , the training module 100 of the embodiment of the present invention specifically includes: a construction module 10 , an optimization module 20 , a modeling module 30 and a learning module 40 .

Wherein, the construction module 100 is configured to construct a feature matrix and a label matrix according to the training data set.

Specifically, for a given training data set containing m test instances, select the appropriate feature type according to the attributes of the data itself, and extract the corresponding feature vector x=[x ₁ ,x for each test instance ₂ ,...,x _d ], where xi is the i-th dimension of the feature vector x. After obtaining the eigenvectors of all test instances, they can be concatenated into the required eigenmatrix X by rows in any order, where X is an m×d matrix, where d is the dimension of the eigenvectors.

At the same time, for the training data set containing m test instances, count the number k of different labels appearing in it, and construct the corresponding label vector y=[y ₁ ,y ₂ ,...,y _k ], where y _j indicates whether the instance belongs to the jth label. If yes, the value is 1, otherwise, the value is 0, and so on. Similarly, after obtaining the label vectors of all test instances, they can be concatenated into a label matrix Y by row, which is consistent with the concatenation sequence of the feature matrix, and Y is an m×k matrix.

The optimization module 20 is used to obtain the optimal correlation function between the dimensionality reduction matrix and the feature matrix by using the feature matrix, and obtain the optimal restoration error function between the dimensionality reduction matrix and the labeling matrix according to the labeling matrix.

Specifically, on the one hand, the optimal correlation function between the dimensionality reduction matrix and the feature matrix is obtained according to the feature matrix.

In the actual operation process, combined with the implicit coding method, it is assumed that there is a dimensionality reduction matrix C. The correlation between the dimensionality reduction matrix C and the feature matrix X can be decomposed into the sum of the correlations between each column of the dimensionality reduction matrix C and the feature matrix X. For any column c in the dimensionality reduction matrix C, the correlation between it and the feature matrix X can be described by cosine correlation, expressed as a function as follows:

Among them, r is a linear map of X, which is used to project X to the space where c is located.

At the same time, in order to reduce the redundant information in the dimensionality reduction results, it is assumed that the columns of the dimensionality reduction matrix C are orthogonal to each other, that is, the dimensions of the latent semantic space described by the dimensionality reduction matrix C are orthogonal to each other, and the corresponding mathematical expression The formula is C ^T C = E. From C ^T C = E, it can be known that c ^T c = 1, and the linear scaling of r does not affect the value of the cosine correlation ρ, so the following optimization function can be constructed to solve the optimal linear map r, and then Get the optimal correlation between c and X.

ρ ^* =max _r (Xr) ^T c

Constraints: (Xr) ^T (Xr)=1

The optimal linear mapping can be obtained by the method of Lagrange multipliers Re-substituting into the above optimization function can get the optimal correlation as Among them, Δ=X(X ^T X) ^-1 X ^T .

Therefore, the optimal correlation between the dimensionality reduction matrix C and the feature matrix X can be expressed as the following function:

Among them, C _.,i represents the i-th column of the dimensionality reduction matrix C.

On the other hand, the optimal recovery error function of the dimensionality reduction matrix and the labeling matrix is obtained according to the labeling matrix.

Combined with the method of implicit coding, it is assumed that there is a dimensionality reduction matrix C, and C is an m×l matrix, where l is the dimension size after dimensionality reduction, so l<<k.

Assuming that the dimensionality reduction matrix C exists, the error restored from C to the label matrix can be expressed as the following function:

Among them, D is the linear decoding matrix introduced to ensure the decoding efficiency, Represents the square of the Frobenius normal form of the matrix. Given the dimensionality reduction matrix C, the optimal restoration error is the smallest ε. Therefore, the optimal decoding matrix D and the optimal recovery error can be obtained by minimizing ε. Optimize the function by constructing the following:

The optimal decoding matrix D ^* =(C ^T C) ^-1 C ^T Y of the decoding matrix D can be solved. Since C ^T C = E is an identity matrix, D ^* can be further simplified to D ^* = C ^T Y, and re Substituting the above optimization function, the optimal recovery error function is obtained: ε ^* =Tr[Y ^T YY ^T CC ^T Y], where Tr[ ] represents the trace of the matrix, that is, the sum of the diagonal elements.

The modeling module 30 is configured to construct an objective function according to the optimal correlation function and the optimal recovery error function, and after applying the objective function to optimize the dimensionality reduction matrix, use the optimized dimensionality reduction matrix to solve the decoding matrix.

Specifically, the optimal restoration error function E ^* =Tr[Y ^T YY ^T CC ^TY ] from the dimensionality reduction matrix C restored to the label matrix Y can be obtained through the optimization module 20, and its optimal correlation function with the feature matrix X Therefore, the optimal dimensionality reduction matrix should be able to minimize the above optimal restoration error function and maximize the above optimal correlation function at the same time.

According to the properties of the matrix trace, ε ^* =Tr[Y ^T YY ^T CC ^T Y]=Tr[Y ^T Y]-Tr[Y ^T CC ^T Y], since Tr[Y ^T Y] is a constant, then minimize ε ^* is equivalent to maximizing Tr[Y ^T CC ^T Y]. Furthermore, for the optimal correlation part, since maximizing is equivalent to maximizing Therefore, maximizing P ^* is equivalent to maximizing Therefore, in order to minimize ε ^* and maximize P ^* at the same time, the optimal dimensionality reduction matrix C can be solved by constructing the following equivalent objective function.

Ω=max _C Tr[Y ^T CC ^T Y]+αTr[C ^T ΔC]

Constraints: C ^T C = E

Among them, α is a weight parameter, which is used to adjust the weight relationship between the optimal recovery error and the optimal correlation. According to the properties of the matrix trace, the above objective function can be rewritten as follows:

Ω=max _C Tr[C ^T (YY ^T +αΔ)C]

Constraints: C ^T C = E

Through the objective function Ω, the optimal dimensionality reduction matrix C can be obtained. The solution of the optimal dimensionality reduction matrix C can be decomposed into the optimization problem of each column. For the optimal solution of the i-th column C _.,i of the dimensionality reduction matrix C, the following optimization sub-problems can be constructed:

Restrictions:

Using the Lagrange multiplier method, it can be obtained that the optimal C _.,i must satisfy the following optimality conditions:

(YY ^T +αΔ)C _.,i =λ _i C _.,i

Among them, λ _i is the introduced Lagrangian multiplier, and substituting it into the above optimization sub-problem can get the optimal Ω _i , namely λ _i .

It can be observed from the above optimality conditions that the optimal C _.,i is a unit eigenvector (Eigenvector) of the matrix (YY ^T +αΔ), and due to the orthogonality of the eigenvectors, the following orthogonal constraints can be naturally Satisfy. So far, it can be found that the dimensionality reduction matrix C is actually composed of unit eigenvectors corresponding to the largest l eigenvalues (Eigenvalue) in the matrix (YY ^T + αΔ) concatenated by columns. Therefore, the process of solving the dimensionality reduction matrix C is the process of performing eigenvalue decomposition on the matrix (YY ^T + αΔ), so that all eigenvalues of the matrix and the unit eigenvectors corresponding to each eigenvalue can be obtained. Since the complexity of eigenvalue decomposition is not greater than And because (YY ^T +αΔ) is a symmetric matrix, and in one embodiment of the present invention, only the unit eigenvectors corresponding to the largest l eigenvalues are required, so the complexity of the dimensionality reduction matrix C can be lower than This ensures that the process of solving the dimensionality reduction matrix C is sufficiently efficient.

In addition, after the dimensionality reduction matrix C is solved by the eigenvalue decomposition method, the optimal decoding matrix D can be obtained by minimizing the optimal restoration error from the dimensionality reduction matrix C to the original labeling matrix Y. It can be seen from the calculation process of the optimization module 20 that the optimal decoding matrix D ^* =C TY ^.

The learning module 40 is configured to use the optimized dimensionality reduction matrix to learn and train to obtain a prediction model.

According to the optimal dimensionality reduction matrix C obtained by the modeling module 30, corresponding prediction models are trained for each dimension of the latent semantic space described by it. Specifically, for the i-th dimension (1≤i≤l), the value of the j-th test instance in this dimension is C _j,i , so that the values of the training data of all test instances in this dimension constitute the The vector is the ith column C _.,i of C. According to the feature matrix X of the training data of all test instances and the value vector C _.,i in the i-th dimension of the latent semantic space, the corresponding prediction model h _i :X→C _.,i can be learned and trained to predict any The value of the instance on this dimension, the input is the feature vector of the test instance, and the output is the value of the test instance on the i-th dimension.

In actual operation, the choice of prediction model can be set according to the specific situation of the application, and commonly used ones include linear regression (Linear regression). After label space dimensionality reduction, the dimension l of the latent semantic space is often much smaller than the dimension k of the original label space, which greatly reduces the number of required prediction models and effectively reduces the training cost.

In addition, the prediction module 200 of the embodiment of the present invention specifically includes:

(1) When a test instance to be classified is given, it is necessary to extract the same features as the training process, and obtain the d-dimensional feature vector of the test instance

(2) After getting the feature vector of the test instance After that, use the l prediction models learned in the training module 100 to predict the values of the test instance in each dimension of the latent semantic space, thereby obtaining its l-dimensional representation vector in the latent semantic space

(3) Use the decoding matrix to decode the representation of the test instance in the latent semantic space to obtain the classification result of the test instance in the original label space.

Using the optimal decoding matrix D ^* obtained by the optimization module 30, the l-dimensional representation vector of the test instance in the latent semantic space is decoded, and the k-dimensional representation vector of the original label space is restored that is

The vector obtained at this time The value is a real value, which needs to be binarized by setting a threshold (usually 0.5). Specifically, if the value of each dimension exceeds the set threshold, the value is 1, otherwise the value is 0, thus indicating the label attribution of the test instance in the original label space, that is, each dimension with a value of 1 The label corresponding to the dimension is the multi-label classification result of the test instance to be classified.

In one embodiment of the present invention, each dimension of the latent semantic space is orthogonal to each other, which minimizes the redundant information in the dimensionality reduction result, so that this method can encode more information in the original label space with a lower dimension. information, which ensures a greater compression rate for the original label space during dimensionality reduction.

In one embodiment of the present invention, the dimensionality of the latent semantic space is smaller than the dimensionality of the original label space, and there is no need to pre-set an explicit encoding function, thereby ensuring that the method of the embodiment of the present invention can be used in different data scenarios Adaptively learn the optimal dimensionality reduction result with better stability.

According to the label space dimensionality reduction system based on feature-related implicit coding according to the embodiment of the present invention, in the process of learning dimensionality reduction results, the recovery error with the labeling matrix and the correlation with the feature space are also fully considered. Through the optimization process It ensures that the dimensionality reduction results can be well restored to the label matrix, and can also be described by the prediction model learned in the feature space, so that better multi-label classification performance can be achieved at a lower training cost.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims (8)

1. A label space dimensionality reduction method based on feature correlation implicit encoding, characterized in that, comprising the following steps: Provide training data set; Construct feature matrix and label matrix according to described training data set; Obtaining the optimal correlation function between the dimensionality reduction matrix and the feature matrix according to the feature matrix, and obtaining the optimal restoration error function between the dimensionality reduction matrix and the labeling matrix according to the labeling matrix, wherein, according to the labeling matrix The feature matrix obtains the optimal correlation function between the dimensionality reduction matrix and the feature matrix specifically includes: The dimensionality reduction matrix is obtained by combining the labeling matrix with an implicit coding method; The correlation between the dimensionality reduction matrix and the feature matrix is decomposed into a sum of correlations, and expressed in the form of a cosine correlation function as follows: Among them, r is a linear mapping of the feature matrix X, which is used to project the feature matrix X to the space where any column c in the dimensionality reduction matrix C is located; Obtain the optimal linear mapping r according to the cosine correlation function, and obtain the optimal correlation between any column c in the dimensionality reduction matrix C and the feature matrix X; The optimal linear mapping r ^* is obtained by the Lagrange multiplier method, and the optimal correlation function is obtained according to the optimal linear mapping r ^* : Among them, C _{, i} represents the i-th column of the dimensionality reduction matrix C; Wherein, the optimal restoration error function obtained from the dimensionality reduction matrix and the labeling matrix according to the labeling matrix specifically includes: The annotation matrix is combined with implicit coding to obtain the dimensionality reduction matrix; The error function expression of restoring the dimensionality reduction matrix to the labeling matrix is as follows: Among them, when ε is the smallest, the recovery error is the smallest, D is the linear decoding matrix introduced to ensure the decoding efficiency, Represents the square of the Frobenius normal form of the matrix; The optimal recovery error function is obtained by minimizing ε, and the expression is as follows: Constructing an objective function according to the optimal correlation function and the optimal restoration error function; Optimizing the dimensionality reduction matrix by applying the objective function, and solving the decoding matrix according to the optimized dimensionality reduction matrix; Using the optimized dimensionality reduction matrix to learn and train to obtain a prediction model; extracting test instance features, and using the predictive model to predict the representation of the test instance in a latent semantic space; and Decoding the representation of the test instance in the latent semantic space by using the decoding matrix to obtain a classification result of the test instance in the original label space. 2. The method according to claim 1, wherein the dimensions of the latent semantic space are mutually orthogonal. 3. The method according to claim 1, wherein the classification result of the test instance in the original label space is binarized. 4. The method according to claim 1, wherein the dimensionality of the latent semantic space is smaller than the dimensionality of the original label space. 5. A label space dimensionality reduction system based on feature-related implicit coding, characterized in that it comprises: A training module, configured to perform learning and training according to the training data set to obtain a prediction model, wherein the training module specifically includes: A construction module for constructing a feature matrix and a label matrix according to the training data; The optimization module is used to obtain the optimal correlation function between the dimensionality reduction matrix and the feature matrix according to the feature matrix, and obtain the optimal restoration error function between the dimensionality reduction matrix and the labeling matrix according to the labeling matrix, wherein , the optimal correlation function between the dimensionality reduction matrix and the feature matrix obtained according to the feature matrix specifically includes: The dimensionality reduction matrix is obtained by combining the labeling matrix with an implicit coding method; The correlation between the dimensionality reduction matrix and the feature matrix is decomposed into a sum of correlations, and expressed in the form of a cosine correlation function as follows: Among them, r is a linear mapping of the feature matrix X, which is used to project the feature matrix X to the space where any column c in the dimensionality reduction matrix C is located; Obtain the optimal linear mapping r according to the cosine correlation function, and obtain the optimal correlation between any column c in the dimensionality reduction matrix C and the feature matrix X; The optimal linear mapping r ^* is obtained by the Lagrange multiplier method, and the optimal correlation function is obtained according to the optimal linear mapping r ^* : Among them, C _{., i} represents the i-th column of the dimensionality reduction matrix C; Wherein, the optimal restoration error function obtained from the dimensionality reduction matrix and the labeling matrix according to the labeling matrix specifically includes: The annotation matrix is combined with implicit coding to obtain the dimensionality reduction matrix; The error function expression of restoring the dimensionality reduction matrix to the labeling matrix is as follows: Among them, when ε is the smallest, the recovery error is the smallest, D is the linear decoding matrix introduced to ensure the decoding efficiency, Represents the square of the Frobenius normal form of the matrix; The optimal recovery error function is obtained by minimizing ε, and the expression is as follows: A modeling module, configured to construct an objective function according to the optimal correlation function and the optimal restoration error function, and after applying the objective function to optimize the dimensionality reduction matrix, use the optimized dimensionality reduction matrix to solve the decoding matrix ; A learning module, configured to use the optimized dimensionality reduction matrix to learn and train to obtain a prediction model; The prediction module is used to obtain the classification result of the test instance in the original label space according to the prediction model. 6. The system according to claim 5, wherein the dimensions of the latent semantic space are mutually orthogonal. 7. The system according to claim 5, wherein the classification result of the test instance in the original label space is binarized. 8. The system according to claim 5, wherein the dimensionality of the latent semantic space is smaller than the dimensionality of the original label space.