CN104408129A

CN104408129A - Classifier model establishment method based on distance from point to local feature space

Info

Publication number: CN104408129A
Application number: CN201410699884.4A
Authority: CN
Inventors: 李波; 王运庆; 张晓龙
Original assignee: Wuhan University of Science and Technology WHUST
Current assignee: Wuhan University of Science and Technology WHUST
Priority date: 2014-11-26
Filing date: 2014-11-26
Publication date: 2015-03-11

Abstract

本发明涉及一种基于点到局部特征空间距离的分类器模型建立方法。其技术方案是：对于任一未知类别的样本点X_i，采用K近邻方法确定任一未知类别的样本点X_i的K个已知类别为c_j(j＝1,2,...L)的近邻点，再分别组成类别为c_j(j＝1,2,...L)的局部特征空间，计算未知类别的样本点X_i到每一个具有不同类别的局部特征空间的欧氏距离，然后选择到未知类别样本点X_i欧氏距离最小的局部特征空间的类别，作为未知类别样本点X_i的类别。本发明能有效地抑制噪声或奇异点对数据类别预测的干扰，增强了分类器模型的鲁棒性能，提高了未知类别的样本点的类别预测效果。The invention relates to a method for establishing a classifier model based on the distance from a point to a local feature space. The technical solution is: for any sample point X _i of unknown category, use the K nearest neighbor method to determine the K known categories of sample point X _i of any unknown category as c _j (j=1,2,...L ), and then form a local feature space of category c _j (j=1,2,...L ₎ , and calculate the Euclidean distance, and then select the category of the local feature space with the smallest Euclidean distance to the sample point X _i of the unknown category as the category of the sample point X _i of the unknown category. The invention can effectively suppress the interference of noise or singular points on data category prediction, enhance the robust performance of the classifier model, and improve the category prediction effect of unknown category sample points.

Description

Classifier model building method based on distance from point to local feature space

所属领域Field

本发明属于分类器技术领域。具体涉及一种基于点到局部特征空间距离的分类器模型建立方法。The invention belongs to the technical field of classifiers. Specifically, it relates to a method for establishing a classifier model based on the distance from a point to a local feature space.

背景技术Background technique

数据挖掘的主要任务有分类分析、聚类分析、关联规则分析、序列模式分析等，其中分类分析由于其特殊地位，一直是数据挖掘研究的热点。对于数据分类问题，通常采取的流程是首先对数据进行预处理，然后采用线性或者非线性特征提取方法，从预处理的数据中提取最有利于分类的特征，最后建立分类器模型，对所提取的特征进行类别预测。在整个数据分类过程中，如何建立数据分类模型是关系到数据分类效果好坏的关键之一。分类器模型的任务就是训练一个分类器，分析输入的样本集合，通过在训练集中的数据表现出来的特性，为每一个类寻找一种准确的描述。由所生成的类描述对未来的测试数据进行分类，尽管这些未来的测试数据的类别标签是未知的，但仍可以由此预测这些新数据所属的类别。The main tasks of data mining include classification analysis, cluster analysis, association rule analysis, sequential pattern analysis, etc. Among them, classification analysis has always been a hot spot in data mining research because of its special status. For data classification problems, the usual process is to preprocess the data first, then use linear or nonlinear feature extraction methods to extract the most favorable features for classification from the preprocessed data, and finally establish a classifier model to extract features for category prediction. In the whole process of data classification, how to establish a data classification model is one of the keys related to the quality of data classification. The task of the classifier model is to train a classifier, analyze the input sample set, and find an accurate description for each class through the characteristics of the data in the training set. The generated class descriptions are used to classify future test data, and although the class labels of these future test data are unknown, it is still possible to predict the class to which these new data belong.

目前有许多方法和技术可以用于构造分类器模型，例如决策树、决策表、神经网络、K近邻方法、遗传算法、贝叶斯方法以及支持向量机等。然而，这些单一的分类技术在应用中常常会受到一定条件的限制。对于一些非线性分类方法，如神经网络和支持向量机，由于算法复杂，造成计算代价很高。另外决策树、决策表，遗传算法和贝叶斯方法也因为计算复杂，运算量大，给大数据分类带来很大的困扰。作为一种计算简单的数据分类方法，K近邻采用一种以点到点的欧氏距离作为近邻点选择的度量，在K设置为1的情况下，K近邻方法转化为最近邻方法，未知类别数据样本点的类别就可以预测为距离其最近的样本点的类别，因此计算成本相对较低。但是K近邻方法的鲁棒性能比较差，在数据中如果包含有噪声或者奇异点的时候，特别是当噪声或者奇异点距离预测样本点比较近的时候，很容易影响未知类别样本点的预测效果，对数据的分类结果造成很大的影响。There are many methods and techniques that can be used to construct classifier models, such as decision trees, decision tables, neural networks, K-nearest neighbor methods, genetic algorithms, Bayesian methods, and support vector machines. However, the application of these single classification techniques is often limited by certain conditions. For some nonlinear classification methods, such as neural networks and support vector machines, the computational cost is high due to the complexity of the algorithm. In addition, decision trees, decision tables, genetic algorithms, and Bayesian methods also bring great troubles to big data classification because of their complex calculations and large amounts of calculations. As a computationally simple data classification method, K-nearest neighbors use a point-to-point Euclidean distance as a measure of neighbor point selection. When K is set to 1, the K-nearest neighbor method is transformed into the nearest neighbor method, and the unknown category The category of the data sample point can be predicted as the category of the nearest sample point, so the calculation cost is relatively low. However, the robustness of the K-nearest neighbor method is relatively poor. If the data contains noise or singular points, especially when the noise or singular points are relatively close to the predicted sample points, it is easy to affect the prediction effect of unknown class sample points. , which has a great impact on the classification results of the data.

发明内容Contents of the invention

本发明目的在于提出一种基于点到局部特征空间距离的分类器模型建立方法，所建立的分类器模型能有效地抑制噪声或奇异点对数据类别预测的干扰，能增强分类器模型的鲁棒性能，能提高未知类别的样本点的类别预测效果。The purpose of the present invention is to propose a method for establishing a classifier model based on the distance from a point to a local feature space. The established classifier model can effectively suppress the interference of noise or singular points on data category prediction, and can enhance the robustness of the classifier model. Performance, which can improve the category prediction effect of sample points of unknown categories.

为实现上述目的，本发明采用的技术方案是，所述分类器模型建立方法的具体步骤是：In order to achieve the above object, the technical solution adopted in the present invention is that the specific steps of the method for establishing the classifier model are:

步骤一、计算任一未知类别的样本点X_i与类别为c_s的所有样本点之间的欧式距离，取所述欧式距离由小到大排列的前k个类别为c_s的样本点将所述前k个类别为c_s的样本点作为任一未知类别样本点X_i的类别为c_s的局部近邻点，构建类别为c_s的局部特征空间i表示自然数，s∈{1,2，...,L}，L表示所有样本点的类别总数。Step 1. Calculate the Euclidean distance between any sample point X _i of an unknown category and all sample points of the category c _s , and take the first k sample points of the category c _s arranged in ascending order of the Euclidean distance Set the sample points of the first k categories as c _s As the local neighbor point of any unknown category sample point X _i with category c _s , construct a local feature space with category c _s i represents a natural number, s ∈ {1,2,...,L}, L represents the total number of categories of all sample points.

步骤二、重复步骤一，分别从类别为c_t(t＝1,2,...,L-1)的所有样本点中，选取k个类别为c_t(t＝1,2,...,L-1)的局部近邻点构建相应的类别为c_t(t＝1,2,...,L-1)的局部特征空间L表示所有样本点的类别总数，所述的K为大于3的自然数。Step ₂ , repeat step 1, select k categories as c _t (t=1,2,.. .,L-1) local neighbors Construct the corresponding local feature space of category c _t (t=1,2,...,L-1) L represents the total number of categories of all sample points, and the K is a natural number greater than 3.

步骤三、分别计算任一未知类别的样本点X_i到类别为c_j的局部特征空间的欧氏距离 Step 3. Calculate the sample point X _i of any unknown category to the local feature space of category c _j Euclidean distance

${D D.}^{(({c c}_{j j}))} (({X x}_{i i})) = = | | | | {X x}_{i i} - - {f f}^{(({c c}_{j j}))} (({X x}_{i i})) | | | | - - - - - - ((11))$

式(1)中：In formula (1):

c_j表示类别为c₁，c₂，...,c_L中的任一类；c _j means that the category is any one of c ₁ , c ₂ ,...,c _L ;

表示任一未知类别的样本点X_i到类别为c_j的局部特征空间的投影， Represents the sample point X _i of any unknown category to the local feature space of category c _j the projection of

${f f}^{(({c c}_{j j}))} (({X x}_{i i})) = = \underset{m m}{Σ Σ} {a a}_{im im} {X x}_{im im}^{(({c c}_{j j}))} - - - - - - ((22))$

式(2)中：In formula (2):

表示任一未知类别的样本点X_i的类别为c_j的局部近邻点； Represents the local neighbor points of the sample point X _i of any unknown category whose category is c _j ;

a_im(m＝1,2,...,k)表示是在类别为c_j的局部特征空间内，由类别为c_j的局部近邻点所线性表示的任一未知类别的样本点X_i的线性系数， a _im (m=1,2,...,k) means that in the local feature space of category c _j , the local neighbor points of category c _j The linear coefficient of the sample point X _i of any unknown category expressed linearly,

步骤四、预测任一未知类别的样本点X_i的类别Step 4. Predict the category of any unknown category sample point X _i

将任一未知类别的样本点X_i到类别为c_j的局部特征空间的欧氏距离按照由小到大的顺序进行排列，选择到任一未知类别的样本点X_i具有最小欧氏距离的局部特征空间的类别，作为任一未知类别的样本点X_i的类别，其中：c_j表示类别为c₁，c₂，...,c_L中的任一类。Put the sample point X _i of any unknown category into the local feature space of category c _j Euclidean distance Arranged in order from small to large, select the category of the local feature space with the smallest Euclidean distance to the sample point _Xi of any unknown category, as the category of the sample point _Xi of any unknown category, where: c _j Indicates that the category is any one of c ₁ , c ₂ , . . . , c _L .

由于采用上述技术方案，本发明与现有技术相比的有益效果是：Owing to adopting above-mentioned technical scheme, the beneficial effect of the present invention compared with prior art is:

本发明在应用于数据分类时，在传统K近邻分类器的基础上，提供了基于点到局部特征空间距离的分类器模型建立方法。对于任一未知类别的样本点X_i，采用K近邻方法确定任一未知类别的样本点X_i的K个已知类别为c_j(j＝1,2,...,L)的近邻点，组成类别为c_j(j＝1,2,...,L)的局部特征空间；然后计算任一未知类别的样本点X_i到每一个具有不同类别的局部特征空间的欧氏距离最后选择到未知类别样本点X_i欧氏距离最小的局部特征空间的类别，作为任一未知类别样本点X_i的类别。传统K近邻方法中采用点到点距离，容易受噪声或奇异点的影响。本发明采用点到局部特征空间距离作为预测未知样本点的类别的度量，能有效地抑制噪声或奇异点的干扰，增强分类模型的鲁棒性能。When applied to data classification, the present invention provides a method for establishing a classifier model based on the distance from a point to a local feature space on the basis of a traditional K-nearest neighbor classifier. For a sample point X _i of any unknown category, use the K nearest neighbor method to determine K known neighbor points of the sample point X _i of any unknown category whose category is c _j (j=1,2,...,L) , to form a local feature space with category c _j (j=1,2,...,L); then calculate the Euclidean distance from any unknown category sample point _Xi to each local feature space with different categories Finally, the category of the local feature space with the smallest Euclidean distance to the unknown category sample point X _i is selected as the category of any unknown category sample point X _i . The point-to-point distance is used in the traditional K-nearest neighbor method, which is easily affected by noise or singular points. The invention adopts the distance from the point to the local feature space as the measure for predicting the category of the unknown sample point, can effectively suppress the interference of noise or singular point, and enhance the robust performance of the classification model.

因此，本发明能有效地抑制噪声或奇异点对数据类别预测的干扰，增强了分类器模型的鲁棒性能，提高了未知类别的样本点的类别预测效果。Therefore, the present invention can effectively suppress the interference of noise or singular points on data category prediction, enhance the robust performance of the classifier model, and improve the category prediction effect of sample points of unknown categories.

具体实施方式Detailed ways

下面结合具体实施方式对本发明作进一步的描述，并非对其保护范围的限制：The present invention will be further described below in conjunction with specific embodiment, is not the limitation of its protection scope:

实施例1Example 1

一种基于点到局部特征空间距离的分类器模型建立方法。所述分类器模型建立方法的具体步骤是：A method for building a classifier model based on the distance from a point to a local feature space. The concrete steps of described classifier model establishment method are:

步骤二、重复步骤一，分别从类别为c_t(t＝1,2,...,L-1)的所有样本点中，选取k个类别为c_t(t＝1,2,...,L-1)的局部近邻点构建相应的类别为c_t(t＝1,2,...,L-1)的局部特征空间L表示所有样本点的类别总数，所述的K为大于3的自然数。 _Step 2, repeat step 1, select k categories as c _t (t=1,2,.. .,L-1) local neighbors Construct the corresponding local feature space of category c _t (t=1,2,...,L-1) L represents the total number of categories of all sample points, and the K is a natural number greater than 3.

式(1)中：In formula (1):

式(2)中：In formula (2):

本发明与现有技术相比的有益效果是：The beneficial effect of the present invention compared with prior art is:

本具体实施方式在应用于数据分类时，在传统K近邻分类器的基础上，提供了基于点到局部特征空间距离的分类器模型建立方法。对于任一未知类别的样本点X_i，采用K近邻方法确定任一未知类别的样本点X_i的K个已知类别为c_j(j＝1,2,...,L)的近邻点，组成类别为c_j(j＝1,2,...,L)的局部特征空间；然后计算任一未知类别的样本点X_i到每一个具有不同类别的局部特征空间的欧氏距离最后选择到未知类别样本点X_i欧氏距离最小的局部特征空间的类别，作为任一未知类别样本点X_i的类别。传统K近邻方法中采用点到点距离，容易受噪声或奇异点的影响。本具体实施方式采用点到局部特征空间距离作为预测未知样本点的类别的度量，能有效地抑制噪声或奇异点的干扰，增强分类模型的鲁棒性能。When this specific embodiment is applied to data classification, it provides a method for establishing a classifier model based on the distance from a point to a local feature space on the basis of a traditional K-nearest neighbor classifier. For a sample point X _i of any unknown category, use the K nearest neighbor method to determine K known neighbor points of the sample point X _i of any unknown category whose category is c _j (j=1,2,...,L) , to form a local feature space with category c _j (j=1,2,...,L); then calculate the Euclidean distance from any unknown category sample point _Xi to each local feature space with different categories Finally, the category of the local feature space with the smallest Euclidean distance to the unknown category sample point X _i is selected as the category of any unknown category sample point X _i . The point-to-point distance is used in the traditional K-nearest neighbor method, which is easily affected by noise or singular points. In this specific embodiment, the distance from a point to a local feature space is used as a measure for predicting the category of an unknown sample point, which can effectively suppress the interference of noise or singular points, and enhance the robustness of the classification model.

因此，本具体实施方式能有效地抑制噪声或奇异点对数据类别预测的干扰，增强了分类器模型的鲁棒性能，提高了未知类别的样本点的类别预测效果。Therefore, this specific embodiment can effectively suppress the interference of noise or singular points on data category prediction, enhance the robustness of the classifier model, and improve the category prediction effect of sample points of unknown categories.

Claims

1. a classifier model building method based on point to local feature space distance, it is characterized in that the concrete steps of described classifier model building method are:

Step 1. Calculate the Euclidean distance between any sample point X _i of an unknown category and all sample points of the category c _s , and take the first k sample points of the category c _s arranged in ascending order of the Euclidean distance Set the sample points of the first k categories as c _s As the local neighbor point of any unknown category sample point X _i with category c _s , construct a local feature space with category c _s i represents a natural number, s∈{1,2,...,L}, L represents the total number of categories of all sample points;

Step ₂ , repeat step 1, select k categories as c _t (t=1,2,.. .,L-1) local neighbors (t=1,2,...,L-1), construct the corresponding local feature space of category c _t (t=1,2,...,L-1) (t=1,2,...L-1), L represents the total number of categories of all sample points;

Step 3. Calculate the sample point X _i of any unknown category to the local feature space of category c _j Euclidean distance

{D D.}^{(({c c}_{j j}))} (({X x}_{i i})) = = | | | | {X x}_{i i} - - {f f}^{(({c c}_{j j}))} (({X x}_{i i})) | | | | - - - - - - ((11))

In formula (1):

c _j indicates that the category is any one of c ₁ , c ₂ , ..., c _L ,

Represents the sample point X _i of any unknown category to the local feature space of category c _j the projection of

{f f}^{(({c c}_{j j}))} (({X x}_{i i})) = = \underset{m m}{Σ Σ} {a a}_{im im} {X x}_{im im}^{(({c c}_{j j}))} - - - - - - ((22))

In formula (2):

Represents the local neighbor points of any unknown category sample point X _i whose category is c _j , a _im (m=1,2,...,k) means that it is in the local feature space of category c _j , by category is the local neighbor point of c _j The linear coefficient of the sample point X _i of any unknown category expressed linearly,

Step 4. Predict the category of any unknown category sample point X _i

Put the sample point X _i of any unknown category into the local feature space of category c _j Euclidean distance Arranged in order from small to large, select the category of the local feature space with the smallest Euclidean distance to the sample point _Xi of any unknown category, as the category of the sample point _Xi of any unknown category, where: c _j Indicates that the category is any one of c ₁ , c ₂ , . . . , c _L .

2. The method for establishing a classifier model based on the distance from a point to a local feature space according to claim 1, wherein said K is a natural number greater than 3.