CN109815815B - A Pedestrian Re-identification Method Based on Metric Learning and Support Vector Machine Integration - Google Patents

Abstract

The invention relates to a pedestrian re-identification method based on the integration of metric learning and support vector machine, and belongs to the technical fields of image processing and pattern recognition. The invention first generates a pedestrian feature matrix with pedestrian label information; processes the nonlinear space M that measures the distance of pedestrians; sets the pedestrian label information used in the support vector machine; the support vector machine introduces constraint variables, and then the support vector machine As the constraint condition of nonlinear space; scale the constraint condition of nonlinear space M; find the optimal solution of projection matrix and classifier, use the recognition model integrated with metric learning and support vector machine for pedestrian recognition, and get the recognition rate . The present invention integrates metric learning and support vector machine. Compared with the existing methods, the method proposed in the present invention effectively mines and utilizes the label information in the pedestrian data set, so that the pedestrian matching rate is effectively improved.

Description

A pedestrian re-identification method based on metric learning and support vector machine integration

technical field

The invention relates to a pedestrian re-identification method based on the integration of metric learning and support vector machine, and belongs to the technical fields of image processing and pattern recognition.

Background technique

With the promotion of national smart cities and safe cities, video surveillance systems basically cover major cities in my country. The pedestrian information data saved by the video surveillance system is huge, and the manual processing of information is efficient and low-cost. The pedestrian re-identification technology can effectively improve work efficiency and save resources. The main task of pedestrian re-identification is to match whether pedestrians from non-overlapping cameras are the same pedestrian. The technology is to extract a pedestrian from the camera a as the target pedestrian, determine whether the pedestrian has appeared under the camera b, and find the target pedestrian if it does. Therefore, pedestrian re-identification technology has received great attention from researchers.

At present, although pedestrian re-identification with strong robustness has attracted the attention of researchers and proposed some feasible solutions, the experimental results still cannot meet the actual needs, especially when pedestrians deliberately change their external characteristics when performing. The existing pedestrian re-identification technology still exists due to changes in viewing angle and illumination and different camera parameter settings, which cause the same pedestrian to show different underlying visual features under different vision, different lighting conditions and different cameras, but different pedestrians. Therefore, the pedestrian re-identification technology is far from meeting the needs of practical applications.

Pedestrian re-identification research results mainly include feature-based pedestrian re-identification and metric learning-based pedestrian re-identification. The feature-based pedestrian recognition is to extract features with stronger discriminative and expressive capabilities from the underlying features of pedestrians. Pedestrian re-identification based on metric learning is to find an algorithm with better matching effect from the perspective of metric learning. The feature-based pedestrian re-identification method uses the label information of pedestrians to extract the underlying features of pedestrians. However, due to factors such as drastic changes in illumination and different camera parameter settings, the underlying features such as pedestrian colors and textures change greatly, resulting in low pedestrian recognition accuracy. . However, the pedestrian re-identification method based on metric learning takes into account the drastic changes in illumination, different camera parameter settings and pedestrian appearance, etc., and uses the paired label information between the same pedestrian from different perspectives to narrow the metric distance between the same pedestrians. . However, when using the pedestrian label information, only the label information of the same pedestrian from different perspectives is considered, and the label information between different pedestrians from different perspectives is not considered. Aiming at the problem that the pedestrian re-identification algorithm based on metric learning has poor recognition effect and weak robustness under the interference factors of complex backgrounds such as pedestrians' clothes and similar postures.

SUMMARY OF THE INVENTION

The invention provides a pedestrian re-identification method based on the integration of metric learning and support vector machine, which mines and makes full use of label information between pedestrians from different perspectives to improve the accuracy of pedestrian recognition.

The technical scheme of the present invention is: a pedestrian re-identification method based on the integration of metric learning and support vector machine, firstly generating a pedestrian feature matrix with pedestrian label information; processing the nonlinear space M for measuring pedestrian distance; setting support The pedestrian label information used in the vector machine; the support vector machine introduces the constraint variable, and then the support vector machine is used as the constraint condition of the nonlinear space; the constraint condition of the nonlinear space M is scaled; The optimal solution is to use the recognition model integrated with metric learning and support vector machine for pedestrian recognition to obtain the recognition rate.

Further, the specific steps of the pedestrian re-identification method based on metric learning and support vector machine integration are as follows:

找出在a视角下与b视角下最相似但不是自己的行人

来生成带有行人标签信息的行人特征矩阵x_c；Step1. Project all the features of pedestrians in a and b perspectives into the same nonlinear space M∈R ^m×n , using the formula

Find the pedestrian who is most similar to the person in the perspective a and the perspective b but is not yourself

to generate a pedestrian feature matrix x _c with pedestrian label information;

与在b视角下与

最相似的但不是同一个人的行人

之间的度量距离要小于a视角下的第i个人

与它自己在b视角下

之间的度量距离，即Stpe2. In the nonlinear space M, the ith person under each a view angle is required

and in view b and

Pedestrians who are most similar but not the same person

The metric distance between them is smaller than the ith person under a view

with itself in b's perspective

the metric distance between

取0；即Stpe3. If all pedestrian features are processed according to Stpe2, the projected feature itself will meet the above situation, and overfitting will occur. Therefore, if the projected feature itself satisfies the huge difference in the wearing of the same pedestrian, then it will be processed according to Step2. If If different pedestrians wear similar clothes, no action will be taken, then

take 0; that is

与

是同一人，则设置它们的标签信息y_ij为1，如果不是则为-1；即Stpe4, the setting method of the pedestrian's label information is: if

and

are the same person, set their label information y _ij to 1, or -1 if they are not; i.e.

表示在支持向量机中

行人特征之间的度量距离；in,

represented in a support vector machine

the metric distance between pedestrian features;

其中，w为支持向量机的分类器；再把非线性空间M的约束条件引入支持向量机中，即：Stpe5, the traditional support vector machine introduces a constraint variable ξ _ij which is the inequality

Among them, w is the classifier of the support vector machine; then the constraints of the nonlinear space M are introduced into the support vector machine, namely:

st(y _ij (w(Mx _ai -Mx _bj )+c)>1-ξ _ij )

Step 6. Perform appropriate scaling processing on the constraint condition (y _ij (w(Mx _ai -Mx _bj )+c)>1-ξ _ij ) of the nonlinear space M, and eliminate ξ _ij after the conditional constraints are relaxed and optimized, that is,

Stpe7, find the optimal solution of the Stpe6 formula, train and learn to get the projection matrix and the classifier w; then use the projection matrix and the classifier w to perform pedestrian recognition in the recognition model integrated with metric learning and support vector machine, and get the recognition rate s, The metric learning of the recognition rate and the recognition model integrated with the support vector machine are as follows:

Among them, the role of c is to limit the range of categorical similarity values.

The beneficial effects of the present invention are:

The present invention integrates metric learning and support vector machine. Compared with the existing methods, the method proposed in the present invention effectively mines and utilizes the label information in the pedestrian data set, so that the pedestrian matching rate is effectively improved.

Description of drawings

Fig. 1 is a flowchart of the present invention; wherein, y represents the pedestrian label information matrix used in the support vector machine;

Figure 2 shows the comparative experimental results of the metric learning model and the model integrating metric learning and support vector machine on four datasets respectively.

Detailed ways

Embodiment 1: As shown in Figure 1-2, a pedestrian re-identification method based on the integration of metric learning and support vector machine, firstly generates a pedestrian feature matrix with pedestrian label information; do processing; set the pedestrian label information used in the support vector machine; introduce the constraint variable into the support vector machine, and then use the support vector machine as the constraint condition of the nonlinear space; scale the constraint condition of the nonlinear space M; find the projection matrix And the optimal solution of the classifier, the recognition model integrated with metric learning and support vector machine is used for pedestrian recognition, and the recognition rate is obtained.

Further, the specific steps of the pedestrian re-identification method based on metric learning and support vector machine integration are as follows:

找出在a视角下与b视角下最相似但不是自己的行人

来生成带有行人标签信息的行人特征矩阵x_c；Step1. Project all the features of pedestrians in a and b perspectives into the same nonlinear space M∈R ^m×n , using the formula

Find the pedestrian who is most similar to the person in the perspective a and the perspective b but is not yourself

to generate a pedestrian feature matrix x _c with pedestrian label information;

与在b视角下与

最相似的但不是同一个人的行人

之间的度量距离要小于a视角下的第i个人Stpe2. In the nonlinear space M, the ith person under each a view angle is required

and in view b and

Pedestrians who are most similar but not the same person

The metric distance between them is smaller than the ith person under a view

与它自己在b视角下

之间的度量距离，即

with itself in b's perspective

the metric distance between

取0；即Stpe3. If all pedestrian features are processed according to Stpe2, the projected feature itself will meet the above situation, and overfitting will occur. Therefore, if the projected feature itself satisfies the huge difference in the wearing of the same pedestrian, then it will be processed according to Step2. If If different pedestrians wear similar clothes, no action will be taken, then

take 0; that is

与

是同一人，则设置它们的标签信息y_ij为1，如果不是则为-1；即Stpe4, the setting method of the pedestrian's label information is: if

and

are the same person, set their label information y _ij to 1, or -1 if they are not; i.e.

表示在支持向量机中

行人特征之间的度量距离；in,

represented in a support vector machine

the metric distance between pedestrian features;

其中，w为支持向量机的分类器；再把非线性空间M的约束条件引入支持向量机中，即：Stpe5, the traditional support vector machine introduces a constraint variable ξ _ij which is the inequality

Among them, w is the classifier of the support vector machine; then the constraints of the nonlinear space M are introduced into the support vector machine, namely:

st(y _ij (w(Mx _ai -Mx _bj )+c)>1-ξ _ij )

Step 6. Perform appropriate scaling processing on the constraint condition (y _ij (w(Mx _ai -Mx _bj )+c)>1-ξ _ij ) of the nonlinear space M, and eliminate ξ _ij after the conditional constraints are relaxed and optimized, that is,

Stpe7, find the optimal solution of the Stpe6 formula, train and learn to get the projection matrix and the classifier w; then use the projection matrix and the classifier w to perform pedestrian recognition in the recognition model integrated with metric learning and support vector machine, and get the recognition rate s, The metric learning of the recognition rate and the recognition model integrated with the support vector machine are as follows:

Among them, the role of c is to limit the range of categorical similarity values.

In order to compare with existing methods, the present invention uses five datasets of VIPER, iLIDS-IVD, CUHK01, PRID2011 and PRID405S to conduct pedestrian re-identification experiments, and uses the average value of ten-fold cross-validation as the final result.

The evaluation index is consistent with the comparison method, and the CMC cumulative curve is used as the evaluation index (although the CMC curve graph lists the matching rate of rank1-rank20, in actual image retrieval and other applications, the value of rank1 is often the most important).

In order to verify the effect of SVM, the proposed metric model is taken as model one (Ours-svm shown in Fig. 2), and the model of metric learning plus SVM is taken as model two (Ours shown in Fig. 2). The CMC curves of the two models on the four datasets VIPER, iLIDS-IVD, PRID2011 and PRID405S are shown in Figure 2. It can be seen that before rank 5, metric learning and support vector machine have obvious effect on improving the pedestrian recognition rate, and it gradually tends to be stable with the increase of rank value.

On the VIPeR dataset, PCCA, LFDA, KISSME, LADF, Mid-filter, ECM, MFA, kLFDA, RD and SR45 models are compared, and the results are shown in Table 1. Although the CMC curve graph lists the matching rate of rank1-rank20, in actual image retrieval and other applications, the value of rank1 is often the most important. From Table 1, we can see that our proposed methods rank1 and rank5 are the best. .

Table 1: Comparison results of various methods on the VIPER dataset, listing the matching rates (%) of rank1, rank5, rank10 and rank20

Methods Rank1 Rank5 Rank10 Rank20 PCCA 19.3 48.9 64.9 80.3 LFDA 19.7 46.7 62.1 77.0 KISSME 19.6 48.0 62.2 77.0 LADF 29.3 61.0 76.0 86.2 Mid-filter 29.1 52.3 66.0 79.9 ECM 38.2 67.2 78.3 87.9 MFA 32.2 66.0 79.7 90.6 RD 33.3 41.5 78.4 88.5 kLFDA 32.3 65.8 79.7 90.6 SR 32.9 62.0 75.9 89.2 Ours 55.5 67.2 73.2 80.5

On the iLIDS-IDV dataset, SDALF-SS, Color+LBP+DTW, ISR, DVR, DVDL, PHDL+WHOS+STFV3D, Salience+DVR, KISSME are compared, and the results are shown in Table 2. It can be seen from Table 2 that the methods rank1 and rank5 proposed by the present invention have the highest recognition rates.

Table 2: Comparison results of various methods on the iLIDS-IVD dataset, listing the matching rates (%) of rank1, rank5, rank10 and rank20

The comparison results of the experiments on the CUHK01 data set and Rcca, ITML, KISSME, GenericMetric68, SalMatch, kLFDA, MidFilter, MirrorKMFA, LOMO+LADF, LOMO+XQDA are shown in Table 3. It can be seen from Table 3 that the method proposed by the present invention is The recognition effect on rank1 is the best.

Table 3: Comparison results of various methods on the CUHK01 dataset, listing the matching rates (%) of rank1, rank5, rank10 and rank20

Methods Rank1 Rank5 Rank10 Rank20 Rcca 14.9 32.6 43.8 55.5 ITML 16.0 35.2 45.6 59.8 KISSME 10.3 27.2 37.5 49.7 GenericMetric 20.0 43.6 56.0 69.3 SalMatch 28.5 45.9 55.7 68.0 kLFDA 26.1 49.4 58.4 71.8 MidFilter 34.3 55.1 65.0 74.9 MirrorKMFA 40.4 64.6 75.3 84.1 LOMO+LADF 58.0 83.7 90.5 94.9 LOMO+XQDA 63.2 83.9 90.0 94.4 Ours 64.5 81.9 85 88

On the PRID 2011 data set, PPLM, RDC, LOMO+LADF, MetricEnsemble, LOMO+M, XQDA, LOMO+XQDA, DVR, Salience+DVR are compared, and the results are shown in Table 4. Except for rank20, the effect is not ideal. The recognition results of rank1, rank2 and rank10 are all the best.

Table 4: Comparison results of various methods on PRID 2011 dataset, listing the matching rate (%) of rank1, rank5, rank10 and rank20

Methods Rank1 Rank5 Rank10 Rank20 PPLM 15.0 32.0 42.0 54.0 RDC 15.5 38.8 53.2 69.0 LOMO+LADF 16.2 34.0 44.4 59.5 MetricEnsemble 17.9 39.0 50.0 62.0 LOMO+M 15.2 36.1 48.3 60.4 XQDA 24.6 49.3 62.8 76.3 LOMO+XQDA 26.7 49.9 61.9 73.8 DVR 28.9 55.3 65.5 82.8 Salience+DVR 41.7 64.5 77.5 88.8 Ours 72.3 86.8 92 96.7

On the PRID450S data set, ELF, KISSME, EIML, SCNCD, ECM, TSR, MEDVL, KISSME-MGT, LOMO+LADF, KLFDA-MGT, MirrorKMFA and other methods are compared. .

Table 5: Comparison results of various methods on PRID_450S dataset, listing the matching rate (%) of rank1, rank5, rank10 and rank20

Methods Rank1 Rank5 Rank10 Rank20 ELF 30.6 - 73.6 84.2 KISSME 33.0 - 71.0 79.0 EIML 35.0 - 68.0 77 SCNCD 41.5 66.6 75.9 84.4 ECM 41.9 66.3 76.9 84.9 TSR 44.9 71.7 77.5 86.7 MEDVL 45.9 73.0 82.9 91.1 KISSME-MGT 46.1 73.3 83.3 90.7 LOMO+LADF 47.8 74.7 82.8 90.9 KLFDA-MGT 46.1 73.3 83.3 90.7 MirrorKMFA 55.4 79.3 87.8 93.9 Ours 58.4 68.7 73.1 80.2

The specific embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned embodiments, and can also be made within the scope of knowledge possessed by those of ordinary skill in the art without departing from the purpose of the present invention. Various changes.

Claims (1)

1. a pedestrian re-identification method based on metric learning and support vector machine integration, it is characterized in that: first generate a pedestrian feature matrix with pedestrian label information; do processing to the nonlinear space M of measuring pedestrian distance; set support vector The pedestrian label information used in the machine; the support vector machine introduces the constraint variable, and then the support vector machine is used as the constraint condition of the nonlinear space; the constraint condition of the nonlinear space M is scaled; the optimal projection matrix and classifier are found. Solution, use the recognition model integrated with metric learning and support vector machine to recognize pedestrians, and get the recognition rate; The specific steps of the pedestrian re-identification method based on metric learning and support vector machine integration are as follows: Step1. Project all the features of pedestrians in a and b perspectives into the same nonlinear space M∈R ^m×n , using the formula

Find the pedestrian who is most similar to the person in the perspective a and the perspective b but is not yourself

to generate a pedestrian feature matrix x _c with pedestrian label information; Stpe2. In the nonlinear space M, the ith person under each a view angle is required

and in view b and

Pedestrians who are most similar but not the same person

The metric distance between them is smaller than the ith person under a view

with itself in b's perspective

the metric distance between

Stpe3. If all pedestrian features are processed according to Stpe2, the projected feature itself will meet the above conditions, and overfitting will occur. Therefore, if the projected feature itself satisfies the huge difference in the wearing of the same pedestrian, it will be processed according to Stpe2. If If different pedestrians wear similar clothes, no action will be taken, then

take 0; that is

Stpe4, the setting method of the pedestrian's label information is: if

and

are the same person, set their label information y _ij to 1, or -1 if they are not; i.e.

in,

represented in a support vector machine

the metric distance between pedestrian features; Stpe5, the traditional support vector machine introduces a constraint variable ξ _ij which is the inequality

Among them, w is the classifier of the support vector machine; then the constraints of the nonlinear space M are introduced into the support vector machine, namely:

Step 6. Perform appropriate scaling processing on the constraint condition (y _ij (w(Mx _ai -Mx _bj )+c)>1-ξ _ij ) of the nonlinear space M, and eliminate ξ _ij after the conditional constraints are relaxed and optimized, that is,

Stpe7, find the optimal solution of the Stpe6 formula, train and learn to get the projection matrix and the classifier w; then use the projection matrix and the classifier w to perform pedestrian recognition in the recognition model integrated with metric learning and support vector machine, and get the recognition rate s, Among them, the metric learning of the recognition rate and the recognition model integrated with the support vector machine are as follows:

Among them, the role of c is to limit the range of categorical similarity values.

Images