CN116166973A - Portable near infrared spectrum quantitative model and unknown sample matching degree judging method - Google Patents

Abstract

The invention discloses a method for judging the matching degree of a portable near infrared spectrum quantitative model and an unknown sample, which comprises the steps of constructing a quantitative model by using a partial least square method to obtain a load matrix and a score matrix of a PLS main component; respectively calculating score threshold values of the first main component and the second main component, and constructing an elliptic polar coordinate expression by using the two threshold values; mapping the acquired unknown sample spectrum into a main component to obtain a score value; substituting the score value of the first principal component into an elliptic polar coordinate expression to obtain predicted score values of two principal components to form a judging section; and detecting whether the score value of the second principal component of the unknown sample is in a judging section, so as to judge the matching degree of the unknown sample and the quantitative model. The invention judges the matching degree with the quantitative model in the unknown sample prediction process, distinguishes the abnormal sample, can prompt to update the model in time, and ensures the stability and accuracy of the spectrometer in long-time use.

Description

Discrimination method of matching degree between portable near-infrared spectroscopy quantitative model and unknown samples

technical field

The invention relates to the technical field of discrimination of near-infrared abnormal spectrum samples, in particular to a method for discriminating the matching degree between a portable near-infrared spectrum quantitative model and an unknown sample.

Background technique

Near-infrared spectroscopy is widely used in the detection of material components, and it has the characteristics of fast, non-destructive, high efficiency and low cost. Portable near-infrared spectrometer refers to a near-infrared analysis instrument that can be hand-held or portable. This kind of near-infrared spectrometer is light, easy to carry, low power consumption, fast detection speed, low cost, and is widely used in many fields such as agriculture, medicine, geology and environment.

However, portable near-infrared spectrometers are easily affected by light sources, detectors, detection methods, and environmental conditions, resulting in poor stability and low precision of the collected spectral data, which in turn affects its spectral prediction and analysis capabilities. In practical applications, due to the influence of spectrometer itself and sample changes, the spectral data collected by portable near-infrared spectroscopy equipment is more likely to have a low matching degree with the model, which will affect the prediction accuracy of the spectrometer for unknown samples. At the same time, when using a portable spectrometer to collect spectral samples, due to reasons such as manual operation, sample status, and instrument status, spectral abnormalities will occur during the spectral collection of unknown samples. When predicting unknown samples with spectral abnormalities, there will be leading to abnormal predictions.

Contents of the invention

The purpose of the present invention is to provide a portable method for judging the matching degree between a quantitative model of near-infrared spectrum and an unknown sample, which is used to judge the matching degree between a quantitative model and an unknown sample, and then improve the accuracy of predicting an unknown sample by using a quantitative model.

The present invention solves the above problems through the following technical solutions:

A method for discriminating the matching degree between a portable near-infrared spectrum quantitative model and an unknown sample, comprising the following steps:

Step a. Based on the spectral data of the training set, utilize the partial least squares method PLS to construct a quantitative model, obtain the loading matrix X_loadings of each variable of the PLS principal component one and the principal component two and the score matrix Xt_scores of each training set sample;

Step b. Using the multivariate t-test method to calculate the score thresholds of principal component 1 and principal component 2 respectively, and bring the two thresholds into the ellipse polar coordinate formula as the long radius and short radius to construct the ellipse polar coordinate expression;

Step c. Using the loading matrix X_loadings obtained in step a, map the collected unknown sample spectrum to principal component 1 and principal component 2, and obtain the score values of principal component 1 and principal component 2 corresponding to the unknown sample;

Step d. Substituting the score value of principal component 1 into the ellipse polar coordinate expression constructed in step b, and obtaining the predicted score values of two principal components 2 with the same value and opposite sign, the obtained two principal components 2 The predicted score value constitutes the judgment interval;

Step e. Checking one by one whether the score value of principal component 2 of the unknown sample is within the judgment interval, so as to judge the matching degree between the unknown sample and the quantitative model.

As a further improvement of the present invention, in the step e, judging the degree of matching between the unknown sample and the quantitative model includes:

(1), when the principal component 2 score value of the unknown sample is within the judgment interval, it is judged that the matching degree between the unknown sample and the quantitative model is high, and the normal prediction is performed on the unknown sample;

(2) When the principal component 2 score of the unknown sample is not within the judgment interval, it is judged that the matching degree between the unknown sample and the quantitative model is low, and the spectrum acquisition of the unknown sample is performed again.

As a further improvement of the present invention, the matching degree discrimination method also includes:

Step f. After secondary spectrum collection, if it is also determined that the matching degree between the unknown sample and the quantitative model is low, then the unknown sample is regarded as a marginal sample for normal prediction;

Step g. After predicting the unknown samples in a time period, detect the matching degree between the quantitative model and the unknown samples in the collection of unknown samples in the collection time period, and set a threshold to judge the overall matching degree.

As a further improvement of the present invention, in the step f, the edge samples are samples whose calibration values exceed the variation range of the calibration values of the samples in the training set.

As a further improvement of the present invention, in the step g, including:

1) When the overall matching degree of the unknown sample set in this time period is greater than the threshold, the original quantitative model is used to distinguish and predict the unknown samples in the next time period;

2) When the overall matching degree of the unknown sample set in the time period is less than the threshold, the quantitative model is updated.

As a further improvement of the present invention, said step b includes:

b1), calculated according to the multivariate t-test method, the score values of each sample point to principal component 1 and principal component 2 obey the F distribution after being changed;

b2), set the significant level, and obtain the score threshold on principal component 1 and principal component 2, the calculation formula of the score threshold is:

threshold=(s _h F _2,n-2,α 2(n ² -1)/(n(n-2))) ^0.5 ;

Among them, threshold is the score threshold; s _h is the variance of the h-th principal component, and h takes 1 or 2; F _{2,n-2, α} is the boundary of the rejection domain of the F distribution, which can be known by looking up the table; α is the significant level; n is the number of samples.

7. according to claim 6, a kind of portable near-infrared spectrum quantitative model and unknown sample matching degree discriminant method, it is characterized in that, in described step b, according to multivariate t test method, calculate and draw each sample point to principal component The score values of 1 and principal component 2 obey the F distribution after being changed, and the formula is:

Among them, t _hi is the score value of the i-th sample in the training set for the h-th principal component, h is 1 or 2, s _h is the variance of the h-th principal component, n is the number of samples, and m is the number of variables.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

The invention provides a portable near-infrared spectrum quantitative model and unknown sample matching degree discrimination method, the method can identify abnormal samples by judging the matching degree of the unknown sample and the quantitative model in the prediction process, prompting to re-collect the spectrum, and avoid artificial Acquisition factors, when the sample status, internal and external environment, and the status of the spectrometer change, the proportion of samples with a low matching degree in the unknown sample set is relatively large, and it can prompt to update the model in time to ensure the stability and accuracy of the spectrometer for long-term use .

Description of drawings

Fig. 1 is a schematic diagram of a method for discriminating the matching degree between a portable near-infrared spectrum quantitative model and an unknown sample according to the present invention.

Fig. 2 is a schematic diagram of judging the degree of matching between an unknown sample and a quantitative model in the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Example

Figure 1 is a method for discriminating the matching degree between a portable near-infrared spectrum quantitative model and an unknown sample, which includes the following steps:

S101. Based on the spectral data of the training set, use the partial least squares method PLS to construct a quantitative model, and obtain the loading matrix X_loadings of each variable of the PLS principal component 1 and principal component 2 and the score matrix Xt_scores of each training set sample.

Specifically, the partial least squares method PLS is used to convert the overall variables that may be correlated into a set of linearly uncorrelated variables, and the converted set of variables is the principal component; the training set contains the spectra and calibration values of each sample (true value ), there is only spectrum in the unknown sample, by performing eigendecomposition on the correlation coefficient matrix of the spectrum of the training set and the calibration value, the loading matrix and score matrix of the principal components are obtained, the low-order principal components are retained, and the high-order principal components are ignored. While reducing the dimensionality of features with high correlation. In this embodiment, only two principal components, principal component one and principal component two, are kept.

In the process of building a quantitative model, multiple samples need to be processed at the same time. The form of the processing is a matrix. The spectral collection of multiple samples in the training set is used as a spectral matrix. After PLS, the spectral matrix can be expressed by the following formula:

Among them, X is the n×m training set spectral matrix, n is the number of samples, m is the number of variables, Xt _scores is the score matrix of n×p, p is the number of main components, because only the main component 1 and the main component 2 are reserved here , so P is 2, X _loadings is the m×p loading matrix, E is the residual matrix, and T is the transpose.

S102. Using the multivariate t-test method to calculate the score thresholds of principal component 1 and principal component 2 respectively, and taking the two thresholds as the long radius and short radius into the ellipse polar coordinate formula to construct the ellipse polar coordinate expression;

Specific steps include:

b1), according to the multivariate t-test method, it is calculated that the scores of each sample point for the first and second principal components obey the F distribution after being changed, that is:

Among them, t _hi is the score value of the i-th sample in the training set for the h-th principal component, h is 1 or 2, and s _h is the variance of the h-th principal component.

b2), set the significance level to 0.05, the score threshold threshold of principal component 1 and principal component 2 can be obtained, namely:

threshold=(s _h F _2,n-2,0.05 2(n ² -1)/(n(n-2))) ^0.5 ;

Among them, threshold is the score threshold; s _h is the variance of the h-th principal component, and h takes 1 or 2; F _{2,n-2, α} is the boundary of the rejection domain of the F distribution, which can be known by looking up the table; α is the significant level; n is the number of samples.

b3) The obtained score threshold threshold includes a matrix of score thresholds of principal component 1 and principal component 2, which is substituted into the ellipse polar coordinate matrix to construct an ellipse polar coordinate expression.

Substitute this into the elliptic polar coordinate matrix:

x=threshold(1)×cos(θ);

y=threshold(2)×sin(θ);

Among them, θ is the angle, and the value range is [0°, 360°]; threshold(1) is the score threshold of the main component 1; threshold(2) is the score threshold of the main component 2; x and y represent The independent variable and dependent variable of the formula specifically mean the score value of principal component 1 and the score value of principal component 2.

Bring the principal component 1 of the unknown sample into the formula as x, and two y values with the same size and opposite sign will be obtained, which can be used as a range threshold. When the real principal component 2 score of the unknown sample exceeds this range threshold, then judged to be abnormal.

S103. Using the previously obtained loading matrix X_loadings of the principal component, map the collected unknown sample spectrum Xp to the first principal component and the second principal component, and obtain the first principal component score Xp_scores1 and the second principal component score corresponding to the sample value Xp_scores2;

The mapping method is:

Xp _scores = Xp×X _loadings ;

S104. Substituting the score value Xp_scores1 of principal component 1 into the ellipse polar coordinate expression to obtain two predicted score values of principal component 2 with the same numerical values and opposite signs, and the judgment interval formed by these two values can be expressed as: [ -Xk_scores,Xk_scores];

S105. Detect one by one whether the scores of the real principal components 2 of the unknown samples are within the judgment interval, so as to judge the matching degree between the unknown samples and the quantitative model.

specific:

(1) When Xp_scores2∈[-Xk_scores,Xk_scores] of a single unknown sample indicates that the sample has a high degree of matching with the quantitative model and can be directly predicted.

时，表明该样本与定量模型的匹配度低，为了排除人为操作因素的干扰，重新采集光谱。(2) When a single unknown sample

When , it indicates that the matching degree between the sample and the quantitative model is low. In order to eliminate the interference of human operation factors, the spectrum was collected again.

S106. After secondary spectrum collection, if it is also judged that the matching degree between the sample and the quantitative model is low, it means that the influence of external factors such as manual operation and experimental environment can be excluded, and the unknown sample is used as a marginal sample for normal prediction operation.

In this embodiment, an edge sample is a sample whose calibration value exceeds the variation range of the calibration value of the samples in the training set.

S107. After predicting the unknown samples for a period of time, detect the matching degree between the quantitative model and the sample in the unknown sample set for the period of time, and set a threshold to judge the overall matching degree. The time period can be set to one month according to the frequency of the predicted samples. The overall match threshold is set at 80%.

S108. Use the threshold to determine the overall matching degree of the unknown sample set. When the overall matching degree is greater than the threshold, continue to use the original model for judgment and prediction in the next time period. When the overall matching degree is less than the threshold, update the model to strengthen the relationship between the model and the unknown samples. adaptability.

Figure 2 is a schematic diagram of judging the matching degree between an unknown sample and a quantitative model. The abscissa and ordinate of each dot in the figure represent the score values of principal component 1 and principal component 2 corresponding to each unknown sample, and the ellipse is represented by the score It is drawn by the polar coordinate expression composed of value thresholds. The dots inside the ellipse in the figure indicate that the corresponding unknown samples have a high degree of matching with the model, and the dots outside the ellipse indicate that the corresponding unknown samples match the model. lower degree.

The method of the present invention can distinguish abnormal samples by judging the matching degree between the unknown sample prediction process and the quantitative model, prompting to re-collect the spectrum, and avoiding artificial collection factors. When the sample state, internal and external environments, and the spectrometer state change, the unknown In the sample set, the proportion of samples with low matching degree is large, which can prompt to update the model in time to ensure the stability and accuracy of the spectrometer for long-term use.

In addition, the present invention also discloses a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned method for discriminating the matching degree between a portable near-infrared spectrum quantitative model and an unknown sample are realized.

Although the present invention has been described here with reference to the illustrative examples of the present invention, the above-mentioned examples are only preferred implementations of the present invention, and the implementation of the present invention is not limited by the above-mentioned examples. It should be understood that those skilled in the art Many other modifications and embodiments can be devised which will fall within the scope and spirit of the principles disclosed in this application.

Claims (7)

1. A portable near-infrared spectrum quantitative model and unknown sample matching degree discrimination method, is characterized in that, comprises the steps: Step a. Based on the spectral data of the training set, utilize the partial least squares method PLS to construct a quantitative model, obtain the loading matrix X_loadings of each variable of the PLS principal component one and the principal component two and the score matrix Xt_scores of each training set sample; Step b. Using the multivariate t-test method to calculate the score thresholds of principal component 1 and principal component 2 respectively, and bring the two thresholds into the ellipse polar coordinate formula as the long radius and short radius to construct the ellipse polar coordinate expression; Step c. Using the loading matrix X_loadings obtained in step a, map the collected unknown sample spectrum to principal component 1 and principal component 2, and obtain the score values of principal component 1 and principal component 2 corresponding to the unknown sample; Step d. Substituting the score value of principal component 1 into the ellipse polar coordinate expression constructed in step b, and obtaining the predicted score values of two principal components 2 with the same value and opposite sign, the obtained two principal components 2 The predicted score value constitutes the judgment interval; Step e. Checking one by one whether the score value of principal component 2 of the unknown sample is within the judgment interval, so as to judge the matching degree between the unknown sample and the quantitative model. 2. according to claim 1, a kind of portable near-infrared spectrum quantitative model and unknown sample matching method are characterized in that, in the step e, judging the matching degree of unknown sample and quantitative model comprises: (1), when the principal component 2 score value of the unknown sample is within the judgment interval, it is judged that the matching degree between the unknown sample and the quantitative model is high, and the normal prediction is performed on the unknown sample; (2) When the principal component 2 score of the unknown sample is not within the judgment interval, it is judged that the matching degree between the unknown sample and the quantitative model is low, and the spectrum acquisition of the unknown sample is performed again. 3. a kind of portable near-infrared spectrum quantitative model and unknown sample matching degree discrimination method according to claim 2, it is characterized in that, described matching degree discrimination method also comprises: Step f. After secondary spectrum collection, if it is also determined that the matching degree between the unknown sample and the quantitative model is low, then the unknown sample is regarded as a marginal sample for normal prediction; Step g. After predicting the unknown samples in a time period, detect the matching degree between the quantitative model and the unknown samples in the collection of unknown samples in the collection time period, and set a threshold to judge the overall matching degree. 4. according to claim 3, a kind of portable near-infrared spectrum quantitative model and unknown sample matching degree discriminant method, it is characterized in that, in described step f, edge sample is the sample whose calibration value exceeds the range of variation of training set sample calibration value for itself . 5. a kind of portable near-infrared spectrum quantitative model and unknown sample matching degree discriminant method according to claim 3, it is characterized in that, in described step g, comprise: 1) When the overall matching degree of the unknown sample set in this time period is greater than the threshold, the original quantitative model is used to distinguish and predict the unknown samples in the next time period; 2) When the overall matching degree of the unknown sample set in the time period is less than the threshold, the quantitative model is updated. 6. according to any one of claim 1-5 described a kind of portable near-infrared spectrum quantitative model and unknown sample matching degree discriminant method, it is characterized in that, described step b comprises: b1), calculated according to the multivariate t-test method, the score values of each sample point to principal component 1 and principal component 2 obey the F distribution after being changed; b2), set the significant level, and obtain the score threshold on principal component 1 and principal component 2, the calculation formula of the score threshold is: threshold=(s _h F _{2, n-2, α} 2(n ² -1)/(n(n-2))) ^0.5 ; Among them, threshold is the score threshold; s _h is the variance of the h-th principal component, and h takes 1 or 2; F _{2, n-2, α} is the boundary of the rejection domain of the F distribution, which can be known by looking up the table; α is the significant level; n is the number of samples. 7. according to claim 6, a kind of portable near-infrared spectrum quantitative model and unknown sample matching degree discriminant method, it is characterized in that, in described step b, according to multivariate t test method, calculate and draw each sample point to principal component The score values of 1 and principal component 2 obey the F distribution after being changed, and the formula is:

Among them, t _hi is the score value of the i-th sample in the training set for the h-th principal component, h is 1 or 2, s _h is the variance of the h-th principal component, n is the number of samples, and m is the number of variables.

Images