CN116416523A - Machine learning-based rice growth stage identification system and method - Google Patents

Abstract

The invention discloses a system and a method for identifying a rice growth stage based on machine learning; the system comprises a rice characteristic generation module and a rice growth stage identification model construction module; the rice characteristic generation module is used for obtaining fractal dimension of rice and statistic constructed by using a gray level co-occurrence matrix according to the rice image; the rice growth stage identification model construction module is used for obtaining the mapping relation between the characteristics of rice and the rice growth stage according to the neural network model, so as to construct the rice growth stage identification model. The method introduces the discrimination of the fractal dimension and the gray level co-occurrence matrix to the growth period, the accuracy of the single machine learning model and the optimal weighted integration model after the fractal dimension variable is added is respectively improved by about 2-8%, the accuracy of the single machine learning model and the optimal weighted integration model after the gray level co-occurrence matrix variable is added is respectively improved by about 2-7%, and the detection accuracy of the rice growth stage is greatly improved.

Description

A rice growth stage recognition system and method based on machine learning

Technical Field

The invention belongs to the technical field of rice growth stage monitoring, and in particular relates to a rice growth stage identification system and method based on machine learning.

Background Art

Rice is one of the most important crops in the world and plays an important role in agricultural production. A full understanding of the growth stages of rice will enable people to take corresponding cultivation and management measures according to the physiological characteristics of rice at different growth stages and its requirements for external conditions, and reasonably meet its growth and development needs by using appropriate amounts of water, fertilizers and pesticides, so as to achieve the goal of high and stable rice yields; at the same time, automated supervision of crops at different growth stages can make timely and reasonable management decisions at different growth stages of rice, which is the future development direction of agriculture and is of great significance to modern farmland management.

Throughout the growth cycle, we can visually observe that the external morphological structure of rice has changed significantly, and as the growth stage of rice increases, the self-similarity and scale-free nature of the plant shape become more obvious. In recent years, with the development of intelligent agriculture, the application of computer vision technology in rice quality and growth stage detection has emerged. Technologies such as machine learning and the Internet are increasingly being used in agriculture to automatically observe, detect and distinguish different key growth stages of rice, realize automatic classification and prediction of rice, and thus improve the quantity and quality of rice.

At present, the main method of identification is manual inspection, but this is very cumbersome, time-consuming, laborious and often subject to the observer's subjective perception of the rice state. There is a problem that the traditional rice contour feature extraction is difficult and inaccurate, resulting in inaccurate judgment of the growth stage. Therefore, it is urgent to study the automatic recognition method of different growth stages of rice to reduce labor costs, improve the accuracy and real-time performance of observation, and avoid damage to plants. However, the traditional rice contour feature extraction is difficult and inaccurate, which leads to inaccurate judgment of the growth stage and cannot timely and accurately identify the key growth stage of rice plant development.

Summary of the invention

In order to improve the accuracy of rice growth stage judgment, the present invention proposes a rice growth stage recognition system and method based on machine learning.

A rice growth stage recognition system based on machine learning to achieve one of the purposes of the present invention comprises a rice feature generation module and a rice growth stage recognition model construction module;

The rice feature generation module is used to obtain multiple fractal dimensions of rice according to the rice image and to construct multiple statistics for characterizing the texture features of rice using the gray-level co-occurrence matrix;

The rice growth stage recognition model construction module is used to obtain the mapping relationship between rice characteristics and rice growth stages according to the neural network model, so as to construct a rice growth stage recognition model; the rice growth stage recognition model is used to recognize the growth stage of rice according to the rice image; the rice characteristics include multiple fractal dimensions of rice obtained from the rice feature generation module and statistics constructed using the gray-level co-occurrence matrix.

Furthermore, the rice feature generation module includes a first fractal dimension generation module, a second fractal dimension generation module and a texture feature acquisition module;

The first fractal dimension generation module is used to calculate two fractal dimensions D1 and RFD based on the whole rice and the edge of the rice leaf according to the rice image;

The second fractal dimension generation module is used to calculate two fractal dimensions D2 and Sandbox based on the rice as a whole and the circumscribed rectangle according to the rice image;

The texture feature acquisition module is used to obtain a grayscale symbiosis matrix according to the correlation between adjacent pixels and the grayscale changes of the diagonal elements of the symbiosis matrix, and to construct multiple statistics for characterizing the texture features of rice according to the grayscale symbiosis matrix.

Furthermore, the statistics include: contrast, difference, inverse difference matrix, entropy, correlation and angular second moment.

Furthermore, the fractal dimension generation module also includes a grayscale image generation module, which is used to convert the rice image into a grayscale image before obtaining the fractal dimension, and the grayscale image is used to obtain the fractal dimensions D1, RFD, D2 and Sandbox of the rice; the fractal dimension generation module also includes a binary image generation module, which is used to perform edge detection and denoising on the grayscale image of the rice image using the Sobel operator and Gaussian filtering method to obtain a binary image of the rice image, and the binary image is used to obtain the fractal dimensions D1, RFD, D2 and Sandbox of the rice.

The calculation method of the fractal dimension D1 and RFD includes:

S501, randomly selecting a pixel point A from the grayscale image of rice, and recording its coordinate value as (i, j); randomly selecting a pixel point A' from the binary image of rice, and recording its coordinate value as (i', j');

S502, using random walk method to determine another pixel point B and pixel point B' in the grayscale image and binary image of rice respectively, and their coordinate values are recorded as (u, v) and (u', v') respectively; and R = || (i, j) - (u, v) || = || (i', j') - (u', v') ||, R is a randomly set value, R = 1, 2, ..., n;

S503, calculating the difference G between the grayscale values of the two pixels A and B and the difference G' between the grayscale values of the two pixels A' and B':

G＝I(i,j)-I(u,v)

G'＝I(i',j')-I(u',v')

Where:

I(i,j) and I(u,v) are the grayscale values of pixel A and pixel B respectively;

I(i',j') and I(u',v') are the grayscale values of pixel A' and pixel B' respectively;

S503, repeating steps S501 to S502 to obtain multiple difference values G and G' of each grayscale image and binary image, and calculating the average value E(G) of each grayscale image and the average value E(G') of each binary image according to the difference values G and G';

S504, calculate two fractal dimensions of each grayscale image and binary image according to the following formula:

Where:

D1 is the fractal dimension based on the grayscale image;

RFD is the fractal dimension based on binary images;

c is a set constant.

Furthermore, the calculation method of the fractal dimension D2 and Sandbox also includes:

S701, selecting an M×M grid on each rice grayscale image and binary image for division, where M is the number of grid boundaries;

S702. Randomly select the (a, b)th grid (a∈[1,M], b∈[1,M]) on each rice grayscale image; randomly select the (a', b')th grid (a'∈[1,M], b'∈[1,M]) on each rice binary image; the number of frames required to cover each grid is calculated as follows:

Where:

n(a,b) represents the number of boxes required to cover the (a,b)th grid;

n(a',b') represents the number of boxes required to cover the (a',b')th grid;

P _max : represents the maximum value of all pixel values in the (a, b)th grid;

P _min : represents the minimum pixel value of all pixels in the (a, b)th grid;

P' _max : represents the maximum value of all pixel values in the (a', b')th grid;

P' _min : represents the minimum value of all pixel values in the (a', b')th grid;

S703, calculate the sum N of the number of frames covered by each grid:

S704, two fractal dimensions are calculated on the grayscale image and the binary image according to the following formulas:

Where:

D2 is the fractal dimension based on the grayscale image;

Sandbox is a fractal dimension based on binary images.

Furthermore, the calculation method of constructing multiple statistics for characterizing the texture characteristics of rice using the gray level co-occurrence matrix includes:

S801, dividing each rice binary image into L gray levels according to the gray value, and each pixel corresponds to one gray level;

S802, obtaining a gray level co-occurrence matrix p(x,y) according to the gray level of each pixel of each rice binary image;

x, y represent the grayscale levels of two pixels, x∈[0,L-1], y∈[0,L-1];

S803, extracting six texture features from the gray level co-occurrence matrix: contrast, difference, inverse difference matrix, entropy, correlation and angular second moment, respectively denoted as Con, DISL, IDM, ENT, Corr and ASM, and the calculation formula includes:

Where:

μ _x ,μ _y : the mean of the gray levels x and y of two different pixels;

σ _x ,σ _y : standard deviation of gray levels x and y of two different pixels.

A method for identifying rice growth stages based on machine learning to achieve the second objective of the present invention comprises the following steps:

S1. Obtaining multiple fractal dimensions of rice according to the original image of rice and constructing multiple statistics for characterizing the texture characteristics of rice using a gray-level symbiosis matrix; the fractal dimensions are data feature quantities for characterizing the phenotypic traits of rice;

S2. A mapping relationship between rice characteristics and rice growth stages is obtained according to a neural network model, thereby constructing a rice growth stage recognition model; the rice growth stage recognition model is used to recognize the growth stage of rice according to a rice image; the rice characteristics include multiple fractal dimensions of rice obtained from a rice feature generation module and multiple statistics for characterizing texture characteristics of rice constructed using a gray-level co-occurrence matrix.

Furthermore, the step S1 comprises the following steps:

Two fractal dimensions D1 and RFD based on the whole rice and the edge of rice leaves were calculated according to the rice image;

Two fractal dimensions D2 and Sandbox based on the rice as a whole and the circumscribed rectangle are calculated according to the rice image;

According to the correlation between adjacent pixels and the grayscale changes of the diagonal elements of the co-occurrence matrix, a grayscale co-occurrence matrix is obtained, and multiple statistics for characterizing the texture characteristics of rice are constructed according to the grayscale co-occurrence matrix;

Furthermore, the plurality of statistics include: contrast, difference, inverse difference matrix, entropy, correlation and angular second moment;

Furthermore, step S2 comprises the following steps:

S201, combining multiple phenotypic trait data sets of multiple rice samples with data sets containing four fractal dimensions D1, RFD, D2 and Sandbox and six gray-level co-occurrence matrices extracted from grayscale images and binary images to obtain an initial data set of rice;

S202, drawing a heat map of correlation analysis based on all the features of the initial rice data set; obtaining the relationship between each feature and the rice growth stage based on the heat map; retaining some features based on the relationship between the features and the rice growth stage;

S203, on the basis of recursive feature elimination, a random forest model is used to perform combined cross-validation on the remaining features, and the importance of different feature numbers for the accuracy of rice growth stage judgment is obtained by calculating the sum of their decision coefficients, and a plurality of feature combination numbers are retained according to the importance to obtain a modeling data set;

S204, performing data preprocessing and normalization processing on the modeling data set and performing training to obtain a trained rice growth stage recognition model.

A non-transitory computer-readable storage medium is provided to achieve the third objective of the present invention, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the rice growth stage identification method based on machine learning are implemented.

Beneficial effects:

The present invention finds that the introduction of fractal dimension and gray-level symbiosis matrix has positive significance for the discrimination of growth cycle. After adding fractal dimension variable, the accuracy of single machine learning model and optimal weighted integrated model is improved by about 2-8%, and after adding gray-level symbiosis matrix variable, the accuracy of single machine learning model and optimal weighted integrated model is improved by about 2-7%, which greatly improves the detection accuracy of rice growth stage.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic flow chart of an embodiment of the method of the present invention;

FIG2 is a ROC curve of six machine learning models in the embodiment;

FIG3 is a ROC curve of the optimized weighted integrated model in the embodiment;

FIG4 is a confusion matrix of an optimized weighted integration model in an embodiment;

FIG5 is a result of feature importance of input variables of support vector machine, decision tree, Adaboost and optimized weighted integrated model in the embodiment.

DETAILED DESCRIPTION

The following specific implementations are used to explain the technical solutions of the claims of the present invention so that those skilled in the art can understand the claims. The protection scope of the present invention is not limited to the following specific implementation structures. The technical solutions of the claims of the present invention made by those skilled in the art but different from the following specific implementations are also within the protection scope of the present invention.

FIG1 is a flow chart of an embodiment of the method of the present invention. The present invention proposes to construct a machine learning model and an integrated model based on the data of the rice automatic phenotyping platform to detect the rice growth stage. The specific steps are as follows:

(1) Data extraction

a. The rice automatic phenotyping platform measured 28 phenotypic traits of 1,094 rice samples of 521 rice varieties at three different growth stages (tillering stage, jointing stage, and heading stage), which served as the data set measured by the rice automatic phenotyping platform;

Table 1 Data characteristics of the rice automatic phenotyping platform

b. The rice automatic phenotyping platform uses a visible light industrial camera (AVT Stingray FG504) to take rice photos and obtain rice RGB images;

c. A hierarchical method based on kernel linear discriminant analysis and Gaussian process regression is used for the rice image to obtain a grayscale image and a binary image of the rice. Kernel linear discriminant analysis is used to distinguish the target leaf from its similar leaves in two steps: one is to roughly detect the entire leaf, and the other is to finely detect the edge of the leaf. The specific method is as follows:

The first step is to convert the collected rice color image into a grayscale image using RGB channel grayscale conversion. The red, green and blue colors of the image are superimposed and changed, and the value of each channel is taken as the grayscale value of the grayscale image to obtain the grayscale image of the three RGB channels. The grayscale histogram of the three grayscale images is drawn, and the best rice grayscale image is selected according to the distribution of the main information of the rice potted plants.

The second step is to use kernel linear discriminant analysis based on rough segmentation to model a supervised classifier to segment the target leaf from the similar leaf background. The target leaf area is cropped and collected from the grayscale image, and the cropped background leaf area is created. After the target leaf is segmented, the boundary of the rough segmented image is extracted to collect the area containing the target leaf boundary and other areas (such as leaf area and background). This method uses the Remove Image Background tool based on Python, Ruby, deep learning and other technologies, and uses powerful artificial intelligence AI algorithms to automatically identify foreground objects (i.e., target leaves) and backgrounds, thereby achieving large-scale batch image segmentation.

The third step is to perform edge detection on the image after segmentation by kernel linear discriminant analysis. Due to some misclassifications of kernel linear discriminant analysis, the edges detected in the boundary area may not be continuous. In order to eliminate these errors, the leaves need to be masked, that is, part of the area is covered with a selected object and image processing is performed on it. Then, the Sobel operator is combined with Gaussian filtering to perform image edge detection and denoising. The basic principle of Gaussian filtering is to perform weighted averaging of the image pixels in the sliding window and use the Gaussian function to calculate the weight coefficient. The weight value is determined according to the distance between the center pixel of the sliding window image and other pixels in the image in the window, and the weight coefficient will increase with the increase of distance, otherwise the smaller the distance, the lower the weight. The formula is:

Among them, σ ² represents the variance of the Gaussian function, p, q are the horizontal coordinates, and h(p, q) is the function of the Gaussian filter. Currently, the commonly used templates are 3*3 and 5*5, as shown below:

The comparison results show that the 3*3 template is the best. Then we change the color of rice in the grayscale image to white to get the binary image we need.

d. For the rice grayscale image and the binary image, two fractal dimensions based on the rice as a whole and the rice leaf edge are calculated using a random walk method, specifically including:

S501, randomly selecting a pixel point A and a pixel point A' from the grayscale image and the binary image of rice, and recording their coordinate values as (i, j) and (i', j') respectively;

S502, using random walk method to determine another pixel point B and pixel point B' in the grayscale image and binary image of rice respectively, and their coordinate values are recorded as (u, v) and (u', v') respectively; and R = || (i, j) - (u, v) || = || (i', j') - (u', v') ||, R is a randomly set value, R = 1, 2, ..., n;

S503, calculating the difference G between the grayscale values of the two pixels A and B and the difference G' between the grayscale values of the two pixels A' and B':

G＝I(i,j)-I(u,v)

G'＝I(i',j')-I(u',v')

Where:

I(i,j) and I(u,v) are the grayscale values of pixel A and pixel B respectively;

I(i',j') and I(u',v') are the grayscale values of pixel A' and pixel B' respectively;

S503, repeating steps S501 to S502 to obtain multiple difference values G and G' of each grayscale image and binary image, and calculating the average value E(G) of each grayscale image and the average value E(G') of each binary image according to the difference values G and G';

S504, calculate two fractal dimensions of each grayscale image and binary image according to the following formula:

Where:

D1 is the fractal dimension based on the grayscale image;

RFD is the fractal dimension based on binary images;

e. For the rice grayscale image and binary image, two fractal dimensions based on the rice as a whole and the circumscribed rectangle are calculated using the box counting method, specifically including:

S701, selecting an M×M grid on each rice grayscale image and binary image for division, where M is the number of grid boundaries;

S702. Randomly select the (a, b)th grid (a∈[1,M], b∈[1,M]) on each rice grayscale image; randomly select the (a', b')th grid (a'∈[1,M], b'∈[1,M]) on each rice binary image; the number of frames required to cover each grid is calculated as follows:

Where:

n(a,b) represents the number of boxes required to cover the (a,b)th grid;

n(a',b') represents the number of boxes required to cover the (a',b')th grid;

P _max : represents the maximum value of all pixel values in the (a, b)th grid;

P _min : represents the minimum pixel value of all pixels in the (a, b)th grid;

P' _max : represents the maximum value of all pixel values in the (a', b')th grid;

P' _min : represents the minimum value of all pixel values in the (a', b')th grid;

S703, calculate the sum N of the number of frames covered by each grid:

S704, two fractal dimensions are calculated on the grayscale image and the binary image according to the following formulas:

Where:

D2 is the fractal dimension based on the grayscale image;

Sandbox is a fractal dimension based on binary images.

f. For the rice binary image, the performance of the correlation between adjacent pixels and the grayscale change degree of the diagonal elements of the symbiosis matrix are calculated to obtain the statistics constructed by the grayscale symbiosis matrix as the texture feature of rice classification. The specific calculation method is as follows:

A gray-level co-occurrence matrix p(x,y) is obtained according to the gray level of each binary image of rice. The specific method of obtaining the gray-level co-occurrence matrix p(x,y) is as follows:

The image is divided into L levels according to the grayscale value. The grayscale value of each pixel corresponds to a grayscale level. Starting from any pixel with grayscale level x, the probability of leaving a fixed position relationship d = (dx, dy) on a straight line with a direction of θ to reach a pixel with grayscale level y. All estimated values can be expressed in the form of a matrix, namely the grayscale co-occurrence matrix; the grayscale co-occurrence matrix is represented by p(x, y) (x, y = 0, 1, 2...L-1), where L represents the grayscale of the image, x, y represent the grayscale of the pixel, respectively. d represents the spatial position relationship between two pixels; in this embodiment, d of the gray level co-occurrence matrix is set to 1, and the direction θ is set to ["0", "45", "90", "135"]. In this embodiment, the values of the four angles are calculated respectively, and then the average value of the four angles is taken; then six texture features are extracted from the gray level co-occurrence matrix: contrast, difference, inverse difference matrix, entropy, correlation and angular second moment (energy), which are respectively denoted as Con, DISL, IDM, ENT, Corr and ASM. The specific calculation formula is as follows:

Among them, μ _x , μ _y are the means of gray levels x and y of different pixels, and σ _x , σ _y are the standard deviations of gray levels x and y of different pixels.

g. The four fractal dimensions D1, RFD, D2, Sandbox and six gray-level symbiosis matrix data were subjected to K-S tests respectively to determine that the angles at different stages of rice are uniformly distributed and the autocorrelations are very similar, indicating that there is no local specificity in their growth that is affected by themselves, indicating that the fractal dimension of rice and the gray-level symbiosis matrix can be added to the data set as an important feature.

(2) Feature selection

a. combining 28 phenotypic trait data sets of 1094 rice samples measured by the rice automatic phenotyping platform with data sets containing four fractal dimensions and six gray-level co-occurrence matrices extracted from grayscale images and binary images to obtain an initial data set of rice;

b. Draw a heat map of correlation analysis based on all the features of the initial rice data set, observe the relationship between each feature and the rice growth stage, and remove the features that are not obviously related to the rice growth stage. In this embodiment, the features f1-f12 and features LD1-LD6 shown in Table 1 are removed;

c. Based on recursive feature elimination, the random forest model is used to perform combined cross-validation on the remaining features, and the importance of different feature numbers for the accuracy of judging the rice growth stage is finally obtained by calculating the sum of their decision coefficients. The best feature combination number is retained to obtain the modeling data set; the importance is determined according to actual needs. In this embodiment, the best feature combination number is a feature combination consisting of 31 features.

In this step, a recursive feature elimination method is used. The main idea is to use a random forest model to perform multiple training on the initial rice data set. According to the best feature combination number 31 obtained, the features with relatively low weights are removed according to the weight coefficients after each training, and the model is repeatedly constructed. Then, the best features are selected according to the coefficients, and the selected features are extracted. This process is repeated for the remaining features until all features are traversed, and the best feature combination is selected to obtain the modeling data set. The rice data set after feature selection using the recursive feature elimination method is the modeling data set.

(3) Machine Learning Modeling

a. Perform data preprocessing on the modeling data set, including missing value processing, outlier processing and checking whether the data is balanced;

The test data is balanced, that is, the sample ratio of data sets with different labels is likely to be unbalanced. Therefore, if the algorithm is directly used for training and classification, the training effect may be very poor; therefore, the data needs to be tested to make the sample ratio of the labeled data sets equal;

b. The modeling data set is normalized to eliminate the dimensional influence of rice characteristics. The specific calculation formula is as follows:

表示该特征的均值，s_i表示该特征的标准差Among them, _xi represents each feature,

represents the mean of the feature, and _si represents the standard deviation of the feature

c. The modeling data set is divided into a training set and a test set by random sampling at a ratio of 8:2. Each time, 80% of all samples are used for model training, and the remaining 20% are used as a test set to estimate performance indicators. The same random number seed is set for the same model to ensure the consistency of the model;

d. The established model is trained and validated using the 10-fold cross validation method, which gives the model the opportunity to be split into multiple training sets and test sets for training. The training set is randomly divided into 10 subsets of roughly equal size, one of which is used as a validation set to verify the accuracy of the model, and the remaining nine are used as training sets to train the model;

e. The established model is optimized using the Bayesian optimization method, which uses the Gaussian process, considers the previous parameter information, continuously updates the prior, and searches for the hyperparameter combination that optimizes the recognition effect of each model classifier;

f. The rice initial data set and the rice modeling data set after feature selection are used to train a machine learning model, and the classification labels are "tillering period", "jointing period" and "heading period", and a multi-classification machine learning model is established. The machine learning classification models used in this embodiment include: support vector machine (SVM), decision tree, random forest, Adaboost, stacked ensemble and optimized weighted ensemble learning classifier;

The specific calculation method of the stacked integration model is as follows:

The first layer model: use five algorithms including support vector machine, decision tree, random forest and AdaBoost for modeling, fitting and prediction;

Second-layer model: This classification model uses the prediction results of the first five models as features, the labels of the test set as labels, and the XGBClassifier algorithm as the base classifier for modeling, fitting, and prediction.

The specific calculation formula of the optimized weighted integration model is as follows:

是基础模型j对观察i的预测。Where _wj is the weight corresponding to the base model j (j = l, ..., k), n is the total number of samples, _yi is the true value of observation i,

is the prediction of base model j for observation i.

g. The performance of the established model can be evaluated by using the confusion matrix between the model prediction results and the actual results using the confusion_matrix function in Python's sklearn.metrics. The AUC value is calculated and the ROC curve is drawn to evaluate the classification performance of the model and visualize the model results.

h. Use the precision_score, recall_score, accuracy_score, f1_score, and cohen_kappa_score functions in sklearn.metrics in Python to obtain the model's evaluation indicators of precision, recall, accuracy, F1-score value, and kappa coefficient. The specific calculation formula is as follows:

Among them, TP _i is the true positive, indicating rice correctly classified as the i-th growth stage; FP _i is the false positive, indicating rice incorrectly classified as the i-th growth stage; FN _i is the false negative of class i, indicating rice incorrectly classified as the i-th growth stage in other growth stages. _{P i} and R _i are the precision and recall of class i, respectively, n is the number of classes (n = 3 in this study), P _w and R _w are the precision and recall of the weighted F1-score, respectively. p ₀ is the sum of the number of correctly classified samples in each class divided by the total number of samples, that is, the overall classification accuracy, and p _e is the sum of the "product of the actual and predicted numbers" corresponding to all classes;

i. The rice modeling data set after feature selection by correlation analysis and recursive feature elimination method is modeled using a variety of machine learning classifiers, and the classification results of the models before and after feature selection are compared by the evaluation index;

j. The four fractal dimensions D1, D2, RFD, Sandbox calculated from the rice grayscale image and the binary image and the six texture features Con, DISL, IDM, ENT, Corr and ASM obtained by using the gray-level symbiosis matrix are added separately to the rice modeling data set after feature selection to form a new data set, and then modeled using a variety of machine learning classifiers, and the classification results of the model with the four fractal dimensions added separately and the model with the six gray-level symbiosis matrices added separately evaluated by the evaluation index are compared; the specific implementation is as follows:

Embodiment 1:

(1) reading the entire rice data set and the rice modeling data set after the feature selection respectively, and generating two different data sets: data set 1 is the initial data set of rice, and data set 2 is the modeling data set of rice;

(2) Set corresponding classification labels according to the rice growth stage categories. The classification labels of the multi-classification model are tillering stage, jointing stage and heading stage. In each multi-classification model, there are 1094 rice sample data in total. Among them, the rice sample data in the tillering stage has a label set to "-1"; the rice sample data in the jointing stage has a label set to "0"; and the rice sample data in the heading stage has a label set to "1";

(3) Modeling the two data sets separately, using support vector machine (SVM), decision tree, random forest, Adaboost, stacking ensemble and optimized weighted ensemble of multiple machine learning classifiers, dividing them into training set and test set at a ratio of 8:2, and setting the same random number seed for the same model to ensure the consistency of the model. The 10-fold cross-validation method was used for training and verification, and the Bayesian optimization method was used to find the hyperparameter combination that optimizes the recognition effect of each model classifier;

(4) The accuracy indicators of the six machine learning models used in this embodiment on the entire rice data set are shown in Table 2, and the accuracy indicators of the rice data set after feature selection are shown in Table 3; it can be seen that the evaluation indicators of the six machine learning classifiers are improved after feature selection, and the single machine learning model with the best classification effect is the Adaboost model, with an accuracy rate and F1 score of 93.15% and 0.93 respectively, and a Kappa coefficient of 0.91, which is about 0.5% higher than the model without feature selection. Compared with the basic model, the integrated model provides better performance, among which the optimized weighted integration is the most accurate model, which is better than the stacking model, with an accuracy rate and F1-score of 94.06% and 0.94, and a Kappa coefficient of 0.92, which is about 0.6% higher than the model without feature selection. In general, the performance of the machine learning classifier is better after feature selection;

Table 2 Accuracy indicators of the six models of dataset 1

Table 3 Accuracy indicators of six models of dataset 2

(5) The ROC curves of the six machine learning models are shown in Figure 2. The ROC curve uses the false positive rate as the horizontal axis and the true positive rate as its vertical axis. For a data sample of a classification task, the ROC curve needs to calculate the probability that the sample belongs to the correct category. In order to convert the probability into the corresponding category, we need to select a threshold. The ROC curve is obtained by changing the threshold and shows the performance of each classification model. Due to the high accuracy of the model, all varieties are closer to the value representing the true positive rate. Compared with the basic model, the AUC value of the integrated model is higher, reaching 0.98, and the AUC value based on the random forest model and Adaboost model in the single machine learning model also reached about 0.97, indicating that the performance of each classification model is good;

(6) The ROC curve and confusion matrix of the optimized weighted ensemble model are shown in Figures 3 and 4, respectively. Among the three growth stages of all models, the tillering stage is the most accurate and less likely to be confused, followed by the jointing stage and finally the heading stage. The reason may be that the internodes of the rice stems extend upward rapidly during the jointing stage, while during the heading stage, the top leaves of the rice extend as the stems elongate, and the overall morphology of the rice does not differ much.

Embodiment 2:

(1) respectively reading the rice modeling datasets after the feature selection, and generating three different datasets 3, 4 and 5: wherein the rice features in dataset 3 do not include fractal dimension and gray-level symbiosis matrix; the rice features in dataset 4 do not include gray-level symbiosis matrix, but include fractal dimension; the rice features in dataset 5 do not include fractal dimension, but include gray-level symbiosis matrix;

(2) Setting corresponding classification labels according to the rice growth stage categories. The classification labels of the multi-classification models are respectively tillering stage, jointing stage and heading stage. In each multi-classification model, there are 1094 rice sample data in this embodiment, among which the rice sample data in the tillering stage has a label set to "-1"; the rice sample data in the jointing stage has a label set to "0"; and the rice sample data in the heading stage has a label set to "1";

(3) Modeling the three data sets separately, using six machine learning classifiers including support vector machine (SVM), decision tree, random forest, Adaboost, stacked ensemble, and optimized weighted ensemble, and dividing them into training set and test set at a ratio of 8:2, and setting the same random number seed for the same model to ensure the consistency of the model. The 10-fold cross-validation method was used for training and validation, and the Bayesian optimization method was used to find the hyperparameter combination that optimizes the recognition effect of each model classifier;

(4) The accuracy, F1 score and kappa coefficient of the six machine learning models in data sets 3, 4 and 5 are shown in Tables 4, 5 and 6. From the table, it can be found that in different models, whether the fractal dimension or gray-level co-occurrence matrix is added or not, the discrimination effect of the classifier is improved after the fractal dimension or gray-level co-occurrence matrix features are introduced. In general, after the gray-level co-occurrence matrix features are introduced, the evaluation indicators of each model are improved more, and the accuracy is improved by about 2-7%. After the fractal dimension features are introduced, the decision tree model and the optimized weighted ensemble model are improved more than the model after the gray-level co-occurrence matrix features are introduced, and the accuracy is improved by about 3% and 8% respectively. In general, the single machine learning model is improved more, and the introduction of fractal dimension and gray-level co-occurrence matrix is of positive significance for the discrimination of growth cycle.

Table 4 Accuracy indicators of six models of datasets 3, 4, and 5

Table 5 F1 score indicators of six models of datasets 3, 4, and 5

Table 6 Kappa coefficient indicators of six models of data sets 3, 4, and 5

Embodiment three:

(1) Read the rice modeling datasets after the feature selection respectively, and generate a dataset 6, which is the same as the dataset 2 in Example 1.

(2) Set corresponding classification labels according to the rice growth stage categories. The classification labels of the multi-classification model are tillering stage, jointing stage and heading stage. In each multi-classification model, there are 1094 rice sample data in total. Among them, the rice sample data in the tillering stage has a label set to "-1"; the rice sample data in the jointing stage has a label set to "0"; the rice sample data in the heading stage has a label set to "1".

(3) Modeling the dataset 6, using six machine learning classifiers including support vector machine (SVM), decision tree, random forest, Adaboost, stacked ensemble, and optimized weighted ensemble, respectively, using the random forest method in the feature selection tree model to evaluate feature importance, calculating the importance of each feature for the rice growth stage category, and ranking the feature contributions to find the more influential feature variables to optimize the model

(4) The feature importance results of the first 10 input variables are shown in Table 7. The input feature importance was calculated according to the six models to find the most influential independent variables to optimize the model. Table 7 shows the weighted first 10 feature importance results of all 6 models. The parameter SA has a weight of 0.1359 and is the most important feature. The first 10 input variables include two texture feature variables extracted from images and two fractal dimension variables. Therefore, other auxiliary data derived from images should be added because they may lead to higher estimation accuracy. In addition, parameters such as rice relative frequency and structural parameters that reflect the compactness of plants seem to be less important;

Table 7 Feature importance coefficients and rankings of six models

(5) The feature importance results of the input variables of the support vector machine, decision tree, Adaboost and optimized weighted ensemble model are shown in Figure 5. Based on the proposed feature importance method, the feature importance of morphological parameters in the decision tree model accounts for a larger proportion, while the feature importance of texture parameters in the other four models accounts for a larger proportion. Therefore, it can be inferred that texture parameters such as fractal dimension and gray-level co-occurrence matrix are very important for the detection of rice growth stages. Although the feature importance of different models is not the same, in general, the feature importance of RFD, D2, Sandbox, ENT, ASM and G_g in the four models accounts for a relatively large proportion.

k. According to the machine learning classifier model, the random forest method in the feature selection tree model is used to evaluate the feature importance, calculate the importance of each feature to the rice growth stage category, and sort the feature contributions to find the more influential feature variables to optimize the model.

It can be seen from the embodiments that using the method for detecting the growth stage of rice according to the present invention, the optimal single machine learning model is the Adaboost model, whose accuracy and F1 score reach 93.15% and 0.93 respectively, and the Kappa coefficient is 0.91; using the method for detecting the growth stage of rice according to the present invention, the integrated model, especially the optimized weighted integration after feature selection, performs the best classification, and compared with the existing single machine learning model, the accuracy is improved by about 1.5%, the accuracy and F1-score reach 94.06% and 0.94 respectively, and the kappa coefficient reaches 0.92.

It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

This embodiment also provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the steps of the method of the present invention are implemented, which will not be repeated here.

The computer-readable storage medium may be the data transmission device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., provided on the computer device.

Furthermore, the computer-readable storage medium may include both an internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data to be output or has been output.

Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

The present application is described with reference to the flowchart and/or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and/or box in the flowchart and/or block diagram, and the combination of the process and/or box in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and/or one box or multiple boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

Claims (10)

1. The system for identifying the rice growth stage based on machine learning is characterized by comprising a rice characteristic generation module and a rice growth stage identification model construction module;

the rice characteristic generation module is used for obtaining a plurality of fractal dimensions of rice according to the rice image and constructing a plurality of statistics used for representing texture characteristics of the rice by utilizing the gray level co-occurrence matrix;

The rice growth stage identification model construction module is used for obtaining the mapping relation between the characteristics of rice and the rice growth stage according to the neural network model so as to construct a rice growth stage identification model; the rice growth stage identification model is used for identifying the growth stage of the rice according to the rice image; the characteristics of the rice comprise a plurality of fractal dimensions of the rice obtained from a rice characteristic generation module and statistics constructed by using a gray level co-occurrence matrix.

2. The machine learning based rice growth stage identification system of claim 1, wherein the rice feature generation module comprises a first fractal dimension generation module, a second fractal dimension generation module, and a textural feature acquisition module;

the first fractal dimension generation module is used for calculating two fractal dimensions D1 and RFD based on the whole rice and the edge of the rice leaf according to the rice image;

the second fractal dimension generation module is used for calculating two fractal dimensions D2 and Sandbox based on the whole rice and the circumscribed rectangle according to the rice image;

the texture feature acquisition module is used for obtaining a gray level co-occurrence matrix according to the correlation of adjacent pixels and the gray level change of diagonal elements of the co-occurrence matrix, and constructing a plurality of statistics used for representing the texture features of rice according to the gray level co-occurrence matrix.

3. The machine learning based rice growth stage identification system of claim 2, wherein the statistics comprise: contrast, variability, contrast score matrix, entropy, correlation, and angular second moment.

4. A machine learning based rice growth stage identification system as claimed in claim 2 or 3 wherein the fractal dimension generation module further comprises a grey-scale map generation module for converting the rice image into a grey-scale map for obtaining the fractal dimensions D1, RFD, D2 and Sandbox of the rice prior to obtaining the fractal dimension.

5. The machine learning-based rice growth stage identification system as claimed in claim 4, wherein the fractal dimension generation module further comprises a binary image generation module, wherein the binary image generation module is used for performing edge detection and denoising on the image by adopting a Sobel operator and a gaussian filtering method on the gray image of the rice image to obtain a binary image of the rice image, and the binary image is used for obtaining fractal dimensions D1, RFD, D2 and Sandbox of the rice.

6. The machine learning based rice growth stage identification system of claim 5, wherein the fractal dimension D1 and RFD calculation method comprises:

S501, randomly selecting a pixel point A from a gray level image of rice, and marking coordinate values as (i, j); randomly selecting a pixel point A 'from the binary image of the rice, wherein the coordinate value of the pixel point A' is marked as (i ', j');

s502, determining another pixel point B and a pixel point B 'in the gray level image and the binary image of the rice by adopting a random walk method, wherein coordinate values of the pixel point B and the pixel point B' are respectively marked as (u, v) and (u ', v'); and r= | (i, j) - (u, v) |= | (i ', j') - (u ', v')|, R is a randomly set value, r=1, 2, n;

s503, calculating the difference value G of the gray values of the two pixel points A and B and the difference value G ' of the gray values of the two pixel points A ' and B ':

G＝I(i,j)-I(u,v)

G'＝I(i',j')-I(u',v')

wherein:

i (I, j) and I (u, v) are gray values of the pixel point A and the pixel point B respectively;

i (I ', j'), I (u ', v') are gray values of the pixel point A 'and the pixel point B', respectively;

s503, repeating the steps S501-S502 to obtain a plurality of difference values G and G ' of each gray level image and each binary image, and calculating an average value E (G) of each gray level image and an average value E (G ') of each binary image according to the difference values G and G ';

s504, calculating to obtain two fractal dimensions of each gray level image and binary image according to the following formula:

wherein:

d1 is a fractal dimension based on a gray image;

RFD is fractal dimension based on binary image;

c is a set constant.

7. The machine learning based rice growth stage identification system of claim 5, wherein the fractal dimension D2 and Sandbox calculation method comprises:

s701, respectively selecting an M multiplied by M grid on each rice gray level image and each binary image for division, wherein M is the number of grid boundaries;

s702, randomly selecting (a, b) grids (a epsilon [1, M ], b epsilon [1, M ]) on each rice gray level image; randomly selecting the (a ', b') th grid (a 'E [1, M ], b' E [1, M ]) on each rice binary image; the number of frames required to cover each grid is calculated as follows:

wherein:

n (a, b) represents the number of frames required to cover the (a, b) th grid;

n (a ', b') represents the number of frames required to cover the (a ', b') th grid;

P _max : representing the maximum value of the pixel values of all the pixel points in the (a, b) th grid;

P _min : representing the minimum of pixel values of all pixel points in the (a, b) th grid;

P' _max : representing the maximum value of the pixel values of all the pixel points in the (a ', b') th grid;

P' _min : representing the minimum value of the pixel values of all the pixel points in the (a ', b') th grid;

s703, calculating the sum N of the number of covered frames of each grid as follows:

S704, respectively calculating two fractal dimensions in the gray level image and the binary image according to the following formulas:

wherein:

d2 is a fractal dimension based on a gray image;

sandbox is a fractal dimension based on binary images.

8. The machine learning based rice growth stage identification system of claim 5 wherein the method of constructing a plurality of statistics for characterizing a texture feature of rice using a gray level co-occurrence matrix comprises:

s801, dividing a binary image of each rice into L gray levels according to gray values, wherein each pixel corresponds to one gray level;

s802, obtaining a gray level co-occurrence matrix p (x, y) according to the gray level of each pixel of the binary image of each rice;

x and y respectively represent the gray level of two pixel points, x is 0 and L-1, and y is 0 and L-1;

s803, extracting 6 texture features from the gray level co-occurrence matrix: contrast, variability, contrast score matrix, entropy, correlation, and angular second moment, denoted Con, DISL, IDM, ENT, corr and ASM, respectively, the calculation formula includes:

wherein:

μ _x ,μ _y : the average value of gray levels x and y of two different pixel points;

σ _x ,σ _y : standard deviation of gray levels x and y of two different pixels.

9. A method for identifying a growth stage of rice based on machine learning based on the system of claim 1, comprising the steps of:

S1, obtaining a plurality of fractal dimensions of rice according to an original image of the rice and constructing a plurality of statistics used for representing texture features of the rice by using a gray level co-occurrence matrix; the fractal dimension is a data characteristic quantity for representing the phenotypic character of rice;

s2, obtaining a mapping relation between the characteristics of the rice and the growth stage of the rice according to the neural network model, so as to construct a rice growth stage identification model; the rice growth stage identification model is used for identifying the growth stage of the rice according to the rice image; the characteristics of the rice comprise a plurality of fractal dimensions of the rice obtained from a rice characteristic generation module and a plurality of statistics for representing the texture characteristics of the rice by utilizing a gray level co-occurrence matrix.

10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the machine learning based rice growth phase identification method of claim 9.

Images