[go: up one dir, main page]

CN120408098A - A soil microbial community hyperspectral remote sensing monitoring method and storage medium - Google Patents

A soil microbial community hyperspectral remote sensing monitoring method and storage medium

Info

Publication number
CN120408098A
CN120408098A CN202510884714.1A CN202510884714A CN120408098A CN 120408098 A CN120408098 A CN 120408098A CN 202510884714 A CN202510884714 A CN 202510884714A CN 120408098 A CN120408098 A CN 120408098A
Authority
CN
China
Prior art keywords
data
hyperspectral
soil
physical
microorganism
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202510884714.1A
Other languages
Chinese (zh)
Other versions
CN120408098B (en
Inventor
聂文杰
聂炜地
韩冬洁
李苗
潘禹杉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian University of Science and Technology
Original Assignee
Xian University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian University of Science and Technology filed Critical Xian University of Science and Technology
Priority to CN202510884714.1A priority Critical patent/CN120408098B/en
Publication of CN120408098A publication Critical patent/CN120408098A/en
Application granted granted Critical
Publication of CN120408098B publication Critical patent/CN120408098B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217Validation; Performance evaluation; Active pattern learning techniques
    • G06F18/2193Validation; Performance evaluation; Active pattern learning techniques based on specific statistical tests
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/042Knowledge-based neural networks; Logical representations of neural networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0475Generative networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/0895Weakly supervised learning, e.g. semi-supervised or self-supervised learning
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/094Adversarial learning
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/096Transfer learning
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computing arrangements using knowledge-based models
    • G06N5/04Inference or reasoning models
    • G06N5/041Abduction
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16CCOMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
    • G16C20/00Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
    • G16C20/10Analysis or design of chemical reactions, syntheses or processes
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16CCOMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
    • G16C20/00Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
    • G16C20/20Identification of molecular entities, parts thereof or of chemical compositions
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16CCOMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
    • G16C20/00Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
    • G16C20/70Machine learning, data mining or chemometrics
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Chemical & Material Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Crystallography & Structural Chemistry (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Analytical Chemistry (AREA)
  • Databases & Information Systems (AREA)
  • Chemical Kinetics & Catalysis (AREA)
  • Medical Informatics (AREA)
  • Pathology (AREA)
  • Immunology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Biochemistry (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention relates to the technical field of soil microorganism monitoring and discloses a soil microorganism community hyperspectral remote sensing monitoring method and a storage medium, wherein the soil microorganism community hyperspectral remote sensing monitoring method comprises the steps of constructing and generating pseudo hyperspectral data which accords with physical constraint and is generated by an antagonism network based on data enhancement and semi-supervised learning of the antagonism network, predicting soil microorganism community parameters, constructing a physical information neural network, introducing the constraint of a soil hydrothermal migration physical equation, deducing the microorganism distribution and the hydrothermal state of different depths of a soil section, embedding a nitrogen circulation dynamics equation in the physical information neural network, expanding network output variables and physical constraint loss, and optimizing model parameters and process parameters.

Description

Soil microbial community hyperspectral remote sensing monitoring method and storage medium
Technical Field
The invention relates to the technical field of soil microorganism monitoring, in particular to a hyperspectral remote sensing monitoring method for soil microorganism communities and a storage medium.
Background
With the development of precise agriculture and soil ecosystem protection, soil microbial community monitoring is becoming a key link increasingly, and has important significance for crop yield, soil quality and environmental protection. Soil microorganisms participate in a carbon-nitrogen circulation key process, and play an irreplaceable role in nutrient conversion, environmental pollutant degradation and soil structure improvement.
Currently, soil microbial community monitoring is mainly dependent on laboratory analysis methods such as DNA extraction and sequencing, culture counting, functional gene analysis, and the like. Although the methods have high accuracy, the methods have the defects of large sampling destructiveness, high analysis cost, long time consumption, limited space coverage and the like, and the requirements of large-scale, rapid and nondestructive monitoring are difficult to meet. The hyperspectral remote sensing technology provides possibility for rapid, nondestructive and large-scale monitoring of soil characteristics through acquiring the reflection spectrum information of the ground surface target, and advances in monitoring of physical and chemical characteristics of soil organic matters, textures and the like. However, the application of hyperspectral techniques to soil microflora monitoring still faces a number of challenges.
The existing hyperspectral monitoring technology of the soil microbial community mainly has the following problems that a marked sample is difficult to obtain, training data with a tag are sparse due to high laboratory analysis cost, deep learning model performance is limited, profile information is difficult to obtain, the existing hyperspectral technology mainly aims at surface characteristics and is difficult to directly obtain microbial information of different depths of a soil profile, physical process and data driving are not effectively combined, physical process constraint is ignored due to the fact that the physical process constraint is completely depended on, or the physical process is completely depended on a mechanism model but is difficult to parameterize, nitrogen circulation key process parameterization is difficult, nitrogen circulation process parameters driven by microorganisms are difficult to quickly obtain in a large range, result uncertainty assessment is lost, and reliability of decision support is lowered.
Therefore, there is a need for a hyperspectral remote sensing monitoring method of soil microbial communities, which can overcome the above technical problems, and realize accurate prediction under the condition of data sparseness, information inference from the earth surface to the section, fusion of physical and data driving, and accurate inversion of process parameters.
Disclosure of Invention
The invention provides a hyperspectral remote sensing monitoring method and a storage medium for a soil microbial community, which solve the technical problems that a marked sample is difficult to obtain, soil profile information is difficult to obtain and nitrogen cycle parameterization is difficult in the related technology.
The invention provides a hyperspectral remote sensing monitoring method for a soil microbial community, which comprises the following steps:
Based on data enhancement and semi-supervised learning of a generated countermeasure network, hyperspectral remote sensing data and a hyperspectral microorganism data pair with a label are obtained, a pseudo hyperspectral microorganism data pair generated by the generated countermeasure network is constructed, a semi-supervised learning framework is constructed, and a microbial community parameter prediction model is output;
Based on section inference of the physical information neural network, according to the output of the prediction model, combining hyperspectral features, meteorological data and space coordinates, constructing the physical information neural network, introducing soil hydrothermal migration equation constraint, and inferring microorganism distribution and hydrothermal states of different depths;
Embedding a nitrogen circulation dynamics equation in the physical information neural network, and outputting a nitrogen circulation key process parameter by taking profile microorganism distribution as a driving force;
and combining the nitrogen cycle key process parameters and profile microorganism distribution with the observation data by utilizing a data assimilation method, inputting hyperspectral data of a target area, generating monitoring results of soil microorganism community spatial distribution, profile microorganism abundance distribution and nitrogen cycle parameter distribution, and carrying out uncertainty evaluation.
Further, the step of generating pseudo hyperspectral data conforming to physical constraints against the network comprises:
constructing a generation countermeasure network comprising a generator and a discriminator, wherein the generator receives a random noise vector and a microorganism parameter condition vector as inputs to generate hyperspectral data conforming to real distribution;
Training a generator and a discriminator by adopting an alternative optimization strategy until the generator can generate pseudo hyperspectral data conforming to physical constraint;
Generating corresponding hyperspectral data by inputting different microorganism parameter condition vectors to form a large number of pseudo-mark samples;
And constructing a semi-supervised learning framework, and performing model training by using the hyperspectral data in real distribution, the pseudo hyperspectral data conforming to physical constraints and the pseudo-mark samples.
Further, the constructed semi-supervised learning framework comprises a student model and a teacher model, the student model updates parameters through supervised training, the teacher model parameters are exponential moving average values of the student model parameters, and the loss functions of the semi-supervised learning framework comprise supervision loss, consistency loss and regularization loss.
Further, the construction of the physical information neural network includes:
constructing a neural network comprising a forward propagation portion and a physical constraint portion;
to ensure that the output of the neural network meets the physical rule, introducing the constraint of a physical equation in the soil hydrothermal migration process;
Constructing a total loss function comprising data fitting loss and physical constraint loss;
And generating pseudo section data meeting physical constraint by using the generated countermeasure network, and enhancing the training effect of the physical information neural network.
Further, the physical equation constraints include a moisture migration equation and a heat conduction equation.
Further, the step of embedding the nitrogen circulation dynamics equation in the physical information neural network comprises the following steps:
Adding a description equation of a nitrogen circulation key process in a physical information neural network framework;
expanding output variables of a physical information neural network, and increasing prediction of ammonium nitrogen and nitrate nitrogen concentration and output of the nitrification rate and denitrification rate along with depth change;
Expanding physical constraint loss of a physical information neural network and increasing residual terms of a nitrogen circulation equation;
And the constraint of the parameter value range is increased, so that the physical rationality of the inverted parameters is ensured.
Further, the description equations of the key nitrogen cycle process comprise a nitrification process equation and a denitrification process equation, and describe the dynamic change process of ammonium nitrogen and nitrate nitrogen and the influence of microorganism abundance, water content, temperature and organic carbon on the nitrification and denitrification processes.
Further, the step of utilizing the data assimilation method comprises updating the parameter estimation upon arrival of the observed data by constructing a plurality of set members of the model parameters using a set Kalman filtering method.
Further, the uncertainty evaluation includes:
calculating a prediction interval of each monitoring result by using a set prediction method;
Generating an uncertainty distribution map, and identifying a region with lower prediction reliability;
providing a confidence assessment of the predicted result, and providing a reliability reference for decision-making application;
and outputting a quality control report of the monitoring result, wherein the quality control report comprises model performance indexes and applicability evaluation.
The invention provides a storage medium, which comprises a memory and one or more processors, wherein executable codes are stored in the memory, and the one or more processors are used for realizing the hyperspectral remote sensing monitoring method of the soil microbial community when executing the executable codes.
The invention has the beneficial effects that the prediction accuracy of the microbial community index can be maintained under the condition of reduced marked sample quantity by using GAN data enhancement and semi-supervised learning technology, even the prediction of certain functional groups is slightly improved, the dependence on site sampling and laboratory analysis is reduced, and the monitoring cost and time are reduced;
Based on a physical information neural network, the microbial distribution and the hydrothermal state of the soil profile can be deduced from the hyperspectral remote sensing data of the earth surface, the average relative error of the profile microbial abundance prediction is reduced, the average relative error of the profile hydrothermal state prediction is reduced, the 'deep-looking' capability from the earth surface to the profile is realized, and the workload of traditional drilling and sampling is reduced;
inversion of key process parameters of nitrogen circulation driven by soil microorganisms is realized, the parameters are critical to a nitrogen circulation model but are difficult to directly measure by a traditional method, and core parameter support is provided for accurate nitrogen fertilizer management and environmental impact evaluation, so that large-range nitrogen conversion monitoring is possible;
by fusing physical process constraint and data driving technology, not only can accurate prediction result be provided, but also prediction uncertainty can be quantified, reliability assessment is provided for decision making, and the application value of the monitoring result in accurate agriculture and environmental management is enhanced.
Drawings
FIG. 1 is a flow chart of a hyperspectral remote sensing monitoring method of a soil microbial community in the invention;
FIG. 2 is a flow chart of step1 of the present invention;
FIG. 3 is a flow chart of step2 of the present invention;
FIG. 4 is a flow chart of step3 of the present invention;
FIG. 5 is a flow chart of step 4 of the present invention;
FIG. 6 is a graph of comparative combinations of accuracy and data efficiency of the different methods of the present invention in microbial community prediction;
FIG. 7 is a thermodynamic diagram of the spatial distribution and uncertainty estimation of critical parameters of nitrogen cycling in accordance with the present invention;
FIG. 8 is a line graph of parameter predictive performance versus validation for different depths of a soil profile in accordance with the present invention.
Detailed Description
The subject matter described herein will now be discussed with reference to example embodiments. It is to be understood that these embodiments are merely discussed so that those skilled in the art may better understand and implement the subject matter described herein and that changes may be made in the function and arrangement of the elements discussed without departing from the scope of the disclosure herein. Various examples may omit, replace, or add various procedures or components as desired. In addition, features described in some examples may be combined in other examples as well.
In at least one embodiment of the present invention, a method for hyperspectral remote sensing monitoring of a soil microbial community is disclosed, as shown in fig. 1 to 5, comprising:
Step 1, based on data enhancement and semi-supervised learning of a generated countermeasure network, hyperspectral remote sensing data and a hyperspectral microorganism data pair with a label are obtained, a pseudo hyperspectral microorganism data pair generated by the generated countermeasure network is constructed, a semi-supervised learning framework is constructed, and a microbial community parameter prediction model is output;
the method solves the problem of sparse labeling samples in hyperspectral remote sensing monitoring by generating an countermeasure network (GENERATIVEADVERSARIALNETS, GAN), and improves the generalization capability of the deep learning model. It should be understood that the following is specifically performed:
Step 1.1, data preparation;
A small amount of tagged hyperspectral microbial data pairs (tagged data) and a large amount of untagged hyperspectral data (untagged data) are acquired. The hyperspectral features in the marker data include surface reflectance curves (typically covering the 400 to 2500nm band range), microbial tags including community composition (e.g., gate class chemical abundance), diversity index (e.g., shannon index), or specific functional group abundance.
Step 1.2, constructing a GAN model;
A GAN model suitable for the characteristics of hyperspectral data is constructed, and the GAN model comprises a generator and a discriminator. A generator Receiving random noise vectorsAnd a condition vector(Microbial parameters) as input to generate hyperspectral data conforming to the true distributionThe real hyperspectral data is distinguished from the pseudo hyperspectral data generated.
According to an embodiment of the present application, the concrete implementation of the GAN model includes the following structures:
the generator adopts the structure of an encoder and a decoder, and the encoder uses random noise vectors And a condition vectorEncoding as latent features, the decoder reconstructs the latent features into hyperspectral data:
The encoder comprises 3 convolution layers, each layer is followed by batch normalization and LeakyReLU activation functions, the decoder comprises 3 transposed convolution layers for up-sampling, the last layer uses the Tanh activation function to ensure that the output value range is between [ -1,1], and then maps to the [0,1] interval as reflectivity value through linear transformation;
The arbiter adopts a multi-layer convolutional neural network structure, and comprises 4 convolutional layers, each layer is followed by batch normalization and LeakyReLU activation functions, and the last layer outputs a single scalar which represents the probability that the input data is a real sample.
Alternatively, in some embodiments, the generator may employ an attention-based encoder-decoder architecture, where a self-attention layer is introduced between the encoder and decoder, enhancing long-range dependency capture capability between features, particularly for complex correlation modeling between different bands in hyperspectral data.
In other embodiments, the arbiter may employ an objective function based on Wasserstein distance, ensuring training stability through a gradient penalty term, the loss function being modified to:
;
Wherein the method comprises the steps of A loss function (DiscriminatorLoss) representing the arbiter; Representing distribution of real data Sampled (1)Distinguishing deviceFor a pair ofIs used to determine the desired value of the output of (c),In the case of a true hyperspectral sample,Probability distribution for real hyperspectral data; Representing noise distribution Sampled (1)Warp generatorGenerating pseudo-samplesPost-discriminatorThe desired value of the output is set,As a vector of random noise,In order for the noise to be distributed,To generate hyperspectral data; the weight coefficient of the gradient penalty term is represented, and the contribution of the gradient penalty term in the loss function is controlled; representing a uniform interpolation distribution between a real sample and a generated sample Sampled (1)Distinguishing deviceThe expectation of the square of the difference between the L2 norm of the pair gradient and 1,Representation ofFor a pair ofIs used for the gradient of (a),Represents an L2 norm; representation discriminator Input toIs provided.
;
Wherein the method comprises the steps ofA loss function (GeneratorLoss) representing the generator; Representing noise distribution Sampled (1)A generatorGenerating pseudo-samplesPost-discriminatorThe output expected value; Representing the random noise vector(s), Is the noise distribution; representation discriminator Pair generatorAnd outputting the generated sample.
The loss functions of the generator G and the arbiter D are respectively:
;
Wherein the method comprises the steps of Representing generator loss; Representing the slave noise distribution Sampled random noise vector,Is the noise distribution; Representing the distribution of parameters from microorganisms Condition vector of sampling,Probability distribution of microbial parameters; representation discriminator For generating dataConditions and conditionsAn output of (2); Representation generator To be used forAndPseudo hyperspectral data generated for input; representing a desired operator;
;
Wherein the method comprises the steps of Representing a discriminator loss; representing the distribution from real hyperspectral data Sampled (1),Probability distribution for real hyperspectral data; Representing the distribution of parameters from microorganisms Condition vector of sampling;Representation discriminatorFor real samplesConditions and conditionsLog loss of the output of (a); representing input data by a discriminator Under the condition thatThe probability estimation of the real sample follows; Representing the slave noise distribution Sampled random noise vector;Representation discriminatorFor generating samplesConditions and conditionsLog loss of the output of (a); representing the desired operator.
In the application scene of farmland soil microorganism monitoring, the GAN model can be used for generating hyperspectral data under the condition of abundance of different nitrogen circulation functional bacteria. For example, in the application of accurate fertilization decision support, corresponding hyperspectral data are generated by inputting different nitrifying bacteria abundance conditions, the problem of sparse field sampling points is solved, and data support is provided for subsequent fertilization partitions.
Step 1.3, GAN training and data generation;
Training a generator using alternating optimization strategies Distinguishing deviceUntil the generator is able to produce pseudo-hyperspectral data that meets the physical constraints. After training, different microorganism parameter condition vectors are inputCorresponding hyperspectral data are generated, forming a plurality of pseudo-marker samples. The newly generated dummy samples need to satisfy physical constraints at the same time, e.g. reflectance values in the range ofBetween, the spectral curves are continuously smoothed, etc.
Step 1.4, constructing a semi-supervised learning model;
A semi-supervised learning framework is constructed based on the true token samples, the generated pseudo token samples, and a large number of unlabeled exemplars. According to one embodiment of the application, the semi-supervised learning framework employs an average teacher (MEANTEACHER) model structure, including a student model and a teacher model. The student model updates parameters by supervised training, and the teacher model parameters are exponential moving averages of the student model parameters.
The specific implementation of the semi-supervised learning framework comprises the following steps:
the student model and the teacher model adopt the same network structure and consist of four convolution blocks and two full-connection layers, wherein each convolution block comprises a convolution layer, a batch normalization layer and a ReLU activation function;
For tagged data (including true tag samples and high quality false tag samples), the mean square error (MeanSquaredError, MSE) between the predicted value and the true tag is calculated as a supervised loss ;
For label-free data, different data enhancement is applied to the same input, prediction is carried out through a student model and a teacher model, and the mean square error of the prediction results of the student model and the teacher model is calculated to be used as consistency lossRegularization lossL2 regularization is adopted to prevent the model from being overfitted.
Alternatively, in some embodiments, the semi-supervised learning framework may employ a Pseudo-Labeling (Pseudo-Labeling) method, where labeled data is first used to train an initial model, then the model is used to predict unlabeled data to generate Pseudo-labels, and finally the labeled data and the Pseudo-labeled data are used together for model training. To control the quality of the pseudo tag, a confidence threshold may be set, using only high confidence pseudo tags.
In other embodiments, the semi-supervised learning framework may employ a hybrid consistency regularization (MixMatch) approach to generate more diverse training samples by blending the enhanced versions of tagged and untagged data. Specifically, multiple enhancement versions are generated for each unlabeled exemplar, the prediction results of these enhancement versions are averaged as pseudo labels, and then the labeled and unlabeled exemplars are mixed to generate a mixed exemplar for use in calculating the loss function.
The framework contains a three-part loss function, supervised lossConsistency loss (calculated using true mark samples and false mark samples)(Computation using unlabeled data) and regularization loss. The total loss function is:
;
Wherein the method comprises the steps of Representing a total loss function; representing supervision loss, and measuring the prediction error of the model on the labeled data; Representing consistency loss, and measuring consistency of a student model and a teacher model to a label-free data prediction result; representing regularization loss, typically L2 regularization, preventing overfitting; Representing a consistency loss Controlling its contribution in the total loss; representing regularization loss And controls its contribution in the total loss.
In a regional soil monitoring scenario, the semi-supervised learning framework may be used for regional scale soil microbiota prediction. The application example comprises that only a small number of sampling points with laboratory measurement results are needed when agricultural large-area monitoring is carried out, and a large number of positions with hyperspectral data are combined, so that a microorganism function diagram of the whole area can be generated through the framework, and decision support is provided for regional soil health assessment and accurate fertilization management.
Through the steps, the semi-supervised learning model capable of accurately predicting the soil microbial community parameters from the hyperspectral data is obtained, and therefore the model has good generalization capability.
Step 2, based on section inference of the physical information neural network, constructing the physical information neural network by combining hyperspectral features, meteorological data and space coordinates according to output of a prediction model, introducing soil hydrothermal migration equation constraint, and deducing microorganism distribution and hydrothermal states at different depths;
The method solves the problem of deducing the microbial distribution of the soil profile and the coupling relation between the microbial distribution and the hydrothermal process from the hyperspectral data of the earth surface by a physical information neural network (Physics-InformedNeuralNetworks, PINN) method. It should be noted that the following is specifically performed:
step 2.1, constructing a PINN network architecture;
A physical information neural network is constructed, the network comprising a forward propagation portion and a physical constraint portion. The forward propagation part is a multi-layer neural network and is input into the surface hyperspectral eigenvector (Extraction from the hyperspectral data pre-processed in step 1), meteorological data vector(Including precipitation amount)Temperature (temperature)Etc.) and spatial location coordinatesAnd depth coordinatesOutput is the microorganism abundance at that location and depthMoisture content of soilAnd soil temperature
According to one embodiment of the present application, the concrete structure of the PINN network includes:
The input layer is used for receiving the hyperspectral feature vectors (the dimension is reduced to 20 main components), the meteorological data vectors (5 dimensions including precipitation, temperature, humidity, radiation intensity and wind speed), the space coordinates (2 dimensions, longitude and latitude or XY coordinates) and the depth coordinates (1 dimension);
a hidden layer comprising 5 fully connected layers, each layer comprising 64 neurons, using Swish activation functions to increase nonlinear expression capacity;
And the output layer is used for outputting the abundance of microorganisms (which can be multidimensional vectors and represent abundance of different groups), the water content of the soil and the temperature of the soil.
Optionally, in some embodiments, the PINN network may employ a residual network structure, and a jump connection is introduced to alleviate the gradient vanishing problem in deep network training, and improve the model convergence speed and performance. Specifically, a jump connection is added between every two hidden layers, and the output of the previous layer is directly added to the output of the next layer.
In other embodiments, the PINN network may employ a multi-scale fusion structure, process input features of different scales through multiple sub-networks in parallel, and then fusion the multi-scale features for prediction. For example, three parallel sub-networks may be constructed that process the hyperspectral features, the meteorological data, and the space-depth coordinates separately, and then splice or weight-fuse the outputs of the three sub-networks. This structure is particularly suitable for fusion processing of multi-source heterogeneous data.
The network structure can be expressed as:
;
Wherein the method comprises the steps of Is indicated in the positionDepth ofTime ofIs a microorganism abundance of (a); Is indicated in the position Depth ofTime ofIs a soil volume moisture content; Is indicated in the position Depth ofTime ofIs a soil temperature of (a); Representing a neural network function representing a mapping from input features to output variables; Respectively representing a surface hyperspectral feature vector, a meteorological data vector, a space coordinate and a depth coordinate; representing a set of neural network parameters, including all trainable weights and biases.
Step 2.2, defining physical constraint conditions;
In order to ensure that the neural network output meets the physical rule, the following physical equation constraint is introduced for the soil hydrothermal migration process:
moisture migration equation (richard equation):
;
Wherein the method comprises the steps of Representing the water content of the soil volumeFor time of dayRepresents the rate of change of water content over time; Representing a gradient operator, representing a spatial derivative; indicating unsaturated water conductivity, dependent on water content Is a function of (2); representing the spatial gradient of the water potential; Indicating source and sink items, such as root system water absorption, etc., indicating loss or replenishment of water;
heat conduction equation:
;
Wherein the method comprises the steps of Represents the heat capacity of the soil, heat capacity per unit volume of soil; indicating soil temperature For time of dayIs a partial derivative of (2); the thermal conductivity of the soil is represented, and the heat transfer capacity of the soil is represented; a spatial gradient representing soil temperature; represents the density of water; represents the specific heat capacity of water; a water flow velocity vector is represented, and the water flow velocity in the soil is represented; Representing a gradient operator, representing a spatial derivative;
step 2.3, constructing a PINN loss function;
PINN's total loss function includes two parts, data fitting loss and physical constraint loss:
;
Wherein the method comprises the steps of Representing PINN total loss functions; Weight coefficient representing data fitting loss, control Contribution in total loss; Representing data fitting loss, and measuring errors of model output and observation data; Weight coefficient representing physical constraint loss, control Contribution in total loss; Representing physical constraint loss, and measuring the satisfaction degree of model output to a physical equation;
Data fitting loss Calculation of profile observation data based on a small number of borehole sampling points:
;
Wherein the method comprises the steps of Representing the total number of observation points; Representing data fitting loss, and measuring errors of model output and observation data; Represent the first Microorganism abundance actual measurement values of the observation points; Represent the first Microbial abundance model predictive values of the observation points; Represent the first Actual measurement values of the soil volume water content of the observation points; Represent the first Predicted values of soil volume water content models of the observation points; Represent the first Actual measurement values of soil temperature of the observation points; Represent the first Predicted values of soil temperature models of the observation points.
Loss of physical constraintBased on the residual calculation of the physical equation, the difference value between the left end and the right end of the equation is estimated by adopting an automatic differential technology:
Wherein the method comprises the steps of Representing a loss of physical constraint,Representing the number of sampling points (which may be virtual points) used to calculate the physical constraint; Representing the Richards equation at the point Is used for the residual error of (c),Respectively represent the firstSpatial coordinates, depth coordinates and time coordinates of the observation points; representing the point of the heat conduction equation Is used for the residual error of (c),Respectively represent the firstSpatial coordinates, depth coordinates and time coordinates of the individual observation points.
Step2.4, generating a GAN-assisted physical constraint sample;
and (3) generating pseudo-section data meeting physical constraints by using the trained GAN model in the step (1), and enhancing the training effect of PINN. These pseudo-samples meet both the statistical relationship of hyperspectral and microbial abundance and the constraints of the hydrothermal transport physical process.
Through the steps, PINN models which can infer the microbial distribution and the hydrothermal state of different depths of the soil profile from the hyperspectral data of the earth surface are obtained through training, and in addition, the models are fit with actual observation data and accord with the physical law of soil hydrothermal migration.
Step 3, embedding a nitrogen circulation dynamics equation in the physical information neural network, and outputting key process parameters of nitrogen circulation by taking profile microorganism distribution as a driving force;
According to the method, a nitrogen cycle dynamics equation is embedded on the basis of PINN frames, inversion of key process parameters of microorganism-driven nitrogen cycle is achieved, and the problem that the parameters are difficult to directly obtain is solved. Note that the following is specifically performed:
step 3.1, embedding a dynamic equation of the nitrogen cycle process;
In the PINN framework of step 2, the description equations for the critical process of nitrogen cycling are added. Taking nitrification and denitrification as examples, the following equations are introduced:
The nitration process equation:
;
Wherein the method comprises the steps of A spatial gradient representing the concentration of ammonium nitrogen; Representing the concentration of ammonium nitrogen versus time Is a partial derivative of (2); Representing a gradient operator, representing a spatial derivative; represents the diffusion coefficient of ammonium ions, represents Diffusivity in soil; A nitration rate constant, a rate parameter representing the nitration reaction; Representing the function of the influence of the environmental factors on the nitrification, In order to be able to achieve a microbial abundance,For the water content of the soil volume,The soil temperature;
;
Wherein the method comprises the steps of A spatial gradient representing the nitrate nitrogen concentration; Indicating nitrate nitrogen concentration versus time Is a partial derivative of (2); representing the diffusion coefficient of nitrate ions, representing Diffusivity in soil; Representing a denitrification rate constant, representing a rate parameter of the denitrification reaction; representing the influence function of the environmental factor on denitrification, In order to be able to achieve a microbial abundance,For the water content of the soil volume,The temperature of the soil is set to be the temperature of the soil,Is the organic carbon content; Representing a gradient operator, representing a spatial derivative; A nitration rate constant, a rate parameter representing the nitration reaction; represents the concentration of ammonium nitrogen in the soil Is contained in the composition; Representing the function of the influence of the environmental factors on the nitrification, In order to be able to achieve a microbial abundance,For the water content of the soil volume,The soil temperature;
;
Wherein the method comprises the steps of Representing the function of the influence of the environmental factors on the nitrification,In order to be able to achieve a microbial abundance,For the water content of the soil volume,The soil temperature; Representing the effect function of moisture content on process rate, see below; Representing the temperature effect function on the process rate;
;
Wherein the method comprises the steps of Representing the influence function of the environmental factor on denitrification,In order to be able to achieve a microbial abundance,For the water content of the soil volume,The temperature of the soil is set to be the temperature of the soil,Is the organic carbon content; Representing the influence function of water content on the process rate; Representing the temperature effect function on the process rate; Representing the influence function of organic carbon on the process rate.
According to one embodiment of the application, these influencing functions can be expressed in particular as:
When (when) When (1):
;
otherwise ;
Wherein the method comprises the steps ofRepresenting the influence function of water content on the process rate; Represents the water content of the soil volume; Represents a lower limit of water content; represents the optimum water content; Represents an upper limit of water content;
;
Wherein the method comprises the steps of Representing the temperature effect function on the process rate; representing the soil temperature; representing a reference temperature; Temperature sensitivity coefficient is expressed as a multiple of the increase in reaction rate per 10 ℃ increase in temperature;
;
Wherein the method comprises the steps of Representing an influence function of organic carbon on a process rate; Represents the organic carbon content; representing the semi-saturation constant, representing the saturation effect of organic carbon on the process rate;
Step 3.2, expanding PINN output variables;
on the basis of PINN network output in the step 2, adding p-ammonium nitrogen Nitrate nitrogenPrediction of concentration and key parametersAndOutput as a function of depth.
According to one embodiment of the application, the extended PINN network structure adds 4 additional output nodes on the basis of step 2, corresponding to the ammonium nitrogen concentration, nitrate nitrogen concentration, nitrification rate constant and denitrification rate constant, respectively. The network structure can be expressed as:
;
Wherein the method comprises the steps of Represents the abundance of microorganisms; Represents the water content of the soil volume; representing the soil temperature; represents the ammonium nitrogen concentration; Represents nitrate nitrogen concentration; Indicating a nitrification rate constant; representing the denitrification rate constant; Representing the expanded neural network function; respectively representing the surface hyperspectral feature vector, the meteorological data vector, the space coordinate, the depth coordinate and the neural network parameter.
Step 3.3, expanding PINN physical constraint losses;
and (3) adding a residual term of a nitrogen circulation equation on the basis of the physical constraint loss in the step (2):
;
Wherein the method comprises the steps of Representing the physical constraint loss after expansion; representing physical constraint loss; the weight coefficient of the nitrogen circulation equation residual error is expressed, and the contribution of the nitrogen circulation equation residual error in the total loss is controlled; Representing the number of sampling points used to calculate the physical constraint; Representing the point of the kinetic equation of ammonium nitrogen Is a residual error of (2); representing the point of the kinetic equation of nitrate nitrogen Is a residual error of (2); Respectively represent the first Spatial coordinates, depth coordinates and time coordinates of the individual observation points.
Step 3.4, increasing parameter constraint conditions;
to ensure that the inversion obtained parameters are physically reasonable, parameter value range constraint is increased:
;
;
Wherein the method comprises the steps of Representing the minimum value of the nitrification rate constant; Represents the maximum value of the nitrification rate constant; Representing the minimum value of the denitrification rate constant; represents the maximum value of the denitrification rate constant; representing the nitrification rate constant of the model output; Representing the denitrification rate constant of the model output.
These constraints may be implemented by adding penalty terms to the loss function, or using a parameter mapping method (e.g., sigmoid function) to ensure that the output parameters are within reasonable limits.
Step 3.5, data assimilation and parameter optimization;
Adopting data assimilation method to obtain hyperspectral data and small quantity of section The concentration observations are combined to optimize PINN model parameters and process parameters.
According to one embodiment of the application, the data assimilation process is accomplished using an ensemble Kalman filtering (EnsembleKalmanFilter) method that updates the parameter estimates as the observed data arrives by constructing multiple ensemble members of model parameters. The method comprises the following specific steps:
Initializing, namely generating a plurality of set members (such as 100) for PINN model parameters and process parameters based on priori knowledge;
Predicting, namely obtaining a predicted value of a state variable by using a parameter operation model of each set member;
updating, namely updating the parameter value of each set member according to the Kalman gain when new observation data arrives;
And iterating, namely repeating the steps of prediction and updating until the parameters are converged.
Alternatively, in some embodiments, the data assimilation process may employ a particle filtering (PARTICLEFILTER) approach that handles non-linear and non-gaussian distribution through importance sampling and resampling steps. Specifically, each particle (i.e., a member of the parameter set) is assigned a weight that indicates how well the predicted value of the particle matches the observed value, and then resampling is performed according to the weights to generate a new set of particles and adding appropriate perturbations to maintain particle diversity. This method is particularly suitable for strongly nonlinear processes in soil-microorganism systems.
In other embodiments, a variation data assimilation method may be used to optimize model parameters by minimizing a cost function. The cost function typically contains a background error term (deviation from a priori parameter estimates) and an observation error term (deviation from observed data):
;
Wherein the method comprises the steps of Representing a cost function to minimize the target; representing a parameter vector to be estimated; representing an a priori parameter estimate vector; Representing a background error covariance matrix, representing uncertainty of the prior parameter; Representing a transpose operator; Representing the observation operator, and vector the parameters Mapping to an observation space; representing an observation vector comprising observation data; representing an observation error covariance matrix, representing uncertainty of observation data; Representing an inverse matrix operation.
The optimization objective is to minimize the extended PINN total loss function:
;
Wherein the method comprises the steps of Representing the extended PINN total loss function; a weight coefficient representing an extended data loss term; an extended data loss term representing a fitting error of observed data including concentration of nitrogen; a weight coefficient representing an extended physical constraint loss; Representing the physical constraint loss after expansion.
In the environment monitoring application scene, the method can be used for monitoring the nitrogen circulation process in agriculture and natural ecological systems, evaluating the risk of nitrogen loss (such as nitrate leaching, nitrogen and nitrous oxide emission), and providing scientific basis for environmental protection and emission reduction measures.
Through the steps, the key parameters of soil profile nitrogen circulation can be inverted from the hyperspectral data of the earth surfaceAnd) It can be seen that these parameters both conform to physical, chemical and biological process constraints and can interpret the actual observed nitrogen concentration data.
Step4, combining the nitrogen cycle key process parameters and profile microorganism distribution with observation data by utilizing a data assimilation method, inputting hyperspectral data of a target area, generating monitoring results of soil microorganism community spatial distribution, profile microorganism abundance distribution and nitrogen cycle parameter distribution, and carrying out uncertainty evaluation;
And based on the integrated model trained in the three steps, hyperspectral remote sensing monitoring of the soil microbial community in the target area is realized.
Step 4.1, inputting target area data;
inputting hyperspectral remote sensing data of an area to be monitored into a trained model system, comprising:
The ground surface hyperspectral data is a hyperspectral image of a target area acquired through a satellite, an unmanned aerial vehicle or ground equipment;
Auxiliary data, namely meteorological data (temperature, humidity, precipitation and the like) and geographic position information of a target area;
and the space coordinates are geographical coordinate information of each pixel in the target area.
Step 4.2, model prediction calculation;
calculating the input data through an integrated prediction model system:
Firstly, predicting surface microbial community parameters from hyperspectral data through a semi-supervised learning model trained in the step 1;
and then deducing the microbial distribution and the hydrothermal state of different depths of the soil profile through the physical information neural network trained in the step 2;
And finally obtaining the key process parameters of the microbial-driven nitrogen cycle through the nitrogen cycle parameter inversion model trained in the step 3.
Step 4.3, outputting a remote sensing monitoring result;
Obtaining a complete remote sensing monitoring result of the target area through model prediction calculation, wherein the method comprises the following steps:
Displaying spatial distribution characteristics of different microorganism functional groups in a target area;
providing the distribution information of the microbial abundance of different depth layers of the soil profile;
the nitrogen cycle process parameter distribution comprises the spatial distribution of key parameters such as a nitrification rate, a denitrification rate and the like;
The soil environment state information is the profile distribution of the soil water content, temperature and other environmental factors.
Step 4.4, uncertainty evaluation and reliability quantification;
Uncertainty evaluation is carried out on all prediction results:
calculating a prediction interval of each monitoring result by using a set prediction method;
Generating an uncertainty distribution map, and identifying a region with lower prediction reliability;
providing a confidence assessment of the predicted result, and providing a reliability reference for decision-making application;
and outputting a quality control report of the monitoring result, wherein the quality control report comprises model performance indexes and applicability evaluation.
Through the steps, a complete hyperspectral remote sensing monitoring result of the soil microbial community is formed, and scientific basis is provided for accurate agricultural management, environment monitoring and ecological assessment.
A storage medium comprising a memory and one or more processors, the memory having executable code stored therein, the one or more processors, when executing the executable code, for implementing a soil microflora hyperspectral remote sensing monitoring method as described above.
Here, the present invention provides an implementation example:
an example of the true application of the invention in monitoring a farmland soil microbiota is provided:
The application example is aimed at hyperspectral remote sensing monitoring of soil microbial communities in a farmland ecosystem. The farmland occupies about 100 hectares, is mainly used for planting wheat and corn rotation, adopts a unified fertilization mode in the past, causes low nitrogen fertilizer utilization efficiency in partial areas, and has nitrate leaching risk. The method aims at realizing high-precision monitoring of soil microbial community distribution and nitrogen circulation key parameters and providing decision support for accurate fertilization.
In the agricultural field, a hyperspectral imager (wavelength range 400-2500nm, spectral resolution 10 nm) mounted on an unmanned aerial vehicle is used to acquire surface hyperspectral data. Meanwhile, 25 sampling points are arranged in a farmland, soil sample collection is carried out, wherein only 10 points are used for soil microbial community composition measurement (16 SrRNA sequencing) and nitrogen circulation functional gene abundance measurement, the 10 points are used as labeled data, and the other 15 points are used for measuring basic physicochemical properties and are used as unlabeled data. In addition, 5 out of 10 tagged spots were selected for 0-100cm section stratified sampling, one sample was taken every 20cm, and each layer of microorganism composition and nitrogen morphology was determined for PINN model training and validation.
The hyperspectral data preprocessing comprises atmospheric correction, geometric correction, denoising and band selection, and finally 50 characteristic bands are selected for subsequent analysis. The soil microorganism data is processed by a bioinformatics analysis flow to obtain microorganism community composition information, and the functional flora related to nitrogen circulation is focused on, including nitrifying bacteria (AOB, AOA) and denitrifying bacteria.
In order to solve the problem of insufficient marked samples, a conditional GAN model is constructed. The generator adopts a coder-decoder structure, an input layer receives 128-dimensional random noise vectors and 10-dimensional microorganism parameter vectors (comprising relative abundance and diversity indexes of key flora), and outputs hyperspectral characteristic vectors of 50 wave bands through processing of 3 coding convolution layers and 3 decoding transposed convolution layers. The discriminator comprises 4 convolution layers and 1 full connection layer, and outputs true and false discrimination results.
Model training uses a small batch gradient descent method with a batch size of 32, and the learning rate is initially set to 0.0002, using an Adam optimizer. During training, the generator is updated 5 times per training of the arbiter to prevent premature collapse of the generator. To ensure the quality of the generated samples, spectral smoothness constraints and inter-band correlation constraints are introduced. After training, 100 new pseudo hyperspectral microorganism data pairs are generated by inputting different microorganism parameter conditions.
When the semi-supervised learning framework is constructed, MEANTEACHER structures are adopted. The student model and the teacher model both adopt four-layer convolutional neural network structures. For 10 real labeled samples and 100 generated pseudo-labeled samples, calculating a label prediction error as a supervision loss, and for 15 unlabeled samples, applying different data enhancement (such as random band mask, random noise addition and the like), and calculating a mean square error of prediction results of the two as a consistency loss through student model and teacher model prediction. In model training, consistency loss weight coefficientsIs set to 0.5, and is linearly increased to 1.0 along with training, and the regularization loss weight coefficientSet to 0.001.
The PINN networks were constructed containing 5 hidden layers of 64 neurons each, using Swish activation functions. The input features include 20-dimensional hyperspectral feature vectors, 5-dimensional meteorological data vectors (obtained from the nearest meteorological station), 2-dimensional spatial coordinates (farmland grid coordinates) and 1-dimensional depth coordinates (in the range of 0-100 cm) extracted from the GAN-trained discriminators.
In terms of physical constraints, richards water migration equation and heat conduction equation were introduced. The parameter setting comprises the unsaturated water conductivity of soilParameterizing by vanGenuchten model, thermal capacity of soilAnd heat conductivity coefficientBased on soil texture and organic matter content estimation. In the loss function, the data fits the loss weightsSet to 1.0, physical constraint loss weightInitially set to 0.1 and gradually increased to 0.5 during training.
The training process uses data from 5 profile sampling points for a total of 25 depth levels of observation data. Meanwhile, pseudo samples generated by GAN are used for assisting training, and the inference capability of the model on profile information under different soil conditions is enhanced. After training, the model can infer microorganism abundance, soil water content and temperature distribution at different depths (0-100 cm) from surface hyperspectral data at any position.
On the PINN frame basis, a nitrogen cycle dynamics equation is embedded. In the nitrification equation, the conversion of ammonium nitrogen to nitrate nitrogen is affected by microbial abundance, soil moisture and temperature. In practice, the moisture influencing functionThe parameters are set as follows:,, Temperature influencing function In (a)The value is set to 2.0, the reference temperature20 ℃.
The extended PINN network increases the ammonium nitrogen concentration, nitrate nitrogen concentration, nitrification rate constant, and denitrification rate constant by 4 output nodes. To ensure that the parameters are physically reasonable, the constant range of the nitration rate is 0.01 to 0.5dThe denitrification rate constant ranges from 0.001 to 0.1d. The network output is mapped into a reasonable parameter range by a sigmoid function.
The data assimilation process employs a set Kalman filtering method to construct 100 set members, each member comprising different initial PINN parameters and soil parameters. The set member parameters are updated by multiple iterations based on the ammonium nitrogen and nitrate nitrogen concentration observations (observations for a total of 25 depth levels) for 5 profile sampling points. Finally, the spatial distribution of the nitrification rate and the denitrification rate at different positions and different depths in the whole farmland is obtained through inversion.
And carrying out hyperspectral remote sensing monitoring on the whole 100 hectare farmland area based on the integrated model completed by the training. And inputting full farmland hyperspectral image data (2 m spatial resolution, 2500 pixels in total) acquired by the unmanned aerial vehicle into a model system after training.
Input data preparation:
And preprocessing the full farmland hyperspectral image, including radiation correction, geometric registration and spectral feature extraction, and finally obtaining a 50-dimensional hyperspectral feature vector of each pixel. Meanwhile, meteorological data (15.2 ℃ in daily average temperature, 65% in relative humidity and 12mm in precipitation) and space coordinate information of each pixel in a corresponding period are collected.
Model predictive calculation:
the pre-processed hyperspectral data are predicted by three sub-models in sequence:
firstly, predicting surface microbial community parameters through a semi-supervised learning model to obtain nitrifying bacteria abundance, denitrifying bacteria abundance and microbial diversity index of each pixel;
Then deducing soil profile information through PINN model to obtain microorganism abundance distribution, soil water content and temperature distribution of each 20cm level in the depth range of 0-100 cm;
Finally, the nitrification rate constant and the denitrification rate constant of each depth level are obtained through a nitrogen circulation parameter inversion model.
And (3) outputting a monitoring result:
The complete remote sensing monitoring result in the farmland range is obtained through model prediction calculation, which comprises the following steps:
generating a spatial distribution map of nitrifying bacteria, denitrifying bacteria and total microorganism diversity, and identifying 3 high nitrifying active areas, 2 high denitrifying active areas and 4 microorganism diversity hot spot areas;
providing 5 depth layers (0-20 cm, 20-40cm, 40-60cm, 60-80cm, 80-100 cm) of microorganism abundance three-dimensional distribution, showing that the microorganism abundance decreases with depth, but local enrichment occurs in the 40-60cm layers;
Obtaining a spatial distribution diagram of a nitrification rate and a denitrification rate, wherein the nitrification rate is changed within the range of 0.05-0.35 d-1-1, and the denitrification rate is changed within the range of 0.002-0.08 d-1-1;
The soil environment state information is that the water content of the soil changes between 22% and 38%, the temperature of the soil changes between 13.5 ℃ and 16.8 ℃, and obvious spatial heterogeneity is presented.
Uncertainty evaluation:
By using the set prediction method, 95% confidence intervals of each monitoring result are calculated. The results showed that the average uncertainty of the microbial abundance predictions was ± 12.5% and the average uncertainty of the nitrogen cycling parameter predictions was ± 18.3%. An uncertainty profile was generated that identified a lower reliability of prediction (uncertainty > 25%) for the region of about 15 hectares at the southeast corner of the farmland, suggesting that validation samples be added to this region.
Based on the inversion result, the farmland is partitioned into 5 fertilization management areas. Aiming at the areas with high abundance and high nitrification rate of nitrifying bacteria, the method reduces the amount of nitrogen fertilizer applied at one time, adopts a fractional application strategy, increases drainage measures to reduce anaerobic environment for the areas with strong denitrification, and simultaneously uses a nitrification inhibitor to delay the conversion of ammonium nitrogen into nitrate nitrogen, thereby improving the utilization rate of the nitrogen fertilizer and reducing the loss of nitrogen.
As shown in fig. 6 to 8, the present application example focuses on verifying two main technical effects, data utilization efficiency improvement and nitrogen cycle parameterization breakthrough.
In the application scene, compared with the traditional method, a large amount of field sampling and laboratory analysis are needed, and the method only uses 10 labeled samples and 15 unlabeled samples, so that the data acquisition cost and time are reduced. By GAN data enhancement and semi-supervised learning, the prediction accuracy reached the level of the traditional model trained using 30 labeled samples.
Specifically, the Root Mean Square Error (RMSE) of the nitrifying bacteria abundance predictions was compared to 0.42 for the conventional supervised learning method (30 marker samples), 0.68 for the conventional supervised learning method (10 marker samples), and 0.39 for the present invention (10 marker samples+gan enhancement+semi-supervised learning). This shows that the present invention maintains not only the prediction accuracy but also a slight improvement (RMSE reduction of about 7%) with a reduction of the data volume of about 67%.
Meanwhile, the invention greatly reduces the sampling and analysis cost. While the conventional method requires microorganism colony sequencing and functional gene analysis for 30 spots, the total cost is about 45000 yuan and the time is about 15 days, the invention only requires microorganism analysis for 10 spots, the cost is about 15000 yuan and the time is about 7 days (including model training time).
The method successfully realizes inversion of the soil profile nitrogen circulation key parameters from the surface hyperspectral data, and provides key support for accurate nitrogen fertilizer management. The traditional method needs to obtain the parameters through an indoor culture test or an in-situ tracing test, and each point location is high in determination cost, long in period and limited in space representativeness.
The accuracy of the inversion parameters was verified by comparison with laboratory measurements. At 25 depth levels of 5 profile verification points, the correlation coefficient of the inverted nitrification rate constant and the indoor culture measured value reaches 0.82, the average relative error is 16.8 percent, the R 2 of the inverted denitrification rate constant and the measured value is 0.78, and the average relative error is 19.3 percent. This accuracy has good practical value for monitoring the nitrogen cycle parameters of the soil in a large range.
And the fertilizer application management adjustment based on inversion parameters improves the nitrogen fertilizer utilization efficiency. In the comparative test of farmland test areas, the nitrogen fertilizer utilization rate of the traditional uniform fertilization area is 35.2%, and the nitrogen fertilizer utilization rate is improved to 46.8% and the amplification reaches 33% based on the area of the zoned precise fertilization. Meanwhile, nitrate leaching monitoring results show that the nitrogen leaching loss of the precise fertilization area is reduced by 28.5% compared with that of the traditional fertilization area, and the environmental pollution risk is reduced.
Furthermore, the uncertainty evaluation function of the present invention provides reliability information for decisions. And the uncertainty of the parameters obtained through the aggregate prediction is quantified, and a conservative fertilization strategy is recommended to be adopted for the low-reliability region (the uncertainty is higher than 30 percent), so that the risk is further reduced.
While the embodiments of the present invention have been described above, the embodiments are not limited to the above-described embodiments, which are intended to be illustrative only and not limiting, and many equivalents thereof may be made by those of ordinary skill in the art in light of the present disclosure, which fall within the scope of the embodiments.

Claims (10)

1. The hyperspectral remote sensing monitoring method for the soil microbial community is characterized by comprising the following steps of:
Based on data enhancement and semi-supervised learning of a generated countermeasure network, hyperspectral remote sensing data and a hyperspectral microorganism data pair with a label are obtained, a pseudo hyperspectral microorganism data pair generated by the generated countermeasure network is constructed, a semi-supervised learning framework is constructed, and a microbial community parameter prediction model is output;
Based on section inference of the physical information neural network, according to the output of the prediction model, combining hyperspectral features, meteorological data and space coordinates, constructing the physical information neural network, introducing soil hydrothermal migration equation constraint, and inferring microorganism distribution and hydrothermal states of different depths;
Embedding a nitrogen circulation dynamics equation in the physical information neural network, and outputting a nitrogen circulation key process parameter by taking profile microorganism distribution as a driving force;
and combining the nitrogen cycle key process parameters and profile microorganism distribution with the observation data by utilizing a data assimilation method, inputting hyperspectral data of a target area, generating monitoring results of soil microorganism community spatial distribution, profile microorganism abundance distribution and nitrogen cycle parameter distribution, and carrying out uncertainty evaluation.
2. The method of claim 1, wherein the step of creating pseudo hyperspectral microbial data pairs for generating the countermeasure network comprises:
constructing a generation countermeasure network comprising a generator and a discriminator, wherein the generator receives a random noise vector and a microorganism parameter condition vector as inputs to generate hyperspectral data conforming to real distribution;
Training a generator and a discriminator by adopting an alternative optimization strategy until the generator can generate pseudo hyperspectral data conforming to physical constraint;
Generating corresponding hyperspectral data by inputting different microorganism parameter condition vectors to form a pseudo-mark sample;
And constructing a semi-supervised learning framework, and performing model training by using the hyperspectral data in real distribution, the pseudo hyperspectral data conforming to physical constraints and the pseudo-mark samples.
3. The method for hyperspectral remote sensing monitoring of soil microbial communities according to claim 2, wherein the constructed semi-supervised learning framework comprises a student model and a teacher model, the student model updates parameters through supervised training, the teacher model parameters are exponential moving averages of the student model parameters, and the loss functions of the semi-supervised learning framework comprise supervision loss, consistency loss and regularization loss.
4. The method for hyperspectral remote sensing monitoring of soil microbial communities according to claim 1, wherein the construction of the physical information neural network comprises the following steps:
constructing a neural network comprising a forward propagation portion and a physical constraint portion;
to ensure that the output of the neural network meets the physical rule, introducing the constraint of a physical equation in the soil hydrothermal migration process;
Constructing a total loss function comprising data fitting loss and physical constraint loss;
Pseudo-profile data satisfying physical constraints is generated using a generation countermeasure network.
5. The method of hyperspectral remote sensing monitoring of a soil microflora according to claim 4 wherein the physical equation constraints include a moisture migration equation and a heat conduction equation.
6. The method for hyperspectral remote sensing monitoring of a soil microbial community according to claim 1, wherein the step of embedding a nitrogen circulation dynamics equation in a physical information neural network comprises the steps of:
Adding a description equation of a nitrogen circulation key process in a physical information neural network framework;
expanding output variables of a physical information neural network, and increasing prediction of ammonium nitrogen and nitrate nitrogen concentration and output of the nitrification rate and denitrification rate along with depth change;
Expanding physical constraint loss of a physical information neural network and increasing residual terms of a nitrogen circulation equation;
And the constraint of the parameter value range is increased, so that the physical rationality of the inverted parameters is ensured.
7. The hyperspectral remote sensing monitoring method of soil microbial communities as claimed in claim 6, wherein the describing equations of the key process of nitrogen circulation comprise a nitrification process equation and a denitrification process equation, describing dynamic change processes of ammonium nitrogen and nitrate nitrogen and influences of microorganism abundance, water content, temperature and organic carbon on the nitrification and denitrification processes.
8. The method of claim 1, wherein the step of using the data assimilation method comprises updating the parameter estimates when the observed data arrives by constructing a plurality of set members of the model parameters using a set kalman filter method.
9. The method for hyperspectral remote sensing monitoring of a soil microbial community according to claim 1, wherein the uncertainty evaluation comprises:
calculating a prediction interval of each monitoring result by using a set prediction method;
Generating an uncertainty distribution map, and identifying a region with lower prediction reliability;
providing a confidence assessment of the predicted result, and providing a reliability reference for decision-making application;
and outputting a quality control report of the monitoring result, wherein the quality control report comprises model performance indexes and applicability evaluation.
10. A storage medium comprising a memory and one or more processors, the memory having executable code stored therein, the one or more processors configured to implement the method of hyperspectral remote sensing monitoring of a soil microbial community of any one of claims 1-9 when the executable code is executed.
CN202510884714.1A 2025-06-30 2025-06-30 Soil microbial community hyperspectral remote sensing monitoring method and storage medium Active CN120408098B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202510884714.1A CN120408098B (en) 2025-06-30 2025-06-30 Soil microbial community hyperspectral remote sensing monitoring method and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202510884714.1A CN120408098B (en) 2025-06-30 2025-06-30 Soil microbial community hyperspectral remote sensing monitoring method and storage medium

Publications (2)

Publication Number Publication Date
CN120408098A true CN120408098A (en) 2025-08-01
CN120408098B CN120408098B (en) 2025-09-30

Family

ID=96523361

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202510884714.1A Active CN120408098B (en) 2025-06-30 2025-06-30 Soil microbial community hyperspectral remote sensing monitoring method and storage medium

Country Status (1)

Country Link
CN (1) CN120408098B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105043992A (en) * 2015-07-17 2015-11-11 辽宁师范大学 Hyperspectral remote-sensing monitoring method for wetland soil nitrification microbial community
US20220129735A1 (en) * 2019-05-20 2022-04-28 Institute of intelligent manufacturing, Guangdong Academy of Sciences Semi-supervised Hyperspectral Data Quantitative Analysis Method Based on Generative Adversarial Network
CN118396180A (en) * 2024-05-21 2024-07-26 西安石油大学 Water body pollutant prediction method by combining sample expansion with improved particle swarm algorithm
US12260637B1 (en) * 2024-01-25 2025-03-25 Institute Of Mountain Hazards And Environment, Chinese Academy Of Sciences Classification method and system of UAV hyperspectral vegetation species based on deep learning
CN120218406A (en) * 2025-03-10 2025-06-27 海南省林业科学研究院(海南省红树林研究院) A collaborative optimization decision-making system for tropical rainforest biomass and carbon sink based on deep learning

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105043992A (en) * 2015-07-17 2015-11-11 辽宁师范大学 Hyperspectral remote-sensing monitoring method for wetland soil nitrification microbial community
US20220129735A1 (en) * 2019-05-20 2022-04-28 Institute of intelligent manufacturing, Guangdong Academy of Sciences Semi-supervised Hyperspectral Data Quantitative Analysis Method Based on Generative Adversarial Network
US12260637B1 (en) * 2024-01-25 2025-03-25 Institute Of Mountain Hazards And Environment, Chinese Academy Of Sciences Classification method and system of UAV hyperspectral vegetation species based on deep learning
CN118396180A (en) * 2024-05-21 2024-07-26 西安石油大学 Water body pollutant prediction method by combining sample expansion with improved particle swarm algorithm
CN120218406A (en) * 2025-03-10 2025-06-27 海南省林业科学研究院(海南省红树林研究院) A collaborative optimization decision-making system for tropical rainforest biomass and carbon sink based on deep learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
毕银丽;郭晨;王坤;: "煤矿区复垦土壤的生物改良研究进展", 煤炭科学技术, no. 04, 15 April 2020 (2020-04-15) *

Also Published As

Publication number Publication date
CN120408098B (en) 2025-09-30

Similar Documents

Publication Publication Date Title
Jia et al. Physics-guided machine learning for scientific discovery: An application in simulating lake temperature profiles
Yigini et al. Assessment of soil organic carbon stocks under future climate and land cover changes in Europe
CN116822381B (en) Marine temperature salt structure inversion method based on artificial intelligence
Liu et al. KGML-ag: a modeling framework of knowledge-guided machine learning to simulate agroecosystems: a case study of estimating N 2 O emission using data from mesocosm experiments
Tajik et al. Digital mapping of soil invertebrates using environmental attributes in a deciduous forest ecosystem
Yin et al. Ecosystem services assessment and sensitivity analysis based on ANN model and spatial data: A case study in Miaodao Archipelago
Wang et al. Investigating the ability of deep learning on actual evapotranspiration estimation in the scarcely observed region
CN102222313B (en) Urban evolution simulation structure cell model processing method based on kernel principal component analysis (KPCA)
CN114399073A (en) Ocean surface temperature field prediction method based on deep learning
Esmaeilbeiki et al. Exploring the application of soft computing techniques for spatial evaluation of groundwater quality variables
Derot et al. Benefits of machine learning and sampling frequency on phytoplankton bloom forecasts in coastal areas
CN118191976B (en) A runoff forecasting method based on SWAT and LSTM
CN120509682A (en) Cascade reservoir optimization regulation and control method based on water nitrogen and phosphorus circulation
Sudheer et al. Application of a pseudo simulator to evaluate the sensitivity of parameters in complex watershed models
Song et al. Heuristic cellular automaton model for simulating soil organic carbon under land use and climate change: A case study in eastern China
CN117496369A (en) Mining area carbon sink data set construction method
CN119202613A (en) A multi-source remote sensing ecosystem carbon flux estimation method based on phenology
CN119918581A (en) Sound velocity distribution prediction method based on Att-KAN neural network model
Martel et al. Assessing the adequacy of traditional hydrological models for climate change impact studies: A case for long-short-term memory (LSTM) neural networks
Saeed et al. Water temperature prediction in a subtropical subalpine lake using soft computing techniques
Azad et al. Generalization of multiple depths soil temperature estimation using LSTM and CNN
CN115618720B (en) A soil salinization prediction method and system based on altitude
CN120409837B (en) Agricultural production area monitoring big data analysis method and system
CN120408098B (en) Soil microbial community hyperspectral remote sensing monitoring method and storage medium
CN120671556A (en) Runoff process intelligent simulation method for data-missing area

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant