CN117726943A - Ultra-high resolution vegetation canopy height estimation method - Google Patents

Abstract

The invention discloses a method for estimating the height of a canopy of vegetation with ultra-high resolution, and relates to the field of intelligent vegetation monitoring. The method comprises the following steps: acquiring field ultra-high resolution aviation RGB image data, LIDAR point cloud data, golgi digital elevation model data and synthetic aperture radar satellite data aiming at a sample area, and combining the data into a first multi-source data set; performing data preprocessing on the first multi-source data set to obtain a second multi-source data set; establishing an initial ultra-high resolution vegetation canopy height estimation model by taking a depth residual convolutional neural network as an encoder and taking FPN and a transducer as decoders, and training the ultra-high resolution vegetation canopy height estimation model by using a second multi-source data set; and acquiring a vegetation image aiming at the target area, and obtaining vegetation canopy height information of the target area by using an ultra-high resolution vegetation canopy height estimation model. Compared with the prior art, the method improves the overall accuracy of vegetation canopy height prediction.

Description

An ultra-high resolution vegetation canopy height estimation method

Technical field

The present invention relates to the field of vegetation intelligent monitoring technology, and more specifically, to an ultra-high-resolution vegetation canopy height estimation method.

Background technique

Vegetation can maintain basic ecosystem functions of the Earth's climate and biosphere and provide a wide range of ecosystem services, and plays an important role in reducing greenhouse gas emissions and mitigating risks caused by climate change. Vegetation canopy height is a simple but important factor in the vertical structure of the forest. It can not only reflect the growth ability of vegetation, but also reflect the ecological niche needs of plants, and can also indicate the level of forest biomass. A taller vegetation canopy can occupy a more favorable ecological niche, obtain more light, and facilitate photosynthesis. Automatically obtaining vegetation canopy height information from remotely sensed data is useful for estimating aboveground biomass and wood mass of forests, monitoring the impact of forest degradation, measuring the success of forest restoration, and can be used to construct other key ecosystem variables such as primary production and biodiversity. It plays an important role in mold and other aspects.

Existing research shows that deep convolutional neural networks (CNNs) can achieve very good results in processing remote sensing images (such as scene classification and object detection and prediction). CNNs can automatically learn not only low-level and mid-level features of images, but also high-level semantic features in the original images. Since the feature pyramid network (FPN), a variant structure of deep convolutional neural network, was proposed, it has become the current mainstream framework for multi-feature layer prediction. FPN is usually a top-down neural network that can predict each pixel simultaneously when segmenting an image.

Despite this, the FPN-based vegetation canopy prediction method still has the following issues that need to be properly resolved, including:

(1) By using CNNs as its encoder for image feature extraction, although such output contains high-level semantic features, it is too rough and easily loses the edge details of the image;

(2) Although passing low-level features to the FPN decoder through "skip" connections or using the maximum value position of the max pooling layer can optimize the prediction results, this method can easily lead to the generation of redundant features and reduce the network's performance. Learning efficiency.

In addition, the output features usually contain category uncertainty or non-boundary related information, which affects the optimization of the estimation results:

1) Vegetation in most scenarios, especially in developed urban areas, has irregular vegetation distribution and is affected by other environmental factors. Its spectral reflectivity varies greatly and is easily blocked by the shadows of surrounding high-rise buildings;

2) The large intra-class differences and small inter-class differences in high-resolution remote sensing images make the spectral and geometric characteristics of vegetation complex.

Contents of the invention

The present invention provides an ultra-high-resolution vegetation canopy height estimation method to overcome the defects described in the above-mentioned prior art, such as the loss of edge information and the output features containing category uncertainty or non-boundary related information, resulting in low accuracy of estimation results. .

In order to solve the above technical problems, the technical solutions of the present invention are as follows:

The first aspect is a super-resolution vegetation canopy estimation method, including:

Obtain field ultra-high-resolution aerial RGB image data, LIDAR point cloud data, Copernicus digital elevation model data and synthetic aperture radar satellite data for the sample area, and combine them into the first multi-source data set;

Perform data preprocessing on the first multi-source data set to obtain a second multi-source data set;

Using the deep residual convolutional neural network as the encoder and FPN and Transformer as the decoder, an initial ultra-high-resolution vegetation canopy height estimation model is established, and the second multi-source data set is used to train the ultra-high resolution vegetation canopy height estimation model. High-resolution vegetation canopy height estimation model;

Obtain a vegetation image of the target area, and use the ultra-high-resolution vegetation canopy height estimation model to obtain vegetation canopy height information of the target area.

In the second aspect, a super-resolution vegetation canopy height estimation system applies the method described in the first aspect, including:

The multi-source data acquisition module is used to obtain field ultra-high-resolution aerial RGB image data, LIDAR point cloud data, Copernicus digital elevation model data and synthetic aperture radar satellite data for the sample area, and combine them into the first multi-source data set; also used to perform data preprocessing on the first multi-source data set to obtain a second multi-source data set; and also used to obtain vegetation images for the target area;

Model training module, used to establish an initial ultra-high-resolution vegetation canopy height estimation model using a deep residual convolutional neural network as the encoder and FPN and Transformer as the decoder, and train using the second multi-source data set The ultra-high resolution vegetation canopy height estimation model;

A vegetation canopy height information estimation module is used to carry out training to complete the ultra-high-resolution vegetation canopy height estimation model; and is also used to utilize the ultra-high-resolution vegetation canopy height estimation model based on vegetation images of the target area. Obtain vegetation canopy height information in the target area.

In a third aspect, a computer-readable storage medium stores at least one instruction, at least a program, a code set or an instruction set, and the at least one instruction, at least a program, code set or instruction set is processed by The programmer is loaded and executed to implement the method described in the first aspect.

Compared with the existing technology, the beneficial effects of the technical solution of the present invention are:

The present invention discloses an ultra-high-resolution canopy height estimation method. By establishing an ultra-high-resolution vegetation canopy height estimation model using a deep residual convolutional neural network as an encoder and FPN and Transformer as decoders, and Multi-source remote sensing data is used to train it, so that the ultra-high-resolution vegetation canopy height estimation model has image feature self-learning capabilities, and can obtain low-medium-high-level features of the image through nonlinear operations and multiple downsampling. , and filter and fuse effective features through the FT module (FPN and Transformer). Compared with the existing technology, the present invention improves the overall accuracy of vegetation canopy height prediction.

Description of the drawings

Figure 1 is a schematic flow chart of an ultra-high-resolution canopy height estimation method in Embodiment 1 of the present invention;

Figure 2 is a schematic diagram of the data processing flow of the ultra-high-resolution vegetation canopy height estimation model in Embodiment 1 of the present invention;

Figure 3 is a schematic structural diagram of an ultra-high-resolution canopy height estimation system in Embodiment 2 of the present invention.

Detailed ways

The terms "first", "second", etc. in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances, and are merely a way of distinguishing objects with the same attributes in describing the embodiments of the present application. Furthermore, the terms "include" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product or apparatus comprising a series of elements need not be limited to those elements, but may include not explicitly other elements specifically listed or inherent to such processes, methods, products or equipment.

The drawings are for illustrative purposes only and should not be construed as limitations of this patent;

In order to better illustrate this embodiment, some components in the drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product;

It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

In order to facilitate implementation by those skilled in the art, some concepts involved in the embodiments of the present invention are described as follows:

1. Deep residual convolutional neural network (ResNet)

ResNet is a deep convolutional neural network, which usually includes a convolution module (convolution layer, batch layer, ReLU activation layer and maximum pooling layer), four residual modules with similar structures, and a classification module (mean pooling layer). , fully connected layer and Softmax classification layer).

After passing through the first convolution module, the size of the input image becomes 1/2 of its original size. Each time it passes through a residual module, the size of the image also gradually changes to 1/2 of its original size. Finally, the size of the original image is 1 Feature map of /32

Specifically, according to the network depth, there are ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152.

Among them, for Resnet50, ResNet101, and ResNet152, each residual module consists of a projection convolution block (Projection shortcut) and multiple consecutive identity convolution blocks (Identity shortcut). The projected convolution block can double the number of feature maps and reduce the feature map by 1/2, while the identity convolution block does not change the size and number of features of the input and output.

In addition, the convolutional layer is a feature extractor that learns features to represent the input image. The convolutional layer is composed of several convolution units (neurons). Each feature surface is composed of multiple neurons. Neurons on the same feature plane share weights. The parameters of each unit are optimized and calculated by the network backpropagation algorithm. , its function is to obtain different features of the input, such as edges, right angles, textures and other information. Given the feature map X ^l-1 as the input of the convolutional layer, use the kth filter Process the input feature map with equation (1) to obtain the output feature map:

In the formula, where is the feature map obtained after the convolution operation, * is the convolution operation,/> is the k-th bias vector of layer l. Formula (1) can greatly reduce the amount of neural network parameters and at the same time obtain the characteristic result corresponding to each neuron.

The function of the batch normalization/batch processing (Batch Norm, BN) layer is to avoid gradient disappearance or gradient explosion in the neural network. In the BN layer, the normalization process performed on each input batch is transformed as follows:

in, is the result after scale transformation and offset, γ ^l is the normalized scale parameter, and β ^l is the offset parameter. Through the normalization process of equation (2), all inputs can be concentrated near 0, so that the input of each layer It won't make much of a difference.

The activation function layer is to control the activation level of neurons for forward signal transformation. Taking the results obtained from the BN layer as input, the rectified linear unit (ReLU) activation function is used to perform nonlinear mapping of the input features.

The Pooling Layer is mainly used to abstract input features. Maximum pooling or average pooling is usually used to obtain downsampled feature maps; the purpose of the Pooling Layer is to reduce the feature surface, simplify the model complexity, and reduce the parameters of the model. At the same time Achieve spatial invariance. The pooling layer follows the convolution layer and is also composed of multiple feature surfaces. Each feature surface corresponds to a feature surface of the previous layer and does not change the number of feature surfaces. It usually has mean sampling. There are two forms (mean pooling) and maximum sampling (max pooling): each weight in the convolution kernel of mean sampling is 0.25. If the sliding step of the convolution kernel on the input image (Input X) is 2, the effect of mean subsampling is equivalent to reducing the original image blur to 1/4 of the original; Only one of them is 1, and the rest are all 0. The position of 1 in the convolution kernel corresponds to the position with the largest value of the part of the input image covered by the convolution kernel. If the sliding step of the convolution kernel on the input image is 2 and the size is 2*2, the effect of maximum sampling is to reduce the original image to 1/4 of the original image and retain the strongest input of each 2*2 area. .

2. Copernicus digital elevation model data

Copernicus Digital Elevation Model data is recognized as one of the best global open source DEMs. Its absolute elevation accuracy and horizontal accuracy are the best among global open source DEMs. It also has the best terrain detail performance, and the actual effect is also better than 30m (meter) resolution, good consistency.

The technical solution of the present invention will be further described below with reference to the accompanying drawings and examples.

Example 1

This embodiment proposes an ultra-high-resolution vegetation canopy height estimation method, see Figure 1, which includes:

Obtain field ultra-high-resolution aerial RGB image data, LIDAR point cloud data, Copernicus digital elevation model data and synthetic aperture radar satellite data for the sample area, and combine them into the first multi-source data set;

Perform data preprocessing on the first multi-source data set to obtain a second multi-source data set;

Using the deep residual convolutional neural network as the encoder, FPN and Transformer as the decoder, establish an initial ultra-high resolution vegetation canopy height estimation model, and use the second multi-source data set to train the ultra-high resolution Vegetation canopy height estimation model;

Obtain a vegetation image of the target area, and use the ultra-high-resolution vegetation canopy height estimation model to obtain vegetation canopy height information of the target area.

In this embodiment, based on deep learning technology, an ultra-high-resolution vegetation canopy height estimation model (ARFTNet) with an encoder-decoder structure is established. Specifically, the encoder is composed of a deep residual convolutional neural network (ResNet) , used to learn low-medium-high-level features of the input image. The decoder includes FPN and Transformer, which are used to screen and fuse effective features, and use multi-source remote sensing data to train the model so that it can eventually generate high-level features. Quality vegetation canopy height information. Compared with the existing technology, using the ultra-high-resolution vegetation canopy height estimation model described in this implementation to estimate the vegetation canopy height can improve the overall accuracy of the vegetation canopy height prediction results.

In some examples, the synthetic aperture radar satellite data is Sentinel-1 data with a spatial resolution of 10m, which can provide high-resolution radar images to characterize surface features, which is beneficial to removing the effects of surface features. Effects of vegetation canopy height estimation.

It should be noted that the vegetation images for the target area include wild ultra-high-resolution aerial RGB image data and LIDAR point cloud data for the target area.

In some examples, the sample area is part of the target area;

In a specific implementation process, one-third of the wild ultra-high-resolution aerial RGB image data and LIDAR point cloud data for the target area were randomly selected as model training data.

It should also be noted that the ultra-high resolution mentioned in this embodiment means that the spatial resolution reaches 1m.

In some preferred embodiments, the acquisition of field ultra-high-resolution aerial RGB image data, LIDAR point cloud data, Copernicus digital elevation model data and synthetic aperture radar satellite data for the sample area includes:

The image control points are laid out using the regional mesh point layout method;

Based on the preset route, aerial photography is used to collect initial aerial RGB image data and initial LIDAR point cloud data of the sample area;

According to the image control points, the ultra-high-resolution initial aerial RGB image data and the initial LIDAR point cloud data are subjected to control point encryption, aerial triangulation calculation and accuracy monitoring, and at the same time, the outer surface of each image is calculated. For the azimuth element, we obtain ultra-high-resolution aerial RGB image data and LIDAR point cloud data in the wild that have been encrypted by aerial triangulation;

Copernicus digital elevation model data and synthetic aperture radar satellite data are obtained, and combined with the field ultra-high-resolution aerial RGB image data and the LIDAR point cloud data to form the first multi-source data set.

This preferred embodiment corrects field aerial collection data through image control point layout and aerial triangulation, ensuring the accuracy of field collection data.

In a specific implementation process, when laying out image control points, when restricted by terrain and other conditions, irregular area mesh points are used, and image control points should be laid out at the turning points of concave corners or convex corner turns. The span of image control points in the route direction is based on 12 baselines. When encountering special circumstances such as main points and standard points falling into the water, waterside and island areas, etc., and image control points cannot be laid out according to normal conditions, the span will be determined based on the specific circumstances. Image control points are laid out based on the principle of meeting aerial triangulation and mapping requirements. The VRS mode is preferred for image control point measurement. When the network RTK service cannot be received, the static measurement method based on GPS is used. The punctuation and retouching of image control points are all performed on the original image data, and PhotoShop software is used to add retouching information to the photo data.

In a specific implementation process, when laying the route, the height of the datum is determined based on the terrain and flight safety of the aerial photography area, and the spacing and flight direction of the route are set according to the basic parameter table involved in aerial photography technology, and the route generation software is used to automatically generate route.

In a specific implementation process, when performing aerial triangulation, through aerial photography data and field control point files, based on the principle of photogrammetry, control point encryption, aerial triangulation calculation and accuracy monitoring are performed indoors, and each calculation is performed simultaneously. The outer azimuth elements of the image are finally output to the wild ultra-high-resolution aerial RGB image data and LIDAR point cloud data that have been encrypted by aerial triangulation.

In a specific implementation process, when using a flying device for aerial photography, the flight should be as smooth as possible, and the yaw and yaw angles should not exceed the specification requirements. Field ultra-high-resolution aerial RGB should be carried out in accordance with the preset route and aerial photography quality requirements. Image data and airborne LIDAR point cloud data collection.

In some optional embodiments, the second multi-source data set includes canopy height model data CHM, digital orthophoto data, resampled Copernicus digital elevation model data, and resampled synthetic aperture radar satellite data;

The data preprocessing of the first multi-source data set includes:

For the LIDAR point cloud data:

Filter the noise points, ground points and non-ground points in the LIDAR point cloud data respectively, and then use the natural neighborhood interpolation method to extract the digital elevation model data and digital surface model data;

According to the difference between the digital elevation model data (DEM) and the digital surface model data (DSM), canopy height model data (CHM) is obtained;

For the ultra-high-resolution aerial RGB image data in the wild:

According to the Copernicus digital elevation model data and aerial triangulation results, the field ultra-high-resolution aerial RGB image data is resampled, and the central projection is converted into an orthographic projection according to the correction errors on a slice-by-slice and pixel-by-pixel basis. , obtain digital orthophoto data;

For the Copernicus digital elevation model data and the synthetic aperture radar satellite data:

The spatial interpolation method is used to resample the Copernicus digital elevation model data and the synthetic aperture radar satellite data, so that the resolutions of the Copernicus digital elevation model data and the synthetic aperture radar satellite data are consistent with the The data of ultra-high-resolution aerial RGB images in the wild are the same.

In this optional embodiment, by correcting the errors of the wild ultra-high-resolution aerial RGB image data on a slice-by-slice and pixel-by-pixel basis, errors caused by factors such as ground fluctuations and tilt of the aerial photography device can be reduced.

In some examples, 1m resolution digital elevation model (DEM), digital surface model (DSM) and canopy height model data (CHM) are produced based on acquired airborne LIDAR point cloud data.

In some examples, filtering noise points, ground points and non-ground points in the LIDAR point cloud data includes: filtering points or point groups (low points) that are significantly lower than the ground and those that are significantly higher than the ground. The target points or point groups (high points) are separated; the ground points and non-ground points of the point cloud are filtered.

In some examples, when producing digital orthoimage data, those skilled in the art can refer to "Basic Geographic Information Digital Results 1:500 1:1000 1:2000 Digital Orthoimage Map".

In some examples, bilinear interpolation is used to resample Copernicus digital elevation model data and synthetic aperture radar satellite data to generate a smoother surface; by doing one interpolation in the Y direction (or X direction), Then interpolate once in the X direction (or Y direction), and weight the distance from the sampling point to the surrounding 4 neighboring pixels to calculate the raster value of the pixel; based on the resolution of the aerial RGB image, the Copernicus Digital elevation model data and synthetic aperture radar satellite data are resampled to the same resolution as aerial RGB imagery.

Further, using the second multi-source data set to train the ultra-high-resolution vegetation canopy height estimation model includes:

The canopy height model data, resampled Copernicus digital elevation model data, and resampled synthetic aperture radar satellite data are overlaid and combined with the red, green and blue bands of the digital orthophoto data to obtain the original Feature combination image;

Use the canopy height model data as a label image and combine it with the original feature combination image into an image pair;

After performing supervised data enhancement on the image pairs, they are divided into a training set and a verification set;

Using the training set as the input of the encoder, the ultra-high resolution vegetation canopy height estimation model learns the multi-level features of the image on the training set, and uses the verification set to estimate the ultra-high height Parameter tuning of the resolution vegetation canopy height estimation model.

In this embodiment, the canopy height model data produced based on LIDAR point cloud data is used as label data for model training. Through supervised data enhancement, the soil relative input to the model has different combinations each time, increasing the The diversity of training sample data sets avoids overfitting in network training and increases the generalization ability of the model.

In some examples, the training and validation sets are split at a ratio of 70%:30%.

In some examples, the label image and the original feature combination image are respectively cropped into images of 128*128 size to form an image pair.

In some examples, during the training process of the ultra-high-resolution vegetation canopy height estimation model, the basic learning rate is set to 0.001, the learning rate decay rate is set to 0.7, and the learning rate is adaptively updated using the Adam random optimization method, The maximum iterations are set to 800.

Furthermore, the data enhancement includes geometric transformation, adding noise, filling, erasing and/or forming data pairs (SamplePairing).

In some optional embodiments, when aerial photography is used to collect initial aerial RGB image data and initial LIDAR point cloud data, the directional coverage of the photographic area boundary should exceed the photographic area boundary line by 3 baselines; the side coverage should exceed the photographic area boundary line. Route quality control is required to be 50% of the image frame, with a heading overlap of not less than 70% and a side overlap of not less than 55%.

In some preferred embodiments, establishing an initial super-resolution vegetation canopy height estimation model, see Figure 2, includes:

Establish an encoder based on the deep residual convolutional neural network described in any one of Resnet50, Resnet101 and Resnet152, and replace the classification module of the deep residual convolutional neural network with a convolutional layer with an output feature number of 256 , used to learn multi-level features of input images and output high-level feature results about vegetation canopy height;

Set the input feature number of the first convolution module of the deep residual convolutional neural network to 64, and embed an output feature number of 64 and a constant size 3 in front of the deep residual convolutional neural network. *3 convolution module, used to receive input images;

A decoder is established based on two FPN modules, two Transformer modules and multiple upsampling layers, and is used to output canopy height prediction results based on the extracted feature results of different residual modules and the high-level feature results as The output of the ultra-high resolution vegetation canopy height estimation model.

It should be noted that in this preferred embodiment, the image is sequentially upsampled 2 times through multiple upsampling layers, and finally a canopy height prediction result of the same size as the original input image can be obtained.

In a specific implementation process, an encoder was established based on ResNet101 to learn the multi-level features of the image. Through multiple convolution operations or maximum pooling operations, a high-dimensional image with a size of 1/32 of the original image but with deep features was obtained. image (i.e., high-level feature extraction results). Compared with ResNet101, ResNet50 has a smaller depth and the extracted deep features are not obvious enough, while the ResNet152 model has more network layers and requires more computing resources.

It should be noted that a new convolution module is embedded in front of the deep residual convolutional neural network so that the ultra-high-resolution vegetation canopy height estimation model can receive multi-band (not limited to three-band) image inputs , the number of output features is 64 and the size is consistent with the input image. The improved encoder can obtain low-medium-high-level features in the input image through nonlinear operations and multiple downsampling.

Those skilled in the art should understand that the Transformer module implements encoding and representation learning of the input sequence by using a self-attention mechanism to capture the dependencies between different positions in the sequence.

In some optional embodiments, the decoder is established based on two FPN modules, two Transformer modules and multiple upsampling layers, including:

After the second residual module and the third residual module of the deep residual convolutional neural network, the first FPN module and the second FPN module are respectively connected to perform convolution operations, and the encoder output The high-level feature results are upsampled and then added and upsampled to the output results of the two FPN modules to obtain a feature result with a feature number of 256;

Set up a first Transformer module and a second Transformer module, whose inputs are respectively the upsampling result after passing through the first FPN module and the upsampling result after passing through the second FPN module; let the high-level feature results output by the encoder be upsampled. After sampling, it is overlapped with the output of the two Transformer modules in sequence; an upsampling layer, a convolution layer and a Relu activation layer are set to process the overlay results to obtain a canopy height prediction result of the same size as the input image.

Those skilled in the art should understand that in the decoder part of this optional embodiment, for the high-level feature results output by the encoder part, the FPN module first performs a convolution operation on the feature results obtained by different residual modules to obtain 256 The feature results of the band are added and upsampled with the output results of the improved encoder; the Transformer module optimizes the high-level feature results that have passed through the FPN module; and then the high-level feature results that have gone through the Transformer module are Overlay with the upsampling result; finally, the overlay result is upsampled and passed through a convolution layer and ReLU activation layer to obtain a canopy height prediction result with the same size as the input image. It should be understood that when predicting the vegetation image of the target area, the output is the vegetation canopy height information of the target area.

In a specific implementation process, the test set was used to compare and evaluate the accuracy of the aforementioned ultra-high-resolution vegetation canopy height estimation model (ARFTNet) and the existing solutions UNet and SegNet. Specifically, the coefficient of determination (R ² ), average There are four accuracy evaluation indicators: root square error (RMSE), mean absolute error (MAE) and deviation value (B). The experimental data are shown in Table 1:

Table 1 Comparative experimental data

R ² can evaluate and verify the estimated vegetation canopy height and can express the credibility of the predicted tree height. RMSE and MAE are used to represent the deviation between the estimated value of the vegetation canopy height and the true value of the test set vegetation canopy height. Use the deviation value (Bias) to determine how much the vegetation canopy height information estimated by the model is overestimated or underestimated relative to the true height value of the vegetation canopy in the test set; among them, the higher R ² represents the better the model estimation results, RMSE, MAE and Bias The lower the value, the closer the model estimation results are to the real results.

It can be seen that ARFTNet's RMSE is lower than UNET and SegNet, its ^R2 is higher than UNET and SegNet, its MAE is lower than UNET and SegNet, and the absolute value of its Bias is also lower than UNET and SegNet. Overall, the ultra-high-resolution vegetation canopy height estimation model described above in this embodiment performs better and has higher accuracy.

Example 2

This embodiment proposes an ultra-high-resolution vegetation canopy height estimation system, applying the method described in Embodiment 1, see Figure 3, including:

The multi-source data acquisition module is used to obtain field ultra-high-resolution aerial RGB image data, LIDAR point cloud data, Copernicus digital elevation model data and synthetic aperture radar satellite data for the sample area, and combine them into the first multi-source data set; also used to perform data preprocessing on the first multi-source data set to obtain a second multi-source data set; and also used to obtain vegetation images for the target area;

Model training module, used to establish an initial ultra-high-resolution vegetation canopy height estimation model using a deep residual convolutional neural network as the encoder and FPN and Transformer as the decoder, and train using the second multi-source data set The ultra-high resolution vegetation canopy height estimation model;

A vegetation canopy height information estimation module is used to carry out training to complete the ultra-high-resolution vegetation canopy height estimation model; and is also used to estimate the ultra-high-resolution vegetation canopy height based on the vegetation image of the target area. The model obtains vegetation canopy height information in the target area.

It can be understood that the system of this embodiment corresponds to the method of the above-mentioned Embodiment 1, and the options in the above-mentioned Embodiment 1 are also applicable to this embodiment, so the description will not be repeated here.

Example 3

This embodiment proposes a computer-readable storage medium on which at least one instruction, at least one program, code set or instruction set is stored. The at least one instruction, at least one program, code set or instruction set is processed by The processor is loaded and executed, so that the processor executes some or all steps of the method described in Embodiment 1.

It can be understood that the storage medium may be transient or non-transient. Exemplarily, the storage medium includes but is not limited to U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc., which can store various data. The medium for program code.

Exemplarily, the processor may be a Central Processing Unit (CPU), a Microprocessor Unit (MPU), a Digital Signal Processor (DSP) or a Field Programmable Gate Array (Field Programmable Gate). Array, FPGA), etc.

In some examples, a computer program product is provided, which may be implemented by hardware, software, or a combination thereof. As a non-limiting example, the computer program product can be embodied as the storage medium, or can also be embodied as a software product, such as an SDK (Software Development Kit), etc.

In some examples, a computer program is provided, including computer readable code. When the computer readable code is executed in a computer device, a processor in the computer device executes a part for implementing the method. or all steps.

This embodiment also provides an electronic device, including a memory and a processor. The memory stores at least one instruction, at least a program, a code set or an instruction set, and the processor executes the at least one instruction, at least a program, A code set or instruction set implements part or all of the steps of the method described in Embodiment 1.

In some examples, a hardware entity of the electronic device is provided, including: a processor, a memory, and a communication interface; wherein the processor generally controls the overall operation of the electronic device; and the communication interface is used to enable the The electronic device communicates with other terminals or servers through the network; the memory is configured to store instructions and applications executable by the processor, and can also cache data to be processed or processed by the processor and each module in the electronic device (including but not Limited to image data, audio data, voice communication data and video communication data), it can be implemented through flash memory (FLASH) or random access memory (RAM, Random Access Memory).

Further, data can be transmitted between the processor, communication interface and memory through a bus. The bus can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories together.

It can be understood that the options in the above-mentioned Embodiment 1 are also applicable to this embodiment, so the description will not be repeated here.

The terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limitations to this patent;

Obviously, the above-mentioned embodiments of the present invention are only examples to clearly illustrate the present invention, and are not intended to limit the implementation of the present invention. It should be understood that in various embodiments of the present disclosure, the size of the serial numbers of the above steps/processes does not mean the order of execution. The execution order of each step/process should be determined by its functions and internal logic, and should not be The execution of the examples does not constitute any limitations. It should also be understood that the system/device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units. Or components can be combined, or can be integrated into another system, or some features can be omitted, or not implemented. In addition, the coupling, direct coupling, or communication connection between the various components may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be electrical, mechanical, or other forms. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. An exhaustive list of all implementations is not necessary or possible. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the claims of the present invention.

Claims (10)

1. A method for estimating the height of a canopy of vegetation at ultra-high resolution, comprising:

acquiring field ultra-high resolution aviation RGB image data, LIDAR point cloud data, golgi digital elevation model data and synthetic aperture radar satellite data aiming at a sample area, and combining the data into a first multi-source data set;

performing data preprocessing on the first multi-source data set to obtain a second multi-source data set;

establishing an initial ultra-high resolution vegetation canopy height estimation model by taking a depth residual convolutional neural network as an encoder and taking FPN and a transducer as decoders, and training the ultra-high resolution vegetation canopy height estimation model by using the second multi-source data set;

and acquiring a vegetation image aiming at the target area, and obtaining vegetation canopy height information of the target area by using the ultra-high resolution vegetation canopy height estimation model.

2. The method of claim 1, wherein the acquiring field ultra-high resolution aerial RGB image data, LIDAR point cloud data, cobweb digital elevation model data, and synthetic aperture radar satellite data for the sample area comprises:

arranging image control points in a region mesh point mode;

based on a preset route, acquiring initial aviation RGB image data and initial LIDAR point cloud data of a sample area in a aerial shooting mode;

according to the image control points, performing control point encryption, aerial triangulation operation and precision monitoring on the ultra-high resolution initial aviation RGB image data and the initial LIDAR point cloud data, and simultaneously calculating to obtain external azimuth elements of each image to obtain field ultra-high resolution aviation RGB image data and LIDAR point cloud data subjected to air-to-three encryption;

and acquiring the Goinby digital elevation model data and the synthetic aperture radar satellite data, and combining the Goinbred digital elevation model data, the outdoor ultra-high resolution aviation RGB image data and the LIDAR point cloud data into the first multi-source data set.

3. The ultra-high resolution vegetation canopy height estimation method of claim 2, wherein the second multisource dataset comprises canopy height model data, digital orthophoto data, resampled gothic digital elevation model data, and resampled synthetic aperture radar satellite data;

the data preprocessing of the first multi-source data set includes:

for the LIDAR point cloud data:

respectively carrying out filtering treatment on noise points, ground points and non-ground points in the LIDAR point cloud data, and then extracting digital elevation model data and digital surface model data by adopting a natural neighborhood interpolation method;

obtaining canopy height model data according to the difference value between the digital elevation model data and the digital surface model data;

for the field ultra-high resolution aviation RGB image data:

resampling the field ultra-high resolution aviation RGB image data according to the Goinby digital elevation model data and the blank three results, and converting the central projection into the orthographic projection according to correction errors of the pixels on a piece-by-piece basis to obtain digital orthographic image data;

for the cobicini digital elevation model data and the synthetic aperture radar satellite data:

and resampling the Golbini digital elevation model data and the synthetic aperture radar satellite data by adopting a spatial interpolation method, so that the resolution of the Golbini digital elevation model data and the synthetic aperture radar satellite data is the same as that of the outdoor ultra-high resolution aviation RGB image data.

4. A method of ultra-high resolution vegetation canopy height estimation according to claim 3, wherein the training the ultra-high resolution vegetation canopy height estimation model using the second multi-source dataset comprises:

overlapping and combining the canopy height model data, resampling the Goinbred digital elevation model data and resampling the synthetic aperture radar satellite data with red, green and blue wave bands of the digital orthophoto data respectively to obtain an original feature combined image;

combining the canopy height model data as a label image with the original feature combined image into an image pair;

after the image pair is subjected to supervised data enhancement, dividing the image pair into a training set and a verification set;

and taking the training set as the input of the encoder, enabling the ultra-high resolution vegetation canopy height estimation model to learn multi-level features of images on the training set, and performing parameter tuning on the ultra-high resolution vegetation canopy height estimation model by utilizing the verification set.

5. The ultra-high resolution vegetation canopy height estimation method of claim 4, wherein the data enhancement comprises geometric transformation, noise addition, padding, erasure, and/or data pair formation.

6. The ultra-high resolution vegetation canopy height estimation method according to claim 2, wherein when the initial aviation RGB image data and the initial LIDAR point cloud data are acquired by aerial photography, 3 baselines exceeding a border line of a photographed area are covered according to a border course of the photographed area; and the lateral coverage exceeds the shot boundary line by 50% of the image frame, the course overlap is not less than 70%, and the lateral overlap is not less than 55% for performing route quality control.

7. The method of any one of claims 1-6, wherein the establishing an initial ultra-high resolution vegetation canopy height estimation model comprises:

establishing an encoder based on the depth residual convolutional neural network of any one of Resnet50, resnet101 and Resnet152, replacing a classification module of the depth residual convolutional neural network with a convolutional layer with 256 output feature numbers, and learning multi-level features of an input image and outputting a high-level feature result about vegetation canopy height;

setting the input feature number of a first convolution module of the depth residual convolution neural network as 64, and embedding a 3*3 convolution module with the output feature number of 64 and unchanged size in front of the depth residual convolution neural network for receiving an input image;

and a decoder is established based on the two FPN modules, the two transducer modules and the plurality of upsampling layers and is used for outputting a canopy height prediction result as the output of the ultrahigh resolution vegetation canopy height estimation model according to the extracted characteristic results of different residual modules and the high-level characteristic results.

8. The method of claim 7, wherein the building a decoder based on two FPN modules, two transform modules, and a plurality of upsampling layers comprises:

the second residual error module and the third residual error module of the depth residual error convolution neural network are respectively connected with the first FPN module and the second FPN module to carry out convolution operation, and the high-level characteristic result output by the encoder is subjected to up-sampling and then sequentially added and up-sampled with the output results of the two FPN modules to obtain a characteristic result with 256 characteristic numbers;

setting a first transducer module and a second transducer module, wherein the inputs of the first transducer module and the second transducer module are respectively an up-sampling result after passing through the first FPN module and an up-sampling result after passing through the second FPN module; the high-level characteristic result output by the encoder is subjected to up-sampling and then sequentially overlapped with the output of the two transducer modules; and setting an upsampling layer, a convolution layer and a Relu activation layer to process the superposition result so as to obtain a canopy height prediction result with the same size as the input image.

9. A ultra-high resolution vegetation canopy height estimation system employing the method of any of claims 1-8, comprising:

the multi-source data acquisition module is used for acquiring field ultra-high resolution aviation RGB image data, LIDAR point cloud data, goinbred digital elevation model data and synthetic aperture radar satellite data aiming at a sample area, and combining the data into a first multi-source data set; the data preprocessing module is also used for carrying out data preprocessing on the first multi-source data set to obtain a second multi-source data set; and is also used for acquiring a vegetation image aiming at the target area;

the model training module is used for establishing an initial ultra-high resolution vegetation canopy height estimation model by taking a depth residual convolutional neural network as an encoder and taking FPN and a transducer as decoders, and training the ultra-high resolution vegetation canopy height estimation model by utilizing the second multi-source data set;

the vegetation canopy height information estimation module is used for carrying and training the ultra-high resolution vegetation canopy height estimation model; and the vegetation canopy height information of the target area is obtained by utilizing the ultra-high resolution vegetation canopy height estimation model according to the vegetation image of the target area.

10. A computer readable storage medium having stored thereon at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by a processor to implement the method of any of claims 1-8.