RU2707138C1 - Intelligent space system for monitoring buildings and structures - Google Patents

Abstract

FIELD: monitoring systems.

SUBSTANCE: invention relates to intelligent space monitoring system. System includes a set of computer devices structured on the basis of a convolutional neural network, associated with Earth remote sensing spacecrafts and providing the resulting images, generated based on images received from Earth remote sensing spacecraft, and Earth remote sensing data, wherein computer means structured on the basis of the convolutional neural network are connected to Earth remote probing apparatus of Resource-P type, convolution neural network processes images of building objects, received from spacecraft of remote sensing of the Earth of the specified type, with formation by means of a sliding window method of rectangular matrixes of pixels of an image with a given pitch – crops, generating a plurality of mappings for each rectangular matrix by rotating and mirroring each of the display segments by means of a convolutional neural network and classifying it using the convolutional neural network as one of the building objects.

EFFECT: remote probing of the Earth for monitoring buildings and structures.

1 cl, 3 dwg

Description

The present invention relates to the field of means and technologies for remote sensing of the Earth, and in particular to the application of the results of remote sensing of the Earth for monitoring buildings and structures.

Planung, Politik und

Copyright 2000 Metropolis Verlag, Marburg]. Способ включает обработку спутниковых снимков по различным алгоритмам и сочетание полученных результатов с данными GIS.The prior art method for processing satellite images [Thomas Blaschke, Stefan Lang, Eric Lorup, Josef Strobl and Peter Zeil. Object-Oriented Image Processing in an Integrated GIS / Remote Sensing Environment and Perspectives for Environmental Applications. EnviroInfo 2000: Umweltinformatik '00 Umweltinformation

Planung, politik und

Copyright 2000 Metropolis Verlag, Marburg]. The method includes processing satellite images by various algorithms and combining the results with GIS data.

As the closest analogue of the invention, a satellite-based system for acquiring and processing satellite images proposed in the application for the invention CN 107945146, Univ can be selected. Nanjing Information Science & Tech., 2018 publication. The system from CN 107945146 includes a set of computer tools structured on the basis of a convolutional neural network associated with LandSat Earth remote sensing spacecraft and Aqua research spacecraft with Modis Earth remote sensing equipment. The convolutional neural network provides the final images formed on the basis of images received from the spacecraft for remote sensing of the Earth, and refinement data of remote sensing of the Earth.

In turn, in our invention, it is proposed to use the technical capabilities of a convolutional neural network to solve a narrower problem - monitoring buildings and structures to control the construction of unauthorized construction objects, ensuring the speed of data processing and using fewer computing resources. We have proposed an intelligent space monitoring system that includes a set of computer tools based on a convolutional neural network that allows you to process data from Resurs-P Earth remote sensing spacecraft. The difference from the analogue lies in the structure of the neural network and the use of data from spacecraft for remote sensing of the Earth of the Resource-P type. A convolutional neural network processes images of building objects received from the Earth’s remote sensing spacecraft of the indicated type, with the help of a sliding window method of creating rectangular image pixel matrices with a given step - crop, generating several images for each rectangular matrix by rotating and mirroring with segmentation of each image using convolutional neural network and its classification by means of the convolutional neural network as one of the systems Yelnia objects and obtain the final image.

The proposed system is illustrated by the following images:

FIG. 1 is a diagram of an algorithm for recognizing and classifying building structures;

FIG. 2 - architecture of a convolutional neural network with a display of the encoding and decoding process;

FIG. 3 - splitting the original image into pieces of a fixed size using the sliding window method.

Satellite images of building structures obtained from Resurs-P spacecraft arrive at the input of a convolutional neural network (Fig. 1). Currently, satellite images are used in processing level 4A - a complex image of panchromatic (processing level 2A) and multispectral (processing level 2A1) images of the same territory. The use of a convolutional neural network in the proposed architecture will allow faster data processing due to fewer operations and time required to complete a single pass of the network, since the neural network is less demanding on technical resources, and also allows processing in close to real time mode. The use of Resurs-P spacecraft will make it possible to use high and ultra-high resolution Earth remote sensing data when monitoring building structures.

The architecture of the proposed convolutional network can be described (Fig. 2) as a set of sequences of coding and decoding blocks. Encoding blocks reduce the spatial resolution of the original image in this case, the resolution of the crop - rectangular matrix of image pixels. Decoding blocks increase spatial resolution by combining input data with feature maps obtained using the forwarding method from coding blocks of the corresponding resolution, which ensures the ensemble of the results of all layers and resolutions.

The encoding block is a set of three functional sub-blocks (operations on feature maps): a convolution layer, a ReLU activation function, a subsampling layer. The decoding unit is a set of four functional sub-blocks (a set of two sequences of two operations): a convolution layer, a ReLU activation function. The convolution layer includes its own filter for each channel, the convolution core of which processes the previous layer into fragments, summing up the results of the matrix product for each fragment, and the convolution is indicated ([k × k], m, n), where k × k is the size of the convolution core, m and n are the number of input and output filters for the layer, respectively. The activation function ReLU, which is a function max (0, x) ReLU = x, avoids the problems of a damping and exploding gradient, and is also computationally simple, the scalar result of each convolution falls on the activation function. The subsampling layer (subsampling layer) is a nonlinear compaction of the feature map, while a group of points (usually 2 × 2 in size) is condensed to one point through a nonlinear transformation. The maximum function is most commonly used. Transformations affect disjoint rectangles or squares, each of which is squeezed into one point, while the point with the maximum value is selected. The pooling operation (that is, if some features were already detected in the previous convolution operation, then such a detailed image is no longer needed for further processing, and it is compressed to a less detailed image and serves to generate new feature maps of a larger dimension), it allows to significantly reduce the spatial volume of the image .

The result decoding block is a set of four functional subblocks (sequential operations): spatial exclusion - turns off the neuron layer with probability p; a convolution layer with a core of 1 × 1 — necessary to reduce the dimension of the feature map; activation function - the scalar result of each convolution falls on the activation function, which is a non-linear function sigmoid = 1 / (1 + e ^-x ), which allows amplifying weak signals and not saturating from strong signals; linear increase in dimension is the inverse operation of subsampling, that is, linear repetition of the feature map, each point is converted to a group of 2 × 2 points, undergoing a linear transformation, the transformations affect all points, each of which turns into a group of points, while they have the same value, given operation allows you to increase the volume of the image.

To process the received satellite images by the method of a sliding window, cropes are formed - rectangular image pixel matrices (Fig. 3), the cropes are formed with a given step, for which the dimensions of the sliding window H _SW × W _SW are set, the step of the sliding window is set — S _H pixels vertically and S _W pixels horizontally; the original image I with a height H and a width W is supplemented at the edges to a multiple of the size of the sliding window; from the augmented image with a step S _H × S _W form a matrix of pixels - crop size H _SW × W _SW . It is proposed to use H _SW × W _SW = 512 pixels to ensure that, on the one hand, sufficiently large objects fall into one crop, which will achieve the given accuracy and practical feasibility of creating a software implementation, on the other hand, since the window size directly determines the size of the neural network layer, an increase in which leads to an increase in the requirements for memory size and hardware operation speed and an exponential increase in training time. It is also proposed to use the step S _H = H _SW / 2 = 256 pixels and S _W = W _SW / 2 = 256 pixels to ensure that the crop is superimposed so that the edge of one crop coincides with the center adjacent to it, which avoids conflicts at the edges of the crop when restoring the segmentation of the entire image from the segmentation of individual crop.

For each rectangular matrix - crop, several mappings are generated by rotation and mirror image of the original image. It is proposed to generate eight mappings per crop, that is, variations of the display of the original piece using the operations of rotation through the angle π / 2 and mirror image. That is, for the crop of the image C (ϕ (x) is the rotation through the angle π / 2, ψ (x) is the mirror image), it is fashionable to represent the set of eight maps O as O = {C, ϕ (C), ϕ (ϕ (C)), ϕ (ϕ (ϕ (C))), ψ (C), ψ (ϕ (C)), ψ (ϕ (ϕ (C))), ψ (ϕ (ϕ (ϕ (C)) )))}. Using the convolutional neural network, a map of segments is obtained for each display, with subsequent classification as one of the objects of building structures, for which the source crop C is passed through the neural network, the final prediction function is P = h (C) for the crop image C. As a result, the output of the convolutional neural network segmentation results in a set of eight (for the considered example) probability maps that each point of the original crop can be classified as changes in building structures .

Inverse probability mapping operations (rotation by - π / 2 and mirror image) are applied to the obtained set of probability maps in order to obtain the inverse images of the images used in relation to the output probability maps. For x = ψ (ψ (x)) and x = ϕ ^-1 (x) = ϕ (ϕ (ϕ (x))), the desired set О _p has the form О _р = {Р ₁ , ϕ ^-1 (ϕ ^{- 1} (ϕ ^-1 ) (P ₂ ))), ϕ ^-1 (ϕ ^-1 (P ₃ )), ϕ ^-1 (P ₄ ), ψ (P ₅ ), ψ (ϕ ^-1 (ϕ ^-1 ) (ϕ ^-1 ) (P ₆ )))), ψ (ϕ ^-1 (ϕ ^-1 ) (P ₇ ))), ψ (ϕ ^-1 (P ₈ ))}. Using the averaging of predictions, the boundaries of the obtained segments are specified. The final prediction for each point of the crop of the image for the proposed option is calculated by the following formula P _total (i, j) = Σ _{k = 1} ⁸ P _k (i, j) / 8. This approach allows us to improve the segmentation results obtained in the previous step. Combining the obtained intersecting prediction maps using the weighted sum, using the two-dimensional Gaussian distribution with weights as zero in the center of the crop and the standard deviation σ = H _SW / 2⋅3 = ~ 85 calculated at the points corresponding to the centers of the pixels of the crop to obtain segmentation of the original image . This will eliminate conflicts and artifacts at the borders of the crop, since for each pixel the largest contribution to its value will be made by the crop, to the center of which it is closer, and the contribution of the extreme points of the crop will be ~ 0.2.

Claims (7)

Intelligent space monitoring system, including a set of computer tools, structured on the basis of a convolutional neural network, connected with spacecraft for remote sensing of the Earth and providing final images formed on the basis of images received from the spacecraft for remote sensing of the Earth, and refining data of remote sensing of the Earth, characterized in that computer facilities, structured on the basis of a convolutional neural network, are connected with spacecraft of remote sensing of the Earth of the Resource-P type, while a convolutional neural network processes images of building objects received from spacecraft for remote sensing of the Earth of the indicated type, with the formation by the method of a sliding window of rectangular matrices of image pixels with a given step - crop, generating several mappings for each rectangular matrix by rotating and mirroring with segmentation of each display using the convolutional neural network and its classification by means of a convolutional neural network as one of the construction objects.

Images