JP2018194404A - Machine learning method, and surface deformation determination method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automate the determination of the presence or absence of a surface change based on an interference SAR image. SOLUTION: A component image generation means 221 generates n kinds (n ≧ 2) of component images having pk (θ) (1 ≦ k ≦ n) as a pixel value from a phase difference θ of each pixel of an interference SAR image. do. The learning data generation means 222 generates learning data by combining n kinds of component images and a reading result acquired from a user for a plurality of sample areas. The model generation means 223 generates a fluctuation determination model by machine learning using the learning data. Each function pk (θ) is continuous with D, with the interval D of the width 2πrad corresponding to the range of the phase difference as [α−π, α + π], and pk (α−π) = pk (α + π), and further. When the coordinates defined by the set of pk (θ) for all k are expressed as P (θ), P (θ1) ≠ P (θ1) ≠ P for ∀θ1, θ2 ∈ D such that θ1 ≠ θ2 other than both ends of D. θ2). [Selection diagram] Fig. 1

Description

  The present invention generates a model for use in a system for determining ground surface fluctuations based on interference SAR images generated by interfering two SAR complex images obtained by a synthetic aperture radar (SAR). The present invention relates to a machine learning method and a ground surface fluctuation determination method using the model.

  Interferometric SAR (Interferometric SAR) can measure the displacement of the ground surface that is smaller than the wavelength of the radar used therefor, and is a useful analysis method for monitoring ground surface fluctuation in millimeters, for example. The analysis result by the differential interference SAR analysis is generally represented by an interference image (interferogram) in which the phase difference is indicated by rainbow stripes, and the person visually observes the interference image, for example, the surface of the ground such as landslide. Interpret fluctuations.

JP 2004-309178 A Special table 2003-500658 gazette

  In order to extract the desired variation fringes due to surface fluctuations in the interference image, it is necessary to have the skill of a reader. Therefore, it is not easy to secure human resources for interpretation, and the cost and processing time required for interpretation increase. Further, in visual interpretation, the extraction criteria must be qualitative, so that the interpretation results tend to vary, and it is not easy to ensure quality. And the above issues make it difficult to monitor surface fluctuations over a wide area.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a machine learning method and a ground surface fluctuation determination method capable of automating the determination of the presence or absence of ground surface deformation based on an interference SAR image. To do.

(1) The machine learning method according to the present invention is based on the phase difference θ _S of each pixel of an interference SAR image obtained by performing interference processing between two SAR images obtained by a synthetic aperture radar mounted on a flying object. A component image generating step for generating n types (n is a predetermined number of 2 or more) of component images having pixel values defined, and obtaining a user's interpretation result regarding a predetermined ground surface fluctuation, A learning data generation step for generating learning data combining the n types of component images and the interpretation results for the sample region, and a model generation step for generating a variation determination model by machine learning using the learning data The component image of the k-th (k is an arbitrary natural number less than or equal to n) has a pixel value as a value at θ = θ _S of the function p _k (θ), and each of the functions p _k (Θ) is the phase difference A section D having a width of 2π radians corresponding to the range of [α−π, α + π] is continuous in D and p _k (α−π) = p _k (α + π), and the above p _k for all _k When the coordinates defined by the set of (θ) are expressed as P (θ), P (θ ₁ ) ≠ P (with respect to ∀θ ₁ and θ ₂ ∈D where θ ₁ ≠ θ ₂ other than both ends of D. θ ₂ ).

  (2) In the machine learning method according to (1), the n may be 2, and the function may be sin θ and cos θ.

  (3) In the machine learning method according to (1) or (2) above, the variation determination model can be configured by a convolutional neural network.

  (4) In the machine learning method according to (1) to (3) above, the learning data may further include a map image representing the topography of the sample area.

(5) A ground surface fluctuation determination method according to the present invention determines a ground surface fluctuation in a target region using the fluctuation determination model generated by the machine learning method according to any one of (1) to (4) above. In this method, a phase difference θ _T at each pixel is extracted from the interference SAR image generated for the target region, and the value p _k (θ _T ) (k is an arbitrary natural number less than or equal to n. .) Is generated as a pixel value, and input data including the n types of component images for the target area is input to the fluctuation determination model, and the fluctuation for the target area is input. Obtaining a determination result of the presence or absence of.

  According to the present invention, it is possible to automate the determination of the presence or absence of ground surface fluctuation based on the interference SAR image.

1 is a block diagram showing a schematic configuration of a learning system according to an embodiment of the present invention. It is a schematic diagram which shows the general | schematic flow of the machine learning of the ground surface fluctuation | variation determination model in embodiment of this invention. It is a graph which shows the other example of the conversion function which produces | generates a component image from an interference SAR image. 1 is a block diagram showing a schematic configuration of a ground surface fluctuation determination system according to an embodiment of the present invention. It is a schematic diagram which shows the general | schematic flow of the fluctuation | variation determination process of the ground surface in embodiment of this invention.

  Hereinafter, a learning system 2 and a ground surface fluctuation determination system 4 which are embodiments of the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings. The learning system 2 is a system that generates a variation determination model used for the ground surface variation determination system 4 by the machine learning method according to the present invention based on phase difference information obtained from an interference SAR image. On the other hand, the ground surface fluctuation determination system 4 is a system for determining the ground surface fluctuation in the target region using the fluctuation determination model generated by the learning system 2 by the ground surface fluctuation determination method according to the present invention. With this system, for example, surface fluctuations due to landslides or land subsidence can be determined and detected.

[Learning system]
FIG. 1 is a block diagram illustrating a schematic configuration of a learning system 2 according to the embodiment. The learning system 2 includes a learning operation unit 20, a learning storage unit 21, a learning control unit 22, and a learning output unit 23. The learning operation unit 20, the learning storage unit 21, and the learning output unit 23 are connected to the learning control unit 22.

  The learning operation unit 20 is a user interface device for inputting to the learning control unit 22 and includes a keyboard, a mouse, and the like. The learning operation unit 20 is operated by the user, and inputs the result of interpretation of whether or not the sample area has changed to the learning control unit 22 and also gives the learning control unit 22 an instruction to start learning and an instruction to output the generated variation determination model.

  The learning storage unit 21 is a storage device such as a ROM, a RAM, and a hard disk, and stores programs and data used by the learning control unit 22. The learning storage unit 21 inputs and outputs these programs and data to and from the learning control unit 22. In the present embodiment, the data stored in the learning storage unit 21 includes interference SAR image data 211, terrain image data 212, interpretation results (GT data) 213, and a variation determination model 214.

  The interference SAR image data 211 is data of an interference SAR image generated by interfering two SAR complex images (SAR images) obtained by the synthetic aperture radar mounted on the flying object.

  The terrain image data 212 is two-dimensional information representing terrain such as a digital elevation model (DEM) and a topographic map.

  The interpretation result 213 is correct data (Ground Truth: GT), and is generated, for example, by the user interpreting the interference SAR image data 211.

  The interference SAR image data 211 and the topographic image data 212 are stored in advance in the learning storage unit 21 when learning is performed by the learning system 2. In this embodiment, the interpretation result 213 is acquired by the learning data generation process in the learning system 2. However, the interpretation result 213 may be acquired prior to the learning process by the learning system 2 and stored in the learning storage unit 21. Good.

  The variation determination model 214 is a learning model generated by the learning system 2, and is updated in accordance with the learning control unit 22 sequentially processing learning data in a plurality of sample regions by machine learning.

  The learning control unit 22 is configured using an arithmetic device such as a CPU (Central Processing Unit), for example. In addition, instead of the CPU, the arithmetic device constituting the learning control unit 22 may use an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit) that executes image processing at high speed, or the like. For example, each function according to the present embodiment may be realized by using GPGPU (General-Purpose computing on Graphics Processing Units), which is a technique for diverting the GPU function to applications other than image processing. Specifically, the learning control unit 22 is a computer, and the computer reads out and executes a program from the learning storage unit 21, and functions as a component image generation unit 221, a learning data generation unit 222, and a model generation unit 223, which will be described later. . Each means of the learning control unit 22 will be described later along the processing of the learning control unit 22.

  The learning output unit 23 includes a USB terminal for outputting a variation determination model generated by learning to the outside of the learning system 2, an interface circuit such as a CD drive and a network adapter, and respective driver programs. In the present embodiment, the variation determination model is passed to the ground surface variation determination system 4 via the learning output unit 23. The learning output unit 23 may include a user interface device such as a display or a printer that allows the user to grasp the operation of the learning control unit 22 and the result thereof.

  FIG. 2 is a schematic diagram showing a general flow of machine learning of the ground surface variation determination model. An interference SAR image is generated by the interference SAR analysis processing 304 from the SAR images 300 at the two periods, and is stored in the learning storage unit 21 as interference SAR image data 211.

  Here, in the interference SAR analysis process 304, it is possible to perform a process of removing influences other than the phase change (fluctuation fringes) due to the fluctuation of the ground surface from the interference SAR image. Specifically, in addition to the fluctuation fringes, the interference SAR image is superimposed with phase changes (orbital fringes and terrain fringes) that occur because the orbits of the two satellites do not coincide. Therefore, when the ground surface fluctuation is obtained by the interference SAR, a process of removing only the fluctuation fringes by removing the orbital stripes and the topographic fringes is performed. For example, terrain fringes can be simulated and removed from the DEM, and for this purpose, the DEM 302 is used in the interference SAR analysis process 304.

  Further, processing for further removing errors and noise from the interference SAR image, such as removing a global error using a removal filter or the like, may be performed.

The learning control unit 22 functions as the component image generation unit 221 and has n types (n is a predetermined number of 2 or more) in which pixel values are defined based on the phase difference θ _S of each pixel of the interference SAR image data 211. Are generated and passed to the learning data generation means 222. In the present embodiment, the component image generation unit 221 performs processing 306 for converting the phase difference θ _S into sin θ _S and cos θ _S, and uses the converted interference image 308 as a component image. Specifically, a sin image pixel value is defined by sin [theta _S, the two component images with cos image pixel value is defined by cos [theta] _S is generated.

  The learning control unit 22 functions as the learning data generation unit 222, and performs a process 310 for cutting out a learning patch image for each of a plurality of small regions set in the region where the interference SAR image data 211 is obtained. Data 312 is generated.

  Here, the small area where the learning patch image is cut out is referred to as a sample area. The sample area is set by a user, for example. Incidentally, the number of sample regions is set to a relatively large number so that a variation determination model with sufficient accuracy can be obtained by learning. Basically, the size and shape of each sample area are the same.

  The learning data 312 is generated for each sample region by combining the converted interference image 308, which is n types of component images, and the user's interpretation result regarding the presence or absence of ground surface fluctuation. For this purpose, the learning data generation means 222 performs a process 314 for acquiring the interpretation result. Specifically, as the user interpretation process 314, for example, the entire area of the interference SAR image data 211 is displayed on the display of the learning output unit 23 to allow the user to interpret whether there is a change or not, and the interpretation result 213 from the learning operation unit 20. To get. In the interpretation processing 314, in addition to the interference SAR image data 211, topographic image data 212, other data obtained by a high-precision sensor such as LiDAR (Light Detection and Ranging), and local GPS (Global Positioning System) are used. Data obtained by the measurement used can be presented to the user as data 316 for determination material. These data can be subjected to the interpretation process 314 in combination, for example, by combining the interference SAR image 211 and the terrain image data 212 or the like. In addition, when measurement data higher than the interference SAR can be obtained by LiDAR or GPS, the interpretation result 213 can be generated by using the data alone for the interpretation process 314. For example, the user marks a portion with fluctuation while generating an interpretation result (GT data) while performing operations such as enlargement / reduction of the display magnification of these data. The generated interpretation result 213 is stored in the learning storage unit 21.

  When the interpretation result 213 is obtained, the learning data generation unit 222 uses the converted interference image 308 that is n types of component images and the interpretation result 213 obtained in the area corresponding to the interference SAR image data 211. Data for a portion corresponding to each sample area is cut out as a patch image, and learning data 312 is generated. Note that the learning data 312 can be combined with other information useful for fluctuation analysis. For example, in the present embodiment, the patch data of the topographic image data 212 is included in the learning data 312.

  The learning control unit 22 functions as the model generation unit 223 and generates the variation determination model 214 by machine learning using the learning data 312 generated for a large number of sample regions. Specifically, the machine learning in the present embodiment is performed by deep learning 318 using a convolutional neural network (CNN), and the learned CNN becomes the variation determination model 214. In the CNN, the sample area is basically rectangular.

  One of the features of the learning system 2 is that the interference SAR image data 211 is not used for machine learning as it is, but the preprocessing for converting the interference SAR image data 211 into a component image is performed, and the learning process is performed using the component image. There is in point to do.

Here, a function for converting a phase difference θ at each pixel value of the interference SAR image data 211 into a pixel value of a k-th component image (k is an arbitrary natural number equal to or less than n) is expressed as p _k (θ). In the above-described embodiment, p ₁ (θ) = sin θ when the sin image is the first component image, and p ₂ (θ) = cos θ where the cos image is the second component image.

Two phase states eta ₁ difference is 2π radians (rad) between the phase theta ₁ and theta _2, where the eta ₂ are physically the same, in the learning using the interference SAR image data 211, theta ₁ and theta _{Since 2} is a different value, the phase states η ₁ and η ₂ are distinguished from each other. On the other hand, in learning using component images, the set of function values (sin θ, cos θ) associated with the phase difference θ is (sin θ ₁ , cos θ ₁ ) = (sin θ ₂ , cos θ ₂ ), and the phase state It is preferable in that η ₁ and η ₂ are considered to be physically the same, and the determination accuracy is improved in the variation determination model generated by the learning.

The function p _k (θ) of component images of n types (n ≧ 2 as described above) that can obtain the effect of such preprocessing basically has the following features (1) to (3). Here, a range [α−π, α + π] having a width 2πrad that defines the phase difference θ is defined as a section D. For example, when θ is defined by 0 to 2π, α = π and D = [0,2π], and when θ is defined by −π to π, α = 0 and D = [ −π, π].

(1) Each function p _k (θ) is continuous in section D (continuity of function)
(2) Each function p _k (θ) has the same value at both ends of the section D, that is, p _k (α−π) = p _k (α + π) (periodicity of the function).
(3) When the coordinate defined by the set of p _k (θ) for all k is expressed as P (θ), P (θ) for different values of θ in the section D except for both ends of the section D ) do not coincide, i.e. θ ₁ = α-π cutlet except where _{θ 2 = α + π θ 1} <θ 2 becomes ∀θ _1, P with respect to _{θ 2 ∈D (θ 1) ≠} P (θ 2 ) (Injectivity of the function).

The feature (2) is described as “periodicity” in the sense that the value of p _k (θ) returns to the original value when θ changes from one end of the section D to the other end. Therefore, if the function p _k (θ) is a periodic function with a period of 2π, it has the characteristics. On the other hand, the function p _k (θ) only needs to be defined in the section D, and need not be defined outside the section D. The function p _k (θ) may not be a periodic function having a period of 2π.

FIG. 3 is a graph showing an example of the function p _k (θ) having the above features (1) to (3), and n types of component images are obtained using the set of the function p _k (θ) shown in the example. It can also be defined. The example shown in FIG. 3A is a case where n = 2, and p ₁ (θ) and p ₂ (θ) are expressed by the following equations. Note that 0 <β <π.

FIG. 3B shows an example where n = 3, and p ₁ (θ) to p ₃ (θ) are expressed by the following equations.
p ₁ (θ) = sin 2θ
p ₂ (θ) = sin 3θ
p ₃ (θ) = cos θ

  In the above-described embodiment, the learning data includes the topographic image data 212 in addition to the n types of component images and the interpretation results. This configuration can be expected to improve the accuracy of the generated variation determination model. On the other hand, the learning data may be configured not to include the terrain image data 212.

  In the above-described embodiment, an example in which machine learning uses CNN has been described, but another algorithm may be used.

[Surface variation judgment system]
FIG. 4 is a block diagram illustrating a schematic configuration of the ground surface fluctuation determination system 4 according to the embodiment. The ground surface fluctuation determination system 4 includes an input unit 40, a storage unit 41, a control unit 42, and an output unit 43. The input unit 40, the storage unit 41, and the output unit 43 are connected to the control unit 42.

  The input unit 40 is a user interface device for performing input to the control unit 42, and includes a keyboard, a mouse, and the like. The user operates the input unit 40 to specify, for example, data to be subjected to variation determination and a target area and instruct the control unit 42. The input unit 40 includes a USB terminal for inputting the variation determination model 214 from the learning system 2, an interface circuit such as a CD drive and a network adapter, and respective driver programs.

  The storage unit 41 is a storage device such as a ROM, a RAM, and a hard disk, and stores programs and data used by the control unit 42. The storage unit 41 inputs and outputs these programs and data to and from the control unit 42. In the present embodiment, the data stored in the storage unit 41 includes interference SAR image data 411, terrain image data 412, and a variation determination model 413.

  Like the interference SAR image data 211 used in the learning system 2, the interference SAR image data 411 is data of the interference SAR image, and is data of the ground surface range including the target region for which the variation determination is performed.

  Similar to the terrain image data 212 of the learning system 2, the terrain image data 412 is two-dimensional information representing the terrain such as a DEM or a topographic map, and is data of the ground surface range including the target region for which the change determination is performed.

  The interference SAR image data 411 and the terrain image data 412 are stored in the storage unit 41 in advance when the ground surface variation determination system 4 determines the variation.

  The variation determination model 413 is a learning model generated by the learning system 2, and introduces and uses the variation determination model 214 stored in the learning storage unit 21.

  The control unit 42 is configured using an arithmetic device such as a CPU, for example. In addition, an arithmetic unit constituting the control unit 42 may use an MPU, a GPU, or the like instead of the CPU. Moreover, you may implement | achieve each function which concerns on this embodiment, for example using GPGPU. Specifically, the control unit 42 is a computer, and the computer reads out and executes a program from the storage unit 41 and functions as a component image generation unit 421 and a determination unit 422 to be described later. Each means of the control unit 42 will be described later along the processing of the control unit 42.

  The output unit 43 is a user interface device such as a display or a printer that allows the user to grasp the operation of the control unit 42 and the result thereof.

  FIG. 5 is a schematic diagram showing a general flow of the ground surface fluctuation determination process using the ground surface fluctuation determination system 4. The process of generating the interference SAR image data 411 by the interference SAR analysis process 504 from the two-time SAR image 500 and the DEM 502 is basically a process of generating the interference SAR image data 211 described with reference to FIG. The same can be done. In the present embodiment, the ground surface fluctuation determination system 4 is assumed to receive the interference SAR image data 411 generated in this way as processing target data. However, the ground surface fluctuation determination system 4 uses the interference SAR image data 411 from the SAR image 500. You may perform the process which produces | generates.

The control unit 42 functions as the component image generation unit 421 and generates n types of component images in which pixel values are defined based on the phase difference θ _T of each pixel of the interference SAR image data 411. The processing content in the component image generation unit 421 is the same as the processing content of the component image generation unit 221 of the learning control unit 22 that has generated the variation determination model 413. That is, the conversion process (function p _k (θ)) for the phase difference θ and the number n of component image types when generating the component image in the component image generating unit 421 are the same as those of the learning system 2 that generated the variation determination model 413. This is common to the component image generation means 221. In this embodiment, component image generating means 221 sinθ phase difference theta _{S S,} corresponding to the process 306 of FIG. 2 to be converted to cos [theta] _S, component image generating means 421 phase difference theta _T SAR interferometry image data 411 the sin [theta _T, performs the process 506 of converting the cos [theta] _T, as converted interference image 508 serving as a component image, and sin image pixel value is defined by sin [theta _T, a cos image pixel value is defined by cos [theta] _T These two types are generated.

  The component image generation unit 421 performs a process 510 of cutting out a portion of a target region for determining whether or not there is a change from the component image (converted interference image 508) as a patch image 512. Here, when the terrain image data 212 is included in the learning data in the learning system 2, a patch image 512 for the terrain image data 412 is also generated in the clipping process 510. Regarding the designation of the target area, for example, the user can designate a specific location within the ground surface range where the interference SAR image data 411 is given, and the control unit 42 sequentially designates the target area for the entire ground surface range. You may perform the process to do.

  The control unit 42 functions as the determination unit 422 and determines whether or not there is a change in the predetermined ground surface in the target area. Specifically, the determination unit 422 performs a determination process 514 using the patch image 512 of the target region as input data of the variation determination model 413, determines the presence / absence of landslide that is a variation to be determined based on the output, for example, A landslide candidate location map 516 is generated.

  2 learning system, 4 ground surface fluctuation determination system, 20 learning operation unit, 21 learning storage unit, 22 learning control unit, 23 learning output unit, 40 input unit, 41 storage unit, 42 control unit, 43 output unit, 211, 411 interference SAR image data, 212, 412 Topographic image data, 213 Interpretation result, 214, 413 Variation determination model, 221, 421 Component image generation means, 222 Learning data generation means, 223 Model generation means, 422 determination means.

Claims (5)

N types (n is defined as a pixel value) based on a phase difference θ _S of each pixel of an interference SAR image obtained by performing interference processing between two SAR images obtained by a synthetic aperture radar mounted on a flying object. A component image generation step of generating a component image of 2).
A learning data generation step of acquiring a user's interpretation result for a predetermined ground surface fluctuation, and generating learning data combining the n kinds of component images and the interpretation result for a plurality of sample regions;
A model generation step of generating a variation determination model by machine learning using the learning data,
In the k-th component image (k is an arbitrary natural number equal to or less than n), the value at θ = θ _S of the function p _k (θ) is a pixel value,
Each function p _k (θ) is continuous in D, with a section D having a width of 2π radians corresponding to the range of the phase difference as [α−π, α + π], and p _k (α−π) = p _k ( α + π), and the coordinates defined by the set of p _k (θ) for all k are represented as P (θ), and θ ₁ ≠ θ ₂ other than both ends of D ∀θ ₁ , θ ₂ P (θ ₁ ) ≠ P (θ ₂ ) for ∈D,
A machine learning method characterized by The machine learning method according to claim 1,
The machine learning method, wherein n is 2 and the functions are sin θ and cos θ. In the machine learning method according to claim 1 or 2,
The machine learning method, wherein the variation determination model is configured by a convolutional neural network. The machine learning method according to any one of claims 1 to 3,
The machine learning method, wherein the learning data further includes a map image representing the topography of the sample area. A method for determining a ground surface variation in a target region using the variation determination model generated by the machine learning method according to any one of claims 1 to 4.
A phase difference θ _T at each pixel is extracted from the interference SAR image generated for the target region, and the value p _k (θ _T ) of the function (k is an arbitrary natural number equal to or less than n) is used as a pixel value. Generating the n types of component images,
Inputting input data including the n types of component images for the target region into the variation determination model to obtain a determination result of the presence or absence of the variation for the target region;
A method for determining ground deformation, comprising:

Images