JP2017220058A - Surface information analysis system and surface information analysis method - Google Patents

Abstract

A system for calculating a change amount for each substance is provided. A terrestrial surface information analysis system (100) for analyzing a terrestrial surface from an image acquires data of a two-dimensional image including information on an area of a material on the terrestrial surface at a plurality of times, and includes shape information of the terrestrial surface at a plurality of times. Acquires data of a two-dimensional image, identifies regions of each of a plurality of types of substances using the acquired data of a two-dimensional image, and identifies regions and shapes corresponding to each of times included in a plurality of times. The data of the time-series multispectral three-dimensional image is generated as a combination of a plurality of times and a plurality of types of materials from the regions of the obtained materials and the shape specified by the obtained data of the three-dimensional image. The amount of change of each substance is calculated based on the data at a plurality of times included in the data of the time-series multispectral three-dimensional image. [Selection diagram] Fig. 1

Description

  The present invention relates to a surface information analysis system and a surface information analysis method.

  There is a need for mining companies to pile up and store excavated minerals and fuels and to grasp the volume and changes in detail. On the other hand, conventionally, deposits were manually identified and their volumes were measured. There is also a need for mining companies to know the road surface in detail in order to efficiently monitor and maintain transportation roads. Conventionally, the road surface has been manually evaluated. In such a manual operation, it is difficult to target a large area, and there is a demerit that accuracy varies depending on the operator.

As a technique related to this, Patent Document 1 discloses that an imaging process for imaging a surface layer portion of the waste T supplied and deposited in the hopper 5 provided in the garbage processing furnace in a stereo view from an obliquely upward direction, and an imaging process. Based on the obtained stereo vision image, the three-dimensional shape detection step for obtaining the three-dimensional shape of the surface portion of the waste T, and the three-dimensional shape of the surface portion of the waste T obtained in the three-dimensional shape detection step The volume of the garbage T from the columnar model conversion process for converting into a columnar model in which a plurality of different columnars are gathered and the columnar model height change in the columnar model obtained in the columnar model conversion process And a volume change amount calculating step for obtaining a change ”.

JP 2014-126321 A

  If the technique described in Patent Document 1 is used, it is possible to calculate the volume change amount of garbage. However, there is no description of a technique that can only calculate the volume change amount of the entire garbage, and that can calculate the amount related to the change of each substance such as the volume change amount of each substance.

  Therefore, an object of the present invention is to calculate the amount of change for each substance.

  A representative surface information analysis system according to the present invention is a surface information analysis system for analyzing a ground surface from an image, acquires two-dimensional image data including information on a region of a material on the surface at a plurality of times, and a plurality of times 3D image data including the shape information of the ground surface is acquired, and the acquired 2D image data is used to identify areas of a plurality of types of substances, corresponding to each of the times included in the plurality of times. A time-series multispectral 3D image as a combination of a plurality of times and a plurality of types of materials, based on the region and shape of each identified substance and the shape specified by the acquired 3D image data The data of each substance is generated based on the data at a plurality of times included in the generated time-series multispectral 3D image data. And calculating the amount.

  According to the present invention, the amount of change for each substance can be calculated.

It is a block diagram which shows the example of a structure of a surface information analysis system. It is a block diagram which shows the example of the hardware of a surface information analysis system. It is a flowchart which shows the example of a process of a surface information analysis system. It is a figure which shows the example of a ground coefficient calculation. It is a figure which shows the example of ground estimation. It is a figure which shows the example of time series multispectral three-dimensional information image data. It is a figure which shows the example of a deposition model. It is a flowchart which shows the example of sediment volume calculation. It is a figure which shows the example of a variation | change_quantity image. It is a flowchart which shows the example of a maintenance determination.

  Hereinafter, preferred embodiments will be described with reference to the drawings.

  The ground surface information analysis system described below calculates the volume of stock pile deposits where minerals or fuel from a mine, which is the target area for acquiring high-resolution remote sensing image data, is piled up on the ground, And a deposit change amount image. Also, the road surface state of the acquisition target area is output as an image. As described above, the target for which the high-resolution remote sensing image data is acquired and analyzed is, for example, a mine deposition site, but may be another location.

  FIG. 1 is a block diagram illustrating a configuration example of the ground surface information analysis system 100. The surface information analysis system 100 includes a remote sensing image data acquisition unit 102 that acquires remote sensing image data 101a, and a three-dimensional information image data acquisition unit 103 that acquires three-dimensional information image data 101b. These image data may be of a plurality of types. These image data are acquired as time-series data.

  The ground surface information analysis system 100 includes a database generation unit 104 that generates a database 115 from the acquired image data. The database 115 manages time-series image data and also manages data obtained by processing image data described below. The data management format of the database 115 is not limited to a specific format, and may be a format used in general database products.

  The ground surface information analysis system 100 includes a material classification unit 105 that classifies the material of the deposit from the acquired image data, a ground coefficient calculation unit 106 that calculates a coefficient representing the ground condition, and a ground estimation unit 107 that estimates the altitude of the ground. Is provided. When a plurality of deposits and a plurality of ground standards are used as spectra, for example, when a spectrum such as the state of deposits is represented in a time series in a three-dimensional image, a time-series multispectral three-dimensional image is generated. Therefore, the time-series multispectral three-dimensional image generation unit 108 is configured.

  The surface information analysis system 100 includes a deposition model generation unit 109 that generates a deposition model according to the material because the deposition method varies depending on the material of the deposit, and the deposition that calculates the volume of the modeled sediment. An object volume calculation unit 110 is provided. In addition, a change amount image generation unit 111 that generates a change in the calculated volume of each deposit and a change in the state of the ground as an image is provided.

  Furthermore, the ground surface information analysis system 100 displays a maintenance determination unit 112 that determines whether maintenance is necessary based on the situation of the ground, when it is determined that maintenance is necessary, or based on a change in the volume of the deposit. A control unit 113 that supports mining and construction by controlling equipment and an image update unit 114 that newly acquires image data are provided.

  FIG. 2 is a block diagram illustrating a hardware example of the ground surface information analysis system 100. The ground surface information analysis system 100 may be a general computer, a CPU (Central Processing Unit) 201 for executing processing, a RAM (Random Access Memory) 202 for storing programs and data to be processed by the CPU, An I / F (Interface) 203 connected to the remote sensing observation device 211 and the surface database 212 and used to acquire the remote sensing image data 101a and the three-dimensional information image data 101b is provided.

  The ground surface information analysis system 100 also includes a storage unit 204 that is a hard disk drive (SSD) or a flash memory having a capacity larger than that of the RAM 202, a liquid crystal display that prompts input, and displays processing results. A display unit 205 such as a display and an input unit 206 such as a keyboard and a mouse are provided. The data read from the RAM 202 may be written to the storage unit 204 by the operation of the CPU 201, or the data read from the storage unit 204 may be written to the RAM 202. Further, a program stored in the storage unit 204 may be loaded into the RAM 202.

  The display unit 205 and the input unit 206 may not be provided, and may be operated remotely via a network (not shown). Further, a plurality of CPUs 201 may be provided, and the ground surface information analysis system 100 may be configured by a plurality of computers.

  Each unit from the remote sensing image data acquisition unit 102 to the image update unit 114 described with reference to FIG. 1 may be realized by the CPU 201 executing a program stored in the RAM 202 or the storage unit 204 and corresponding to each unit. . Further, a plurality of CPUs 201 and the RAM 202 may be combined, and each unit may be realized by each combination. For this reason, the description that mainly includes the following units may be replaced with the description that mainly includes the CPU 201.

  The program stored in the storage unit 204 may be loaded from a storage medium reading device (not shown) or a network. Data in the database 115 may be stored in the RAM 202 or the storage unit 204. The I / F 203 may be a part of the remote sensing image data acquisition unit 102 and the three-dimensional information image data acquisition unit 103. The I / F 203 may be an I / F connected to a network or an I / F connected to a unique external bus or the like.

  The remote sensing observation device 211 is a remote sensing sensor such as an observation satellite, a manned aircraft, an unmanned aircraft, and the type is not limited as long as it performs remote sensing. The remote sensing image data 101a of the remote sensing observation apparatus 211 is, for example, RapidEye, Terra, Aqua (mounted with MODIS: MODerate resolution Imaging Spectroradiometer), LandSat satellite photograph, and aerial photograph data.

  The remote sensing image data 101a is time-series image data, and may be time-series image data acquired from one remote sensing observation apparatus 211 or may be acquired from a plurality of remote sensing observation apparatuses 211. The time-series image data may be used. The type of data of the remote sensing image data 101a is not limited, but data including information that can determine the color is preferable. When multiple types of data are included, the difference in image resolution may be unified to a lower resolution. Hereinafter, an RGB image acquired by an unmanned aerial vehicle (UAV) or a drone will be described as an example.

  The surface database 212 is a database including the three-dimensional information image data 101b, and the information in the database may be information acquired by a remote sensing sensor or may be information acquired from another device. . The 3D information image data 101b is, for example, a digital surface model (DSM: Digital Surface Model) or a digital elevation model (DEM: Digital Elevation Model). However, the type of data is not limited to these. In the following, the elevation information of the numerical surface model (DSM) will be described as an example.

  FIG. 3 is a flowchart showing an example of processing of the ground surface information analysis system 100. First, the ground surface information analysis system 100 uses the display unit 205 to prompt the user to specify the area and period to be analyzed, and stores the information specified by the input unit 206 in the RAM 202 or the storage unit 204.

  The remote sensing image data acquisition unit 102 acquires the remote sensing image data 101a (step 301), and the three-dimensional information image data acquisition unit 103 acquires the three-dimensional information image data 101b (step 302). Step 301 and step 302 may be executed in the reverse order, or may be executed simultaneously.

  The acquired remote sensing image data 101a and three-dimensional information image data 101b include image data of the same region in time series of the same period, and include image data of the region and period designated by the input unit 206. Since the image update unit 114 acquires new image data, the region and period specified by the input unit 206 need not be covered at a time.

  The database generation unit 104 generates the database 115 from the acquired remote sensing image data 101a and 3D information image data 101b (step 303). The image data in the database 115 is managed on the basis of the time when each image data was captured or measured in the time series and the planar position such as the GPS (Global Positioning System) position where each image data was captured or measured. Also good. Then, the database 115 is referred to and updated in the subsequent processing.

  In the case of shooting or measurement for an area of a certain size, the time when the image data was shot or measured cannot be covered by one shooting or measurement, and multiple shooting or measurement is performed. For example, from 10:10 to 10:30. Further, the time for photographing or measuring another target is, for example, 10:40 to 10:50. For this reason, the expression of time below includes not only 10 minutes but also time. Also, the time may be replaced with the expression time, with one point in time as a representative.

  The material classification unit 105 uses the remote sensing image data 101a stored in the database 115 to classify the material within the region and period specified by the input unit 206, and assigns a number to the classification result, and the database 115 (Step 304). Since the remote sensing image data 101a is an RGB image, the material classification may be RGB spectrum analysis, texture classification, or machine learning.

  For example, classification may be performed based on color information for each material from the RGB image, or other classification processing may be performed. The classification target is a deposit, but the ground may be a classification target, and the classification of the ground may be soil or may be a road other than a road on which a vehicle travels and a road other than a road. Further, the material classification unit 105 may also generate the material image data in which the material classification information is pasted on the three-dimensional image including the altitude using the three-dimensional information image data 101b and store the material image data in the database 115. . The classification information to be pasted may be information such as color and density that can be visually recognized by the user.

  In the classification of materials, the boundary lines of materials classified as different may be determined by binarization extraction, energy minimization extraction, edge detection, or the like of image processing, or may be machine-learned. Further, the extraction may be performed in stages from coarse extraction to fine extraction, binary extraction may be used for coarse extraction, and energy minimization extraction may be used for fine extraction. Since energy minimization extraction is computationally intensive, a graph cut technique may be used.

  The ground coefficient calculation unit 106 uses the 3D information image data 101b stored in the database 115 to calculate the ground coefficient within the region and period specified by the input unit 206, and generates ground coefficient image data. Store in the database 115 (step 305). The ground coefficient may be, for example, a coefficient of surface inclination or unevenness, or may be another coefficient. The ground coefficient will be described later with reference to FIG.

  The ground estimation unit 107 estimates the ground elevation within the area and period specified by the input unit 206 using the surface elevation included in the three-dimensional information image data 101b stored in the database 115, and the ground image data Is generated and stored in the database 115 (step 306). The altitude of the ground will be described later with reference to FIG.

  Here, Step 304 to Step 306 constitute a time-series multispectral three-dimensional image generation process 307, and the data stored in the database 115 by execution of Step 304 to Step 306 is time-series multispectral three-dimensional information image data. Configure. Therefore, it can be said that the time-series multispectral three-dimensional image generation process 307 is a process of the time-series multispectral three-dimensional image generation unit 108. The time-series multispectral three-dimensional information image data will be described later with reference to FIG.

  The deposition model generation unit 109 generates a deposition model for each material using the data of the material classification unit 105 in the time-series multispectral three-dimensional information image data stored in the database 115 and stores it in the database 115. (Step 308). The deposition model will be described later with reference to FIG. 7 by taking three examples of coal, wood powder, and wood as examples.

  The deposit volume calculation unit 110 uses the three-dimensional information image data 101b stored in the database 115, the ground image data generated by the ground estimation unit 107, and the deposition model generated by the deposition model generation unit 109 in time series. The volume for each is calculated, volume image data is generated, and stored in the database 115 (step 309). The calculation of the volume will be described later with reference to FIG.

  The change amount image generation unit 111 uses the volume image data and the ground coefficient image data stored in the database 115 to generate volume change image data for each material and time series ground coefficient change image data in time series. And stored in the database 115 (step 310). The volume variation image data and the ground coefficient variation image data will be described later with reference to FIG.

  The maintenance determination unit 112 uses the ground coefficient image data or the ground coefficient change amount image data stored in the database 115 to calculate an index indicating the road surface state or an index indicating the change in the road surface state, and sets a preset threshold value. In comparison, it is determined whether or not maintenance is necessary, and the determination result is provided to the control unit 113 (step 311). The process for determining whether or not maintenance is necessary will be described later with reference to FIG.

  The control unit 113 uses the time-series multispectral three-dimensional information image data, the volume change amount image data, the ground coefficient change amount image data, and the determination result of the maintenance determination unit 112 stored in the database 115 to use the apparatus related to the deposit. Is controlled (step 312). This control may be, for example, traveling control of the sediment transporting vehicle, road surface maintenance instruction control, or other control. The travel control of the vehicle may be a change of a travel path from a road with a poor road surface condition to a road with a good road surface condition. Each image data may be displayed on the display unit 205.

  When the control unit 113 receives the determination result from the maintenance determination unit 112, the image update unit indicates that the processing of the image data acquired by the remote sensing image data acquisition unit 102 and the three-dimensional information image data acquisition unit 103 has ended. 114 may be notified. For example, since the vehicle travel control can be executed independently of the processing from step 301 to step 311, there is no effect even if the process proceeds to step 313 and subsequent steps before or during the start of the vehicle travel control.

  When the control unit 113 notifies the end of the processing of the image data, the image updating unit 114 determines whether or not to end the processing (step 313). An image update to be acquired and processed is executed (step 314). Whether to end the process may be determined by determining whether the process for the specified region and period has ended, or may be determined not to always end in a system that continues processing, or by the input unit 206. Some input information by may be determined.

  When executing step 314, the image update unit 114 instructs the remote sensing image data acquisition unit 102 and the three-dimensional information image data acquisition unit 103 to acquire new image data, and returns to step 301.

  FIG. 4 is a diagram illustrating an example of ground coefficient calculation. Here, the ground coefficient is a gradient, unevenness, inclination, inclination angle, or the like. The calculation of the ground coefficient may be, for example, calculation of TRI (Terrain Ruggedness Index). In order to calculate the TRI, the ground coefficient calculation unit 106 acquires the altitude information of each point included in the three-dimensional information image data 101b stored in the database 115, and obtains nine adjacent points, that is, the grid cells 401. The center point elevation Z (0, 0) is subtracted from the elevation points Z (i, j) at 8 points other than the center point and squared, and the square result is calculated by adding the 8 squared results.

  Here, the calculation of TRI may be averaged by dividing by the value 8 instead of the calculation of the square root. The ground coefficient calculation unit 106 calculates the TRI of each point in this way, and compares with a preset threshold value, a region composed of points greater than or equal to the threshold value, and a region composed of points less than the threshold value. It may be divided into For example, a hatched area such as the area 402 is an area composed of points greater than or equal to the threshold value, that is, an area having a large slope, and an unhatched area such as the area 403 is composed of points less than the threshold value. The ground coefficient may be represented by image data such that the area is an area close to a plane. The ground coefficient calculation unit 106 may store such image data in the database 115.

  FIG. 5 is a diagram illustrating an example of ground estimation. Since the altitude information included in the three-dimensional information image data 101b, which is a numerical surface model (DSM), includes the altitude of the deposit, the altitude of the ground is not necessarily reflected. For this reason, the ground estimation unit 107 estimates the altitude of the ground using the time-series three-dimensional information image data 101b. The image 501 is an image obtained by viewing the elevation of the first time from a substantially vertical direction at the first point. A solid line area such as the area 502 has a higher elevation than the surrounding area, and a broken line area such as the area 503 has substantially the same elevation as the surrounding area.

  The image 504 is an image obtained by viewing the altitude of the second time at the first point from the same viewpoint as the image 501. A broken-line area like the area 505 has almost the same elevation as the surrounding area, and a solid-line area like the area 506 has a higher elevation than the surrounding area. For such altitude image data, the altitude of the area 502 and the altitude of the area 505 are compared, the area 508 having the same altitude as the area 505 is selected, and the altitude of the area 503 and the altitude of the area 506 are compared. By selecting an area 509 having the same altitude as that of the area 503, the altitude of the ground is estimated and data of the ground image 507 is obtained. In FIG. 5, the images 501 and 504 and the ground image 507 are flat for ease of explanation, but may be three-dimensional image data including altitude.

  FIG. 6 is a diagram illustrating an example of time-series multispectral three-dimensional information image data. The time 604 is a plurality of times forming a time series, and may be a time when the remote sensing image data 101a and the three-dimensional information image data 101b are captured or measured, or may be a preset time. The material 601 is a material classified by the material classification unit 105. For example, the number in parentheses of the material 601 is a classification result number, “Material (1)” is coal, and “Material (2) “Is a wood powder and“ material (3) ”is wood.

  The “three-dimensional image (3, 1)” of the three-dimensional image 605 in the “material (3)” of the material 601 is an image in which the color of the wood accumulation region is different from other regions in the three-dimensional image of the altitude. It may be image data for display. The altitude 602 may be image data representing a 3D image of the altitude of the 3D information image data 101b.

  The ground condition coefficient 603 is data of a ground coefficient image generated by the ground coefficient calculation unit 106, and may be data representing an image such as the area 402 and the area 403 shown in FIG. Further, the ground condition coefficient 603 may include image data of a plurality of coefficients, and the plurality of coefficients may be an unevenness coefficient, a slope coefficient, a soil coefficient, and the like.

  FIG. 7 is a diagram illustrating an example of a deposition model. The deposition model is a model for specifying the area of the deposit covering the ground, and is generated every time according to the material. That is, it is generated for each three-dimensional image 605 for each time 604 of each material 601 shown in FIG. In the three-dimensional image 605 of coal that is “material (1)”, as shown in FIG. 7, regions 701 classified as coal have a regular shape. For this reason, the area | region 702 which extracted the outline of the lump of the several area | region 701 becomes a coal deposition model.

  In addition, the three-dimensional image 605 of the wood powder that is “material (2)” is a collection of pieces of various shapes in which regions 703 classified as wood powder are dispersedly arranged. For this reason, one large sediment in which the nearby regions 703 are fused becomes a wood powder deposition model. In the three-dimensional image 605 of wood that is “material (3)”, a region 705 classified as wood is a large rectangle. For this reason, the area | region 706 which match | combined the large rectangular area | region 705 becomes a pile model of wood.

  Further, a deposition model corresponding to each material may be constructed for other materials. The dotted line shown in FIG. 7 indicates a region that coincides with the region of the deposition model, and is a line for facilitating understanding of the region of the deposition model. The deposition model generated for each material and time may be stored in the database 115.

  FIG. 8 is a flowchart showing an example of deposit volume calculation. The deposit volume calculation unit 110 acquires the material generated by the deposition model generation unit 109 and the deposition model for each time (step 801). As shown in FIG. 7, since the deposition model includes the area of a plane like areas 702, 704, and 706, the area of each area is calculated (step 802). Here, for example, the area of the region 702 may be a set of areas calculated by dividing the region 702 into a plurality of regions and calculating the area of each divided region. Since the area 702 is an image, it may be divided into units of one pixel.

  Next, the deposit volume calculation unit 110 acquires the elevation of the deposition model (step 803). Here, if the area is divided in step 802, the acquired altitude is the altitude of the divided area. If the divided area is wide, the elevation representing the area may be used. If the divided area is 1 pixel, it is the altitude of that pixel. The altitude data itself may be acquired from the three-dimensional information image data 101b.

  The sediment volume calculation unit 110 acquires the altitude of the lower ground on which each deposition model is arranged, that is, the ground covered with each deposition model (step 804). If the area is divided in step 802, the acquired ground elevation is the ground elevation of each divided area. The ground elevation itself is the ground elevation estimated by the ground estimation unit 107.

  The sediment volume calculation unit 110 subtracts the ground elevation obtained in step 804 from the elevation of the deposition model obtained in step 803, and multiplies the subtraction result by the area calculated in step 802 to obtain the deposition. Calculate the volume of the object. If the area is divided in step 802, the volume is calculated for each divided area, and the calculated volumes are totaled. In this way, the volume of each of the plurality of deposition models is calculated.

  FIG. 9 is a diagram illustrating an example of the change amount image. Each of the material 901, the ground condition coefficient 903, and the time 904 shown in FIG. 9 corresponds to the material 601, the ground condition coefficient 603, and the time 604 shown in FIG. The three-dimensional volume image 905 is a three-dimensional three-dimensional image of the volume calculated by the deposit volume calculation unit 110. The three-dimensional volume image 905 is calculated for each of the material 901 and each of the times 904 and converted into a three-dimensional image. It is a thing.

  The material (1) volume change amount image 906 is a change of the three-dimensional volume image 905 in which the material 901 represents the volume of “material (1)” (coal), and the time 904 is changed from “time (1)” to “time”. (N) ”is an image showing a change until“ n ”. For example, the image may be an image in which the change can be visually recognized by changing the color and density of each image from “3D volume image (1, 1)” to “3D volume image (1, n)”. .

  The three-dimensional situation image 907 is obtained by converting the ground coefficient image data generated by the ground coefficient calculation unit 106 into a three-dimensional three-dimensional image at each time 904. The ground situation change amount image 908 is an image representing a change in the three-dimensional situation image 907 from time “904” to “time (n)”. In order to represent this change, the color and density of the image at each of the times 904 may be changed and combined.

  FIG. 10 is a flowchart illustrating an example of maintenance determination. The maintenance determination unit 112 acquires the ground state change amount image 908 generated by the change amount image generation unit 111 (step 1001). Here, for example, the ground state change amount image 908 may be acquired for each of the unevenness coefficient, the inclination coefficient, and the soil coefficient.

  When a plurality of ground condition change amount images 908 are acquired, the maintenance determination unit 112 merges the plurality of ground condition change amount images 908 into one image (step 1002). For example, the coefficient of inclination of the coarse image resolution and the coefficient of unevenness of the fine image resolution may be merged to generate image data that becomes an image that can also express unevenness in the inclination. Further, the generated image data represents, for example, a change amount of the inclination angle within a predetermined time on the basis of a time series. If the change amount of the inclination angle is large even within the same predetermined time, It can also be interpreted as information that maintenance should be performed immediately.

  Therefore, the maintenance determination unit 112 quantifies the amount of change included in the image data that is a fused image (step 1003). As described above, the numerical change amount may include the change amount of the inclination angle. On the other hand, the preset threshold value is compared with the numerical change amount, and if it is determined that the numerical change amount is greater than or equal to the threshold value, the process proceeds to step 1005; move on.

  The maintenance determination unit 112 sets information that maintenance is necessary in step 1005, and sets information that maintenance is unnecessary in step 1006. Then, this information is sent to the control unit 113.

  Note that the determination is not limited to one coefficient. The variation obtained by quantifying the unevenness coefficient is multiplied by the first constant, the variation obtained by quantifying the soil coefficient is multiplied by the second constant, and an added value of these two multiplied values; The determination may be made by comparing with a preset threshold value. Further, instead of the change amount, the time 904 may be determined based on “time (n)”, that is, a ground coefficient included in the three-dimensional situation image 907 at the last time stored in the database 115.

  In addition, the change amount image generation unit 111 specifies the change amount of the volume calculated by the deposit volume calculation unit 110 for each material or deposition model, and compares the specified change amount with a preset threshold value. It may be determined whether maintenance is necessary for the deposit being full. For this reason, the data of the three-dimensional image representing the change amount may be converted into a numerical value representing the change amount.

  As described above, the amount of change for each substance can be calculated. In particular, it is possible to calculate a deposit or the ground as a substance, and it is also possible to calculate a change amount of a volume or a change amount of a ground coefficient for each deposit. This calculation uses two-dimensional RGB image data and numerical surface model (DSM) data taken by an unmanned aerial vehicle or the like, and therefore does not require much cost. Moreover, since the change amount of the ground coefficient can be calculated together with the change amount of the volume for each deposit from these same data, the mining company or the like is useful for both management and transportation of the deposit.

  The volume calculation of the deposit can be calculated from time-series image data without requiring a special investigation for calculating the volume. At this time, the shape of the ground under the deposit can be included in the calculation, and a volume suitable for each deposit can be calculated by a deposition model corresponding to the material of the deposit. And since it is imaging | photography from the sky, it is possible to target a large area and the dispersion | variation in accuracy does not generate | occur | produce by an operator.

  Since the amount of change is imaged as a result of calculation or estimation, the state of change is easily understood by the user by displaying the image. It is also possible to determine from a plurality of viewpoints by merging a plurality of ground coefficients and determining maintenance. Then, the original mining operation can be continued by controlling when maintenance is necessary.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. For example, a part of the configuration may be added, deleted, or replaced with another configuration. Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit.

  Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files that realize each function is stored in a recording device such as a memory, an HDD, or an SSD, or a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc). May be.

  Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

100 ground surface information analysis system 102 remote sensing image data acquisition unit 103 3D information image data acquisition unit 104 database generation unit 105 material classification unit 106 ground coefficient calculation unit 107 ground estimation unit 108 time-series multispectral 3D image generation unit 109 deposition model Generation unit 110 Deposit volume calculation unit 111 Change amount image generation unit 112 Maintenance determination unit 113 Control unit 114 Image update unit

Claims (15)

In the ground information analysis system that analyzes the ground surface from images,
Obtain two-dimensional image data containing information about the surface area of the material at multiple times,
Acquire 3D image data including shape information of the ground surface at multiple times,
Using the acquired two-dimensional image data, specify the area of each of the multiple types of substances,
The region and shape corresponding to each of the times included in the plurality of times, each of the identified substance region, and the shape identified by the acquired three-dimensional image data, are divided into a plurality of times and a plurality of types. Generate time-series multispectral 3D image data as a combination of materials,
A surface information analysis system characterized in that the amount of change of each substance is calculated based on data at a plurality of times included in the generated time-series multispectral three-dimensional image data. In the ground information analysis system according to claim 1,
A surface information analysis system characterized in that remote sensing image data obtained by photographing an unmanned aerial image of the surface is acquired as data of a two-dimensional image. In the ground information analysis system according to claim 1,
Obtain two-dimensional image data including color information as information about the substance area,
A surface information analysis system characterized by specifying regions of a plurality of types of substances based on color information included in acquired two-dimensional image data. In the ground information analysis system according to claim 1,
Surface material is sediment on the ground,
Obtaining two-dimensional image data containing information about the area of the deposit at multiple times;
A surface information analysis system characterized in that the region and type of each of a plurality of types of deposits are specified based on color information contained in acquired two-dimensional image data. In the ground information analysis system according to claim 1,
The surface shape information includes ground elevation information,
Obtain 3D image data including ground elevation information at multiple times,
A ground surface information analysis system, characterized in that a ground coefficient representing an uneven state of the ground is calculated based on altitude information included in acquired three-dimensional image data. In the ground information analysis system according to claim 1,
The surface shape information includes elevation information of sediment on the ground,
Acquire 3D image data including elevation information of sediments at multiple times,
A ground surface information analysis system characterized in that an altitude of a ground is calculated based on a plurality of altitude information at a plurality of times included in acquired three-dimensional image data. In the ground information analysis system according to claim 4,
A surface information analysis system characterized by generating a deposition model with a region adjacent to each of the identified deposits as one region according to the type of each identified deposit. In the surface information analysis system according to claim 7,
The surface shape information includes elevation information of sediment on the ground,
Acquire 3D image data including elevation information of sediments at multiple times,
Based on a plurality of altitude information at a plurality of times included in the acquired three-dimensional image data, the altitude of the ground is calculated,
A surface information analysis system characterized in that the volume of a deposit is calculated based on the area of the generated deposition model, the acquired elevation information of the deposit, and the calculated elevation of the ground. In the surface information analysis system according to claim 8,
Using the data of the generated time-series multispectral 3D image, calculate the volume of the deposit for a combination of a plurality of times and a plurality of types of deposits,
A surface information analysis system characterized in that, for each deposit, data of a volume change amount image is generated based on a change amount corresponding to a plurality of times of the calculated volume. In the surface information analysis system according to claim 9,
A surface information analysis system characterized in that, for each deposit, the generated volume change amount image data is displayed as an image. In the surface information analysis system according to claim 5,
Using the data of the generated time-series multispectral 3D image, calculate the ground coefficient at multiple times,
A ground surface information analysis system that generates data of a ground state change amount image based on a change amount of a calculated ground coefficient according to a plurality of times. In the surface information analysis system according to claim 11,
Using the data of the generated ground condition change amount image, the change amount of the ground coefficient is digitized, the digitized change amount is compared with a preset threshold value, and it is determined that the change amount is equal to or greater than the threshold value. A ground surface information analysis system, characterized in that maintenance is set and maintenance is not required when it is determined that the amount of change is less than a threshold. In the surface information analysis system according to claim 12,
A ground surface information analysis system that controls to maintain the ground surface when maintenance is required. In the ground information analysis method that analyzes the ground surface from images,
Obtain two-dimensional image data containing information about the surface area of the material at multiple times,
Acquire 3D image data including shape information of the ground surface at multiple times,
Using the acquired two-dimensional image data, specify the area of each of the multiple types of substances,
The region and shape corresponding to each of the times included in the plurality of times, each of the identified substance region, and the shape identified by the acquired three-dimensional image data, are divided into a plurality of times and a plurality of types. Generate time-series multispectral 3D image data as a combination of materials,
A surface information analysis method, comprising: calculating a change amount of each substance based on a plurality of time data included in the generated time-series multispectral three-dimensional image data. In the surface information analysis method according to claim 14,
Obtain two-dimensional image data including color information as information about the substance area,
A method for analyzing ground surface information, characterized in that a region of each of a plurality of types of substances is specified based on color information contained in acquired two-dimensional image data.

Images