WO2024260008A1 - Road damage detection method and apparatus, method and apparatus for determining layout position of road sensor, and computer device and storage medium - Google Patents

Abstract

A road damage detection method and apparatus, and a computer device and a storage medium. The detection method comprises: acquiring sensing data information and image information, which are sent by collection boxes, of a road, and a plurality of different sample damage ranges, and on the basis of the sensing data information of the road, determining the current damage range of the road and a damage sensing map of the road (S101); identifying a target damage area of the road on the basis of the image information of the road and the current damage range of the road, and identifying a damage type of the road on the basis of the damage sensing map of the road (S102); acquiring damage sensing data of the target damage area, and on the basis of the damage type of the road and a sample damage range of the damage type at each degree, calculating a damage level corresponding to the damage sensing data of the target damage area (S103); and using the target damage area, the damage level and a damage type as target damage information of the road (S104).

Description

Road damage detection method and device, road sensor layout location determination method and device, computer equipment and storage medium

Related Applications

This application claims priority to Chinese patent applications filed on June 21, 2023, with application number 202310740091.1, entitled “Road Damage Detection Method, Device, Computer Equipment and Storage Medium”, filed on June 21, 2023, with application number 202310745064.3, entitled “Method, Device and Computer Equipment for Determining the Location of Road Sensors”, and filed on June 21, 2023, with application number 202310745521.9, entitled “Road Damage Detection Method, Device, Computer Equipment and Storage Medium”, the entire text of which is hereby incorporated by reference.

Technical Field

The present application relates to the field of artificial intelligence technology, and in particular to a method and device for detecting road damage, a method and device for determining the location of road sensors, computer equipment, storage media, and computer program products.

Background Art

During the service life of road infrastructure, due to the reciprocating effects of vehicle and environmental loads, cracking, hollowing and other defects may occur, which reduces the service capacity of the road. Monitoring the service status of the road so that the maintenance party can repair the road in the early stage of the disease is an important way to extend the service life of the road and reduce the cost of the road throughout its life cycle. Therefore, how to detect road damage is the current research focus.

The traditional road damage detection method is to manually detect the current state of the road and judge the damage of the road based on the state and the experience of the staff. This solution requires a lot of manpower costs, and due to the different experience of the staff, manual judgment will lead to large deviations in the judgment results of the road damage, resulting in low accuracy in road damage detection.

In addition, road health monitoring is an important part of road maintenance. In order to be able to timely infer possible defects inside the road based on the stress conditions of the road, road health monitoring is generally carried out by burying sensors inside the road.

Generally speaking, the more sensors there are inside the road, the higher the accuracy of road health monitoring. However, deploying too many sensors will also increase the cost of building a monitoring system. Therefore, it is necessary to provide a sensor deployment solution that can achieve higher detection accuracy with fewer sensors and reduce the cost of the monitoring system.

In addition, my country currently ranks first in the world in terms of highway mileage, and a large number of roads need to be maintained and managed. Due to environmental erosion, aging and other factors, highway road damage may occur. Therefore, road damage detection technology has emerged.

Traditional road damage detection technology usually installs a camera on the side of the road to collect road image data. Based on the large amount of collected road image data and convolutional neural networks, image recognition processing is performed on the collected road surface image data to identify road damage in the image data to complete road damage detection.

However, in traditional road detection technology, road surface detection is achieved through road surface image data, which can often only identify road surface diseases and damage. For cement concrete pavement, the bottom of the road slab is hollowed out, which is the most fundamental disease. When the road surface is damaged, it is often caused by more serious damage inside the road surface. Therefore, traditional road detection technology cannot identify internal road diseases and damage, and the accuracy of road damage detection is poor.

Summary of the invention

Based on this, it is necessary to provide a road damage detection method and device, a road sensor deployment location determination method and device, computer equipment, computer-readable storage medium and computer program product to address the above technical issues.

The present application provides a road damage detection method. The method comprises:

Acquire sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road; the sample damage range includes damage ranges of different damage types;

Based on the image information of the road and the current damage range of the road, a target damage area of the road is identified through a damage location identification strategy, and based on the damage sensor map of the road, a damage identification network is used to identify the damage type of the road;

Acquire damage sensor data of the target damage area, and calculate the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type;

The target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as target damage information of the road.

Optionally, the sensor data information of the road includes sensor information of multiple sensors of the road and location information of each of the sensors; and determining the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road includes:

Based on the sensing information of each sensor and the position information of each sensor, three-dimensional sensing map data of the road is established, and damage map data that meets the road damage condition is screened in the three-dimensional sensing map data of the road, and the range of all damage map data included in the three-dimensional sensing map data is used as the current damage range of the road; the damage map data includes the position information of the corresponding pixel points of the damage map data and the sensing information of the corresponding pixel points of the damage map data;

Based on the position information of each image data within the current damage range and the sensor information of each image data, a damage sensor map corresponding to each image data within the current damage range is established.

Optionally, the identifying a target damaged area of the road by a damage location identification strategy based on the image information of the road and the current damage range of the road includes:

Establishing three-dimensional image data of the image information of the road, and identifying the damaged location area of the image information of the road through a damaged image recognition network;

Establishing a correspondence between the three-dimensional sensing image data and the three-dimensional image data, and identifying a sub-damage range corresponding to a damage position area within the current damage range based on the correspondence;

Clustering the damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculating the average distance between each damage map data in each damage map data group and the sub-damage range;

Filter each target damage map data in the damage map data group corresponding to the average distance below the preset distance threshold, and use the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as the sub-damage area, and use all the sub-damage areas as the target damage areas of the road.

Optionally, the identifying the damage type of the road through a damage identification network based on the damage sensor map of the road includes:

Extracting damage feature data of each sub-damage area of the damage sensing map, and inputting each damage feature data into a damage identification network to obtain a sub-damage type corresponding to each damage feature data;

The sub-damage type corresponding to each damage feature data is used as the damage type of the road.

Optionally, the calculating the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type includes:

For each sub-damage region, based on the sample damage ranges of the degrees corresponding to the sub-damage types of the sub-damage region, the sample damage range to which the damage sensing data of the sub-damage region belongs is identified, and the degree of the sub-damage type corresponding to the sub-damage region is obtained;

Based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, the damage level corresponding to the sub-damage area is determined through a damage level classification strategy.

Optionally, after taking the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road, the method further includes:

For each sub-damage region, determining a damage repair strategy for the sub-damage region based on the sub-damage type of the sub-damage region and the degree of the sub-damage type of the sub-damage region;

According to the damage level of each sub-damage area from high to low, the maintenance order of each sub-damage area is arranged to obtain the repair sequence of each sub-damage area; and the damage repair strategy of each sub-damage area is filled into the repair sequence to obtain the repair task information of the target damage area, and the early warning information including the repair task information and the damage information of the target damage area is sent to the display module.

Optionally, the method further includes: in response to the user's sensor acquisition system update operation, obtaining the acquisition system update information of each sensor Information, and send the acquisition system update data information to the acquisition box; the acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information.

Optionally, the method further includes: in response to a user's sensor acquisition task upload operation, generating an acquisition instruction for each sensor, and sending the acquisition instruction to the acquisition box; the acquisition instruction includes the acquisition task of each sensor, and the acquisition instruction is used to instruct each sensor to execute the acquisition task in the acquisition instruction.

The present application also provides a road damage detection device, which includes: an acquisition module, an identification module, a re-acquisition module and a determination module.

an acquisition module, used to acquire sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road; the sample damage range includes damage ranges of different damage types;

an identification module, configured to identify a target damaged area of the road through a damage location identification strategy based on the image information of the road and the current damage range of the road, and to identify a damage type of the road through a damage identification network based on a damage sensor map of the road;

a re-acquisition module, configured to acquire the damage sensor data of the target damage area, and calculate the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type;

The determination module is used to use the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road.

Optionally, the sensor data information of the road includes sensor information of multiple sensors on the road and location information of each of the sensors.

The acquisition module is specifically used to: establish three-dimensional sensor map data of the road based on the sensor information of each sensor and the position information of each sensor, and screen damage map data that meets the road damage condition in the three-dimensional sensor map data of the road, and use the range of all damage map data in the three-dimensional sensor map data as the current damage range of the road; the damage map data includes the position information of the pixel points corresponding to the damage map data and the sensor information of the pixel points corresponding to the damage map data; based on the position information of each map data within the current damage range and the sensor information of each map data, establish a damage sensor map corresponding to each map data in the current damage range.

Optionally, the recognition module is specifically used to: establish three-dimensional map data of the image information of the road, and identify the damage location area of the image information of the road through a damage image recognition network; establish a correspondence between the three-dimensional sensor image data and the three-dimensional map data, and identify the sub-damage range corresponding to the damage location area within the current damage range based on the correspondence; cluster each damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculate the average distance of each damage map data in each damage map data group from the sub-damage range; screen each target damage map data in the damage map data group corresponding to the average distance below a preset distance threshold, and use the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as a sub-damage area, and use all sub-damage areas as target damage areas of the road.

Optionally, the identification module is specifically used to: extract damage feature data of each sub-damage area of the damage sensor map, and input each damage feature data into the damage identification network respectively to obtain the sub-damage type corresponding to each damage feature data; and use the sub-damage type corresponding to each damage feature data as the damage type of the road.

Optionally, the re-acquisition module is specifically used to: for each sub-damage area, based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage area, identify the sample damage range to which the damage sensing data of the sub-damage area belongs, and obtain the degree of the sub-damage type corresponding to the sub-damage area; based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, determine the damage level corresponding to the sub-damage area through a damage level classification strategy.

Optionally, the device further includes: a strategy determination module and a task determination module.

The strategy determination module is used to determine, for each sub-damage area, a damage repair strategy for the sub-damage area based on the sub-damage type of the sub-damage area and the degree of the sub-damage type of the sub-damage area.

The task determination module is used to determine the repair order of each sub-damage area according to the damage level of each sub-damage area from high to low. Arrange them to obtain a repair sequence for each of the sub-damage areas; and fill the damage repair strategy of each sub-damage area into the repair sequence to obtain the repair task information of the target damage area, and send the warning information including the repair task information and the damage information of the target damage area to the display module.

Optionally, the device also includes an updating module.

The update module is used to respond to the user's sensor acquisition system update operation, obtain the acquisition system update information of each sensor, and send the acquisition system update data information to the acquisition box; the acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information.

Optionally, the device also includes an instruction sending module.

The instruction sending module is used to generate an acquisition instruction for each sensor in response to the user's sensor acquisition task upload operation, and send the acquisition instruction to the acquisition box; the acquisition instruction includes the acquisition task of each sensor, and the acquisition instruction is used to instruct each sensor to execute the acquisition task in the acquisition instruction.

The present application also provides a road damage detection system, which includes a cloud platform and a collection box.

The collection box is communicatively connected to the cloud platform.

The collection box is used to collect sensor data information of the road and image information of the road.

The cloud platform is used to obtain multiple different sample damage ranges, and based on the sensor data information of the road, determine the current damage range of the road and the damage sensor map of the road; the sample damage range includes damage ranges of different damage types; based on the image information of the road and the current damage range of the road, identify the target damage area of the road through a damage location recognition strategy, and based on the damage sensor map of the road, identify the damage type of the road through a damage identification network; obtain the damage sensor data of the target damage area, and based on the damage type of the road and the sample damage ranges of each degree of the damage type, calculate the damage level corresponding to the damage sensor data of the target damage area; use the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road.

The above-mentioned road damage detection method and device obtain sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and multiple different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road; the sample damage range includes damage ranges of different damage types; based on the image information of the road and the current damage range of the road, the target damage area of the road is identified through a damage position identification strategy, and based on the damage sensor map of the road, the damage type of the road is identified through a damage identification network; the damage sensor data of the target damage area is obtained, and based on the damage type of the road and the sample damage ranges of each degree of the damage type, the damage level corresponding to the damage sensor data of the target damage area is calculated; the target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as the target damage information of the road. The target damage area of the road is determined through the sensor data information of the road and the image information of the road, and the damage type and damage level of the target damage area are identified through the damage identification network based on multiple sample damage ranges of different degrees to obtain the target damage information of the road. Therefore, the target damage area can be determined without manual detection, which improves the accuracy of judging the location of road damage. Then, the damage type of the target damage area is identified, which reduces the data processing amount for judging road damage information and improves the detection efficiency of road damage information. Finally, the damage level of the target damage area is identified through multiple sample damage ranges of different degrees, which improves the accuracy of road damage detection.

The present application also provides a method for determining the location of a road sensor. The method comprises:

Determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road;

Determine a reference layout point from the layout points corresponding to each of the road conditions, construct a reference layout point group based on the reference layout point, and use the layout points that do not belong to the reference layout point group among the layout points corresponding to each of the road conditions as candidate layout points;

For each candidate layout point, determine the target road condition corresponding to the candidate layout point, and determine the first condition data corresponding to each reference layout point under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition, and determine the correlation coefficient between the candidate layout point and the reference layout point group according to each of the first condition data and the second condition data, wherein the condition data is used to characterize the stress condition of the road at the layout point;

Selecting a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points;

The target layout point is added to the reference layout point group to obtain a final target layout point group.

In one embodiment, the step of determining the layout points corresponding to each road condition from a plurality of preset layout points of the target road includes:

Constructing road models corresponding to target roads under various road conditions respectively;

For each of the road conditions, the vehicle driving data corresponding to the target road under the road condition is obtained, and based on the vehicle driving data and the road model corresponding to the target road under the road condition, the third condition data corresponding to multiple preset layout points of the target road under the road condition are respectively determined, and based on each of the third condition data, the layout point corresponding to the road condition is determined from each of the preset layout points.

In one embodiment, determining the layout point corresponding to the road working condition from the preset layout points according to the third working condition data includes:

Sorting the preset layout points from large to small according to the third working condition data to obtain a preset layout point queue;

Traverse the preset layout point queue, and when the third operating condition data corresponding to each preset layout point arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue, and use each preset layout point arranged before the current traversal position as the layout point corresponding to the road operating condition.

In one embodiment, determining the reference layout point from the layout points corresponding to each of the road conditions includes:

According to the layout points corresponding to the road conditions, respectively determine the number of layout points corresponding to the road conditions;

The layout points corresponding to the largest number of layout points among the numbers of layout points are taken as the reference layout point group.

In one embodiment, determining the correlation coefficient between the candidate layout points and the reference layout point group according to each of the first operating condition data and the second operating condition data includes:

For any of the reference layout points, determining a correlation coefficient between the candidate layout point and the reference layout point according to the first operating condition data and the second operating condition data corresponding to the reference layout point;

The correlation coefficient between the candidate layout point and the reference layout point group is determined according to the correlation coefficient corresponding to each of the reference layout points.

In one embodiment, selecting a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points includes:

For any of the candidate layout points, when the correlation coefficient corresponding to the candidate layout point is less than a correlation coefficient threshold, the candidate layout point is used as a target layout point.

In one embodiment, the method further comprises:

According to the preset layout point arrangement strategy, a plurality of preset layout points are determined on the target road, and the plurality of preset layout points are evenly distributed on the target road.

The present application also provides a device for determining the location of a road sensor. The device comprises:

A first determination module is used to determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road;

A second determination module is used to determine a reference layout point from the layout points corresponding to each of the road conditions, construct a reference layout point group according to the reference layout point, and use the layout points that do not belong to the reference layout point group among the layout points corresponding to each of the road conditions as candidate layout points;

A third determination module is used to determine, for any of the candidate layout points, the target road condition corresponding to the candidate layout point, and determine the first condition data corresponding to each of the reference layout points under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition, and determine the correlation coefficient between the candidate layout point and the reference layout point group according to each of the first condition data and the second condition data, wherein the condition data is used to characterize the stress condition of the road at the layout point;

A selection module, configured to select a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points;

An adding module is used to add the target layout point to the reference layout point group to obtain a final target layout point group.

In one embodiment, the first determining module is further used to:

Constructing road models corresponding to target roads under various road conditions respectively;

For any of the road conditions, the vehicle driving data corresponding to the target road under the road condition is obtained, and based on the vehicle driving data and the road model corresponding to the target road under the road condition, the third condition data corresponding to multiple preset layout points of the target road under the road condition are respectively determined, and based on each of the third condition data, the layout point corresponding to the road condition is determined from each of the preset layout points.

In one embodiment, the first determining module is further used to:

Sorting the preset layout points from large to small according to the third working condition data to obtain a preset layout point queue;

Traverse the preset layout point queue, and when the third operating condition data corresponding to each preset layout point arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue, and use each preset layout point arranged before the current traversal position as the layout point corresponding to the road operating condition.

In one embodiment, the second determining module is further used to:

According to the layout points corresponding to the road conditions, respectively determine the number of layout points corresponding to the road conditions;

The layout points corresponding to the largest number of layout points among the numbers of layout points are taken as the reference layout point group.

In one embodiment, the third determination module is further used to:

For any of the reference layout points, determining a correlation coefficient between the candidate layout point and the reference layout point according to the first operating condition data and the second operating condition data corresponding to the reference layout point;

The correlation coefficient between the candidate layout point and the reference layout point group is determined according to the correlation coefficient corresponding to each of the reference layout points.

In one embodiment, the selection module is further used to:

For any of the candidate layout points, when the correlation coefficient corresponding to the candidate layout point is less than a correlation coefficient threshold, the candidate layout point is used as a target layout point.

In one embodiment, the device further comprises:

The fourth determination module is used to determine a plurality of preset layout points on the target road according to a preset layout point arrangement strategy, wherein the plurality of preset layout points are evenly distributed on the target road.

The present application also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the above methods when executing the computer program.

The present application also provides a non-volatile computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, any of the above methods is implemented.

The present application also provides a computer program product, which includes executable instructions, and when the executable instructions are executed by a processor, any of the above methods is implemented.

The above method and device for determining the location of road sensor deployment, and computer equipment, select a reference deployment point group, and then calculate the correlation coefficients between the remaining candidate deployment points and the reference deployment point group respectively. When the correlation coefficients meet the requirements, it is determined that the road conditions at the candidate deployment point cannot be accurately monitored based on each reference deployment point in the reference deployment point group, and the candidate deployment point is added to the reference deployment point group, thereby obtaining the final target deployment point group. Therefore, it is possible to detect a variety of typical road diseases with fewer sensors, reducing the detection cost.

The present application also provides a road damage detection method. The method comprises:

Acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be detected;

Extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features;

Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector;

The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

In one embodiment, the implantable sensor is a vibration acceleration sensor pre-installed inside the road to be detected, and the acquisition of the acceleration data set of the road to be detected within a preset detection period includes:

Based on the preset sampling frequency and signal length, the vibration acceleration signal collected by each vibration acceleration sensor when the vehicle passes through the detection area of the road to be detected within the preset detection period is collected; the vibration acceleration signal is generated by the road panel of the vehicle passing through the road to be detected. Response generation;

Performing data preprocessing on each of the vibration acceleration signals, and constructing an acceleration vector based on the vibration acceleration signals collected at the same time;

Based on each of the acceleration vectors, an acceleration data set is obtained.

In one embodiment, the step of obtaining an additional attribute feature data set of a road to be detected within a preset detection period includes:

Acquire attribute characteristic data of the road to be detected, attribute characteristic data of each vehicle passing through the road to be detected within a preset detection period, and internal monitoring environment data of the implanted sensor within the preset detection period;

The attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor are cleaned and normalized to obtain an additional attribute feature data set.

In one of the embodiments, the hidden layer of the recursive neural network includes multiple hidden layer units, and the feature extraction of the acceleration data set according to the recursive neural network to obtain the acceleration feature includes: inputting the acceleration data set into a pre-trained recursive neural network, and extracting the feature of the acceleration vector in the acceleration data set through the multiple hidden layer units included in the hidden layer of the recursive neural network to obtain the acceleration feature.

In one embodiment, after inputting the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected, the method further includes:

Based on the road damage result, determining a target road management strategy in a corresponding relationship between the road damage result and the road management strategy;

Based on the target road management strategy, an instruction is given to perform maintenance management on the road to be inspected.

In one embodiment, the method further comprises:

Acquire training data samples; the training data samples include a training acceleration data set, an additional attribute feature data set, and a road damage category label;

Inputting the training acceleration data set into a recursive neural network, performing feature extraction on the training acceleration data set to obtain acceleration features;

Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector;

Inputting the fused feature vector and the road damage category label into a classification prediction network, performing data processing on the fused feature vector through the classification prediction network to obtain a classification prediction result;

According to the classification prediction result, the road damage category label and the preset loss function, the loss result of the road damage detection model is determined until the loss result meets the preset model loss condition, and it is determined that the road damage detection model training is completed.

The present application also provides a road damage detection device, which includes: an acquisition module, a feature extraction module, a splicing module and a detection and discrimination module.

An acquisition module is used to acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by collecting a vibration acceleration signal generated by a vehicle load through an implanted sensor arranged inside the road to be detected;

A feature extraction module, used for extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features;

A splicing module, used for performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector;

The detection and discrimination module is used to input the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected.

The present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

Acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be detected;

Extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features;

Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector;

The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

The present application also provides a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, When the computer program is executed by a processor, the following steps are implemented:

Acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be detected;

Extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features;

Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector;

The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

The present application also provides a computer program product. The computer program product includes executable instructions, and when the executable instructions are executed by a processor, the following steps are implemented:

Acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be detected;

Extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features;

Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector;

The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

The above-mentioned road damage detection method, device, computer equipment, storage medium and computer program product obtain the acceleration data set and additional attribute feature data set of the road to be detected within a preset detection period; the acceleration data set is obtained by detecting the vibration acceleration signal generated by the vehicle load through an implanted sensor set inside the road to be detected, and the acceleration data set is extracted by a recursive neural network to obtain the acceleration feature, and the acceleration feature and the additional attribute feature in the additional attribute feature data set are spliced to obtain a fused feature vector, and then the fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected. Using this method, the vibration acceleration signal is the vibration response generated by the road to be detected when the vehicle load passes through the road to be detected, so that the acceleration data is processed and analyzed, the condition of the internal structure of the road can be detected, and the road damage result corresponding to the road to be detected can be obtained, thereby improving the damage detection accuracy of the road to be detected.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a process flow of a road damage detection method in one embodiment of the present application;

FIG2 is a schematic diagram of the structure of a road damage detection system in one embodiment of the present application;

FIG3 is a flow chart of an example of road damage detection in another embodiment of the present application;

FIG4 is a structural block diagram of a road damage detection device in one embodiment of the present application;

FIG5 is a schematic diagram of a flow chart of a method for determining a road sensor deployment position in one embodiment of the present application;

FIG6 is a schematic diagram of preset deployment points in one embodiment of the present application;

FIG7 is a schematic diagram of determining reference layout points in one embodiment of the present application;

FIG8 is a schematic diagram of determining a target deployment point group in one embodiment of the present application;

FIG9 is a flow chart of step 102 in one embodiment of the present application;

FIG10 is a flow chart of step 504 in one embodiment of the present application;

FIG11 is a flow chart of step 104 in one embodiment of the present application;

FIG12 is a schematic diagram of a flow chart of step 106 in one embodiment of the present application;

FIG13 is a schematic diagram of a process for determining a preset deployment point in one embodiment of the present application;

FIG14 is a schematic diagram of a method for determining a road sensor deployment position in one embodiment of the present application;

FIG15 is a structural block diagram of a device for determining a road sensor deployment position in one embodiment of the present application;

FIG16 is a diagram showing an application environment of a road damage detection method according to an embodiment of the present application;

FIG17 is a schematic diagram of a process flow of a road damage detection method in one embodiment of the present application;

FIG18 is a schematic diagram of a flow chart of steps for obtaining an acceleration data set in one embodiment of the present application;

FIG19 is a flowchart of the steps of obtaining an additional attribute feature data set in another embodiment of the present application;

FIG20 is a schematic diagram of the internal structure of a long short-term memory recursive neural network in one embodiment of the present application;

FIG21 is a flow chart of steps for determining a target road management strategy in one embodiment of the present application;

FIG22 is a flow chart of a method for training a road damage detection model in one embodiment of the present application;

FIG23 is an example flow chart of a road damage detection method applied to a concrete pavement in one embodiment of the present application;

FIG24 is a structural block diagram of a road damage detection device in one embodiment of the present application;

FIG. 25 is a diagram showing the internal structure of a computer device in one embodiment of the present application.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

The road damage detection method provided in the embodiment of the present application is applied to the road infrastructure monitoring system, as shown in FIG2 , the system includes a cloud platform 201 and a collection box 202, the cloud platform is used to control the collection box 202 to collect sensor data information of the road, the collection box 202 collects the sensor data information of the road through sensors embedded in multiple fixed positions of the road, and sends the sensor data information of the road to the cloud platform 201. Among them, the cloud platform 201 determines the target damage area of the road according to the sensor data information of the road and the image information of the road, and identifies the damage type and damage level of the target damage area based on multiple sample damage ranges of different degrees through the damage identification network, and obtains the target damage information of the road, so that the accuracy of judging the location of the road damage can be improved by determining the target damage area without manual detection, then, the damage type of the target damage area is identified, the data processing amount of judging the road damage information is reduced, and the detection efficiency of the road damage information is improved, and finally, the damage level of the target damage area is identified through multiple sample damage ranges of different degrees, and the accuracy of road damage detection is improved.

In one embodiment, as shown in FIG. 1 , a road damage detection method is provided, which is described by taking the method applied to a cloud platform 201 as an example, and includes the following steps S101 to S104 .

Step S101, obtaining the road sensor data information, road image information, and multiple different sample damage ranges sent by the collection box, and determining the current damage range of the road and the road damage sensor map based on the road sensor data information.

Among them, the sample damage range includes the damage range of different damage types.

In this embodiment, the cloud platform obtains information collected by the collection box from multiple sensors built into the road, obtains the sensor information of the road detected by each sensor, and the cloud platform uses the location information of each sensor preset in the cloud platform, and the sensor information of each sensor, that is, the sensor information of the road detected by each sensor, as the sensor data information of the road. The cloud platform obtains the image information of the road through the road image collected by the image acquisition device set in the collection box. The image acquisition device can be, but is not limited to, a camera, a SLR camera and other photographic equipment. Then, the cloud platform obtains the damage sensor data of the road under different damage degrees for each damage type of the road in the database as multiple sample damage ranges of different degrees of the road. Among them, the damage types of the road include faults, cracks, misalignments, voids, collapses, depressions, potholes, etc. The damage degree is determined according to different damage types, and the damage degree includes mild damage, moderate damage, and severe damage, etc. For example, the degree corresponding to a crack of 1m is moderate damage, the degree corresponding to a misalignment of 10cm is severe damage, and the degree corresponding to a void of 2mm is mild damage, etc. The damage range is the range of the damage sensor data of the road. The sensor data information can be represented in the form of a chart, and the collection box collects the sensor data information of each sensor in real time and stores it in real time in the collection box. The image acquisition device set in the collection box can collect road images and road videos in real time and store them in real time in the collection box.

The sensor is a vibration sensor, and the sensing information of the sensor is the vibration frequency and vibration amplitude of the vehicle colliding with the ground within the detection range of the sensor when the vehicle passes through the road.

Then, the cloud platform analyzes the current damage range of the road based on the sensor location information and sensor information in the sensor data information of the road, and determines the damage sensor map of the road within the damage range. The specific determination process will be described in detail later.

Step S102, based on the image information of the road and the current damage range of the road, a damage location recognition strategy is used to identify the target damage area of the road, and based on the damage sensor map of the road, a damage identification network is used to identify the damage type of the road.

In this embodiment, the cloud platform narrows the current damage range of the road based on the image information of the road, and identifies the target damage area in the current damage range through the damage location recognition strategy. The target damage area contains multiple sub-damage areas, and each sub-damage area may not be connected. The specific recognition process will be described in detail later. Then, the cloud platform inputs the damage sensor map of the road into the damage identification network to identify the damage type corresponding to the damage sensor map. The specific data processing process will be described in detail later.

Step S103, obtaining damage sensor data of the target damage area, and based on the damage type of the road and the samples of each degree of the damage type, In this damage range, the damage level corresponding to the damage sensor data of the target damage area is calculated.

In this embodiment, the cloud platform obtains the damage sensing data of the target damage area based on each sensor, and then the cloud platform determines the damage degree of the damage type corresponding to the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type, and the sample damage range corresponding to the damage sensing data of the target damage area, and determines the damage level of the target damage area based on the damage degree of the damage type corresponding to the target damage area. Among them, the cloud platform pre-stores the level corresponding to each damage degree of each damage type. The same damage degree of different damage types may correspond to different damage levels. For example, when the degree of cracking is moderate damage, the damage level is level 3, when the degree of depression is moderate damage, the corresponding damage level is level 1, and when the degree of hollowing is moderate damage, the damage level is level 2, where the lower the level, the more serious the damage, that is, level 1 is the most serious damage.

Step S104: taking the target damage area, the damage level of the target damage area, and the damage type of the target damage area as target damage information of the road.

In this embodiment, the cloud platform uses the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road.

Based on the above scheme, the target damage area of the road is determined through the sensor data information of the road and the image information of the road, and the damage type and damage level of the target damage area are identified through the damage identification network based on multiple sample damage ranges of different degrees, so as to obtain the target damage information of the road. Therefore, the target damage area can be determined without manual detection, thereby improving the accuracy of judging the location of road damage. Then, the damage type of the target damage area is identified, which reduces the data processing amount for judging road damage information and improves the detection efficiency of road damage information. Finally, the damage level of the target damage area is identified through multiple sample damage ranges of different degrees, thereby improving the accuracy of road damage detection.

Optionally, the sensor data information of the road includes sensor information of multiple sensors of the road and location information of each sensor. Based on the sensor data information of the road, the current damage range of the road and the damage sensor map of the road are determined, including: based on the sensor information of each sensor and the location information of each sensor, three-dimensional sensor map data of the road is established, and damage map data that meets the road damage conditions is screened in the three-dimensional sensor map data of the road, and the range containing all damage map data in the three-dimensional sensor map data is used as the current damage range of the road; the damage map data includes the location information of the corresponding pixel points of the damage map data and the sensor information of the corresponding pixel points of the damage map data; based on the location information of each map data within the current damage range and the sensor information of each map data, a damage sensor map corresponding to each map data of the current damage range is established.

In this embodiment, the cloud platform 201 establishes a three-dimensional data map based on the sensing information of each sensor and the position information of each sensor, and a geodetic coordinate system with the range collected by all sensors as the boundary. Then, the cloud platform fills the sensing data collected by each sensor and the position information of the three-dimensional data map corresponding to the position information of the sensor into the three-dimensional data map to obtain the three-dimensional sensing map data of the road. Among them, the three-dimensional sensing map data includes damage data of pixel points corresponding to multiple map data (i.e., sensing information of pixel points corresponding to the map data).

The cloud platform presets a damage data threshold, and selects the image data corresponding to the pixel points of the damage data greater than the damage data threshold in the three-dimensional sensor image data of the road as the damage image data. Then the cloud platform uses the range of all damage image data in the three-dimensional sensor image data as the current damage range of the road. Among them, the damage image data includes the position information of the corresponding pixel points of the damage image data, and the sensor information of the corresponding pixel points of the damage image data. Based on the position information of each image data within the current damage range and the sensor information of each image data, the cloud platform establishes a damage sensor map corresponding to each image data in the current damage range according to the distribution of the position information of each damage image data.

Based on the above scheme, by establishing three-dimensional sensor map data, the current damage range is screened, and based on the damage map data of the damage range, a damage sensor map is established, thereby improving the calculation accuracy of the damage sensor map.

Optionally, based on the image information of the road and the current damage range of the road, a damage location identification strategy is used to identify a target damage area of the road, including: establishing three-dimensional map data of the image information of the road, and identifying the damage location area of the image information of the road through a damage image recognition network; establishing a correspondence between the three-dimensional sensing map data and the three-dimensional map data, and identifying a sub-damage range corresponding to the damage location area within the current damage range based on the correspondence; clustering each damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculating the average distance of each damage map data in each damage map data group from the sub-damage range; screening each target damage map data in the damage map data group corresponding to the average distance below a preset distance threshold, and treating the sub-damage range and the range contained in the target damage map data corresponding to the sub-damage range as a sub-damage area; treating all sub-damage areas as the target damage area of the road.

In this embodiment, the cloud platform 201 establishes the three-dimensional image data of the image information of the road based on the earth coordinate system, and identifies the damage position area corresponding to the road damage position in the image information of the road through the damage image recognition network. The damage image recognition network is a convolutional neural network based on image feature recognition. Based on the image ratio of the three-dimensional sensor image data and the number of pixels contained in the image of the three-dimensional image sensor data, the cloud platform performs equal-proportion and equal-pixel processing on the three-dimensional sensor image data, and establishes a corresponding relationship between the processed three-dimensional image data and the pixels of the same position information in each three-dimensional sensor image data. Then, based on the corresponding relationship, the cloud platform determines the damage position area within the current damage range of the three-dimensional sensor image data corresponding to the damage position area of the three-dimensional image data. The cloud platform uses the damage position area within the current damage range as the sub-damage range of the current damage range. Then, the cloud platform calculates the straight-line distance between each damage image data within the current damage range and the sub-damage range, and performs clustering processing according to the distance between each damage image data and the sub-damage range to obtain multiple damage image data groups.

The cloud platform presets a distance threshold and calculates the average distance between each damage map data in each damage map data group and the sub-damage range. Then, the cloud platform selects each target damage map data in the damage map data group corresponding to the average distance below the preset distance threshold, and takes the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as the sub-damage area, and takes all the sub-damage areas as the target damage area of the road.

The cloud platform reacquires sample image information of multiple damage types and sample damage image information in each sample image information. Then, for each sample image information of a damage type, the cloud platform inputs the sample image information of the damage type and the sample damage image information in the sample image information of the damage type into the initial damage image recognition network, trains the damage recognition parameters of the damage type of the initial damage image recognition network, and obtains the damage image recognition network. The initial damage image recognition network is a convolutional neural network based on image feature recognition.

Based on the above scheme, the current damage range of the road is limited through the image information of the road, and the current damage range is divided into multiple sub-damage areas, thereby improving the accuracy of identifying the damage information corresponding to each sub-damage area of the road.

Optionally, based on the damage sensor map of the road, the damage type of the road is identified through a damage identification network, including: extracting damage feature data of each sub-damage area of the damage sensor map, and inputting each damage feature data into the damage identification network respectively to obtain the sub-damage type corresponding to each damage feature data; and using the sub-damage type corresponding to each damage feature data as the damage type of the road.

In this embodiment, the cloud platform extracts damage feature data of each sub-damage area of the damage sensor map. The damage feature data is a sub-damage sensor map in the damage sensor map corresponding to the self-damage area. The cloud platform inputs each damage feature data into the damage identification network respectively, and identifies the sub-damage type corresponding to each damage feature data through the damage identification network. Then the cloud platform uses the sub-damage type corresponding to each damage feature data as the damage type of the road. The damage identification network is a reinforcement learning neural network, which inputs the sample damage sensor maps of multiple damage types into the initial reinforcement learning neural network, trains the identification parameter range of each damage type corresponding to the initial reinforcement learning neural network, and obtains the damage identification network.

Based on the above scheme, through the damage identification network, based on the damage feature data of each sub-damage area, the sub-damage type of each sub-damage area is identified, thereby improving the efficiency of identifying the damage type of the damage area.

Optionally, based on the damage type of the road and the sample damage ranges of each degree of the damage type, the damage level corresponding to the damage sensor data of the target damage area is calculated, including: for each sub-damage area, based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage area, identifying the sample damage range to which the damage sensor data of the sub-damage area belongs, and obtaining the degree of the sub-damage type corresponding to the sub-damage area; based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, determining the damage level corresponding to the sub-damage area through a damage level classification strategy.

In this embodiment, the cloud platform identifies the sample damage range to which the damage sensing data of the sub-damage area belongs based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage area for each sub-damage area, and obtains the degree of the sub-damage type corresponding to the sub-damage area. Then, the cloud platform determines the damage level corresponding to the sub-damage area based on the damage level corresponding to the degree of each damage type preset in the cloud platform, the sub-damage type corresponding to the sub-damage area, and the degree of the sub-damage type corresponding to the sub-damage area.

In this embodiment, the cloud platform identifies the damage level of each sub-damage area based on the preset damage level corresponding to the degree of each damage type of the cloud platform, thereby improving the efficiency of identifying the damage level.

Optionally, after the target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as the target damage information of the road, the following further includes: for each sub-damage area, based on the sub-damage type of the sub-damage area and the degree of the sub-damage type of the sub-damage area, determining a damage repair strategy for the sub-damage area; arranging the repair sequence of each sub-damage area in descending order of the damage level of each sub-damage area to obtain a repair sequence for each sub-damage area; and Fill it into the repair sequence to obtain the repair task information of the target damaged area, and send the warning information including the repair task information and the damage information of the target damaged area to the display module.

In this embodiment, the cloud platform presets the maintenance method of each damage type and the maintenance resource consumption information corresponding to the different degrees of each damage type. Then, for each self-damaged area, the cloud platform determines the maintenance method of the sub-damaged area based on the sub-damage type of the sub-damaged area, and determines the maintenance resource consumption information of the sub-damaged area based on the degree of the sub-damage type of the sub-damaged area. Then the cloud platform uses the maintenance method of the sub-damaged area and the maintenance resource consumption information of the sub-damaged area as the damage repair strategy of the sub-damaged area. For example, the sub-damage type of the sub-damaged area and the degree of the sub-damage type of the sub-damaged area are depressions, and the degree of the depression is moderate damage. Based on the maintenance method of each damage type preset in the cloud platform and the maintenance resource demand information corresponding to the different degrees of each damage type, the maintenance method corresponding to the depression is grouting repair, and the maintenance resource consumption information corresponding to the maintenance method of the depression with moderate damage is 0.3 tons/m ^3. The cloud platform clears the maintenance order of each sub-damaged area according to the damage level of each sub-damaged area from high to low, and obtains the repair sequence of each sub-damaged area. Finally, the cloud platform fills the damage repair strategy of each sub-damage area into the repair sequence to obtain the repair task information of the target damage area. After obtaining the repair task information, the cloud platform sends the warning information containing the repair task information and the damage information of the target damage area to the display module.

Based on the above scheme, the repair task information of the target damaged area is determined through the damage information of the target damaged area, thereby improving the accuracy of the repair task information of the determined target damaged area.

Optionally, the method also includes: in response to the user's sensor acquisition system update operation, obtaining the acquisition system update information of each sensor, and sending the acquisition system update data information to the acquisition box; the acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information.

In this embodiment, when the user needs to update the sensor's acquisition system, the cloud platform responds to the user's sensor acquisition system update operation and obtains the acquisition system update information of each sensor. The cloud platform then sends the acquisition system update data information to the acquisition box. The acquisition box sends the acquisition system update data information to each sensor respectively, and controls the sensor to update the current acquisition system data information to the acquisition system update data information.

Based on the above solution, the cloud platform is used to control the collection box in real time to update the sensor, avoiding the process of manually updating each sensor and improving the update efficiency of the sensor.

Optionally, the method also includes: in response to the user's sensor acquisition task upload operation, generating an acquisition instruction for each sensor, and sending the acquisition instruction to the acquisition box; the acquisition instruction includes the acquisition task of each sensor, and the acquisition instruction is used to instruct each sensor to execute the acquisition task in the acquisition instruction.

In this embodiment, when the user needs to adjust the acquisition task of the sensor, the cloud platform responds to the user's sensor acquisition task upload operation, generates an acquisition instruction for each sensor based on the acquisition task, and then the cloud platform sends the acquisition instruction to the acquisition box; wherein the acquisition instruction includes the acquisition task of each sensor. The collector sends the acquisition instruction to each sensor respectively, and controls each sensor to execute the acquisition task in the acquisition instruction. Among them, the acquisition task also includes the numerical values corresponding to the working status of heat dissipation and dehumidification of the collection box when executing the acquisition task, and the time point when the collection box starts/ends the acquisition. Among them, the cloud platform will authenticate different users when different users access the cloud platform. Different users have different permissions to access the cloud platform, and different users can only query the user's own operation information on the cloud platform.

In one embodiment, as shown in FIG. 2 , a road damage detection system is provided, which includes a cloud platform 201 and a collection box 202 .

The collection box 202 is communicatively connected with the cloud platform 201; the collection box 202 is used to collect sensor data information of the road; the cloud platform 201 is used to obtain image information of the road, multiple different sample damage ranges, and based on the sensor data information of the road, determine the current damage range of the road and the damage sensor map of the road; the sample damage range includes damage ranges of different damage types; based on the image information of the road and the current damage range of the road, a target damage area of the road is identified through a damage location recognition strategy, and based on the damage sensor map of the road, a damage type of the road is identified through a damage identification network; damage sensor data of the target damage area is obtained, and based on the damage type of the road and the sample damage ranges of each degree of the damage type, the damage level corresponding to the damage sensor data of the target damage area is calculated; the target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as the target damage information of the road.

In this embodiment, the collection box 202 is connected to the cloud platform 201. The collection box 202 sends the sensor data information of the road detected by each sensor to the cloud platform 201, and the cloud platform 201 processes the sensor data information of the road, the road image information collected by the image acquisition device, The target damage information of the road is obtained by using the sample damage ranges of different degrees in the database. A download port is provided in the collection box 202, and the download port is used by the user to download the sensor data information of the historical road stored in the collection box 202, as well as the road images and road videos of the historical road stored in the collection box 202.

Among them, the cloud platform 201 transmits control information (i.e., the collection system update data information and control instructions) to the collection box 202, and the collection box 202 transmits data information (i.e., the sensor data information of the road and the image information of the road) to the cloud platform 201. The authentication module in the cloud platform 201 is used to store the usage rights of each user, and the configuration issuance module is used to respond to the user's sensor collection system update operation and obtain the collection system update information of each sensor; the transceiver control module is used to transmit data with the collection box 202, and the storage module is used to store each data information processed by the cloud platform 201; the analysis and processing module is used to execute the content between step S101 and step S102; the diagnosis module is used to execute the content between step S103 and step S104; the display module is used to display the damage information of the target damaged area and the repair task information of the target damaged area; the alarm module is used to execute the task of alerting the user after receiving the early warning information.

In one embodiment, as shown in FIG3 , a road damage detection example is provided, which includes the following steps S301 to S312 .

Step S301, acquiring sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges.

Step S302: Based on the sensing information of each sensor and the position information of each sensor in the sensing data information, three-dimensional sensing map data of the road is established, and damage map data that meets the road damage condition is screened in the three-dimensional sensing map data of the road, and the range containing all damage map data in the three-dimensional sensing map data is used as the current damage range of the road. Step S303: Based on the position information in each map data within the current damage range and the sensing information in each map data, a damage sensing map corresponding to each map data of the current damage range is established.

Step S304, creating three-dimensional graph data of the image information of the road, and identifying the damaged location area of the image information of the road through a damaged image recognition network.

Step S305: establishing a correspondence between the three-dimensional sensing image data and the three-dimensional image data, and identifying a sub-damage range corresponding to the damage position area within the current damage range based on the correspondence.

Step S306, clustering the damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculating the average distance between each damage map data in each damage map data group and the sub-damage range.

Step S307, screening each target damage map data in the damage map data group corresponding to the average distance below the preset distance threshold, and taking the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as the sub-damage area, and taking all the sub-damage areas as the target damage areas of the road.

Step S308, extracting damage feature data of each sub-damage area of the damage sensing map, and inputting each damage feature data into the damage identification network respectively to obtain the sub-damage type corresponding to each damage feature data.

Step S309, and taking the sub-damage type corresponding to each damage feature data as the damage type of the road.

Step S310, for each sub-damage area, based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage area, identify the sample damage range to which the damage sensing data of the sub-damage area belongs, and obtain the degree of the sub-damage type corresponding to the sub-damage area.

Step S311, based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, the damage level corresponding to the sub-damage area is determined through a damage level classification strategy.

Step S312: The target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as target damage information of the road.

It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application also provides a road damage detection device for implementing the road damage detection method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the following is provided The specific limitations in one or more embodiments of the road damage detection device can refer to the above limitations on the road damage detection method, which will not be repeated here.

In one embodiment, as shown in FIG. 4 , a road damage detection device is provided, including: an acquisition module 410 , an identification module 420 , a re-acquisition module 430 and a determination module 440 .

The acquisition module 410 is used to acquire the sensor data information of the road sent by the collection box, the image information of the road sent by the collection box, and a plurality of different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road. The sample damage range includes damage ranges of different damage types.

The identification module 420 is used to identify the target damage area of the road based on the image information of the road and the current damage range of the road through a damage location identification strategy, and to identify the damage type of the road based on the damage sensor map of the road through a damage identification network.

The re-acquisition module 430 is used to acquire the damage sensor data of the target damage area, and calculate the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type.

The determination module 440 is configured to use the target damage area, the damage level of the target damage area, and the damage type of the target damage area as target damage information of the road.

Optionally, the sensor data information of the road includes sensor information of multiple sensors on the road and location information of each of the sensors.

The acquisition module 410 is specifically used to: establish three-dimensional sensor map data of the road based on the sensor information of each sensor and the position information of each sensor, and screen damage map data that meets the road damage condition in the three-dimensional sensor map data of the road, and use the range of all damage map data in the three-dimensional sensor map data as the current damage range of the road; the damage map data includes the position information of the corresponding pixel points of the damage map data and the sensor information of the corresponding pixel points of the damage map data; based on the position information of each map data within the current damage range and the sensor information of each map data, establish a damage sensor map corresponding to each map data in the current damage range.

Optionally, the recognition module 420 is specifically used to: establish three-dimensional map data of the image information of the road, and identify the damage location area of the image information of the road through a damage image recognition network; establish a correspondence between the three-dimensional sensor image data and the three-dimensional map data, and identify the sub-damage range corresponding to the damage location area within the current damage range based on the correspondence; cluster each damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculate the average distance of each damage map data in each damage map data group from the sub-damage range; filter each target damage map data in the damage map data group corresponding to the average distance below a preset distance threshold, and use the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as a sub-damage area, and use all sub-damage areas as target damage areas of the road.

Optionally, the identification module 420 is specifically used to: extract damage feature data of each sub-damage area of the damage sensor map, and input each damage feature data into the damage identification network respectively to obtain the sub-damage type corresponding to each damage feature data; and use the sub-damage type corresponding to each damage feature data as the damage type of the road.

Optionally, the re-acquisition module 430 is specifically used to: for each sub-damage area, based on the sample damage ranges of each degree corresponding to the sub-damage type of the sub-damage area, identify the sample damage range to which the damage sensing data of the sub-damage area belongs, and obtain the degree of the sub-damage type corresponding to the sub-damage area; based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, determine the damage level corresponding to the sub-damage area through a damage level classification strategy.

Optionally, the device further comprises: a strategy determination module and a task determination module. The strategy determination module is used to determine, for each sub-damage area, a damage repair strategy for the sub-damage area based on the sub-damage type of the sub-damage area and the degree of the sub-damage type of the sub-damage area.

The task determination module is used to arrange the maintenance order of each sub-damage area in descending order of the damage level of each sub-damage area to obtain a repair sequence for each sub-damage area; and fill the damage repair strategy of each sub-damage area into the repair sequence to obtain the repair task information of the target damage area, and send the warning information including the repair task information and the damage information of the target damage area to the display module.

Optionally, the device further includes an update module. The update module is used to obtain the acquisition system update information of each sensor in response to the user's sensor acquisition system update operation, and send the acquisition system update data information to the acquisition box; the acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information.

Optionally, the device further includes: an instruction sending module. The instruction sending module is used to generate an acquisition instruction for each sensor in response to the user's sensor acquisition task upload operation, and send the acquisition instruction to the acquisition box; the acquisition instruction includes the acquisition task of each sensor, and the acquisition instruction is used to instruct each sensor to execute the acquisition task in the acquisition instruction.

Each module in the above-mentioned road damage detection device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

In one embodiment, as shown in FIG5 , a method for determining the deployment location of a road sensor is provided. This embodiment is illustrated by applying the method to a terminal. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps 102 to 110.

Step 102, determining the layout points corresponding to each road condition from a plurality of preset layout points of the target road.

In the embodiment of the present application, the target road is a road where sensors need to be deployed, and the preset deployment points refer to location points pre-selected on the target road where sensors can be deployed, as shown in Figure 6. The light gray points in the figure are the preset deployment points for a section of road, and the final target deployment points should be generated from the preset deployment points.

Road conditions refer to the state of a road. For example, a road under the influence of a certain disease can be a road condition, and a road in a certain period of time can also be a road condition. The layout points corresponding to the road conditions are the layout points among the preset layout points that can reflect the stress conditions of the road under the road condition, such as the layout points where the road is subjected to greater stress, the layout points where the probability of vehicles passing by is higher, etc. The specific selection criteria can be determined by those skilled in the art according to actual needs, and the embodiments of this application are not specifically limited here.

Step 104, determining a reference layout point from the layout points corresponding to each road condition, constructing a reference layout point group based on the reference layout point, and taking the layout points corresponding to each road condition that do not belong to the reference layout point group as candidate layout points.

In an embodiment of the present application, in order to make the number of final target layout points smaller, a benchmark layout point group can be determined based on the layout points corresponding to each road condition, and the layout points corresponding to each road condition that do not belong to the benchmark layout point group are used as candidate layout points, and the candidate layout points are gradually added to the benchmark layout point group to obtain the final target layout points that can reflect the stress conditions of the target road under various road conditions.

The selection criteria of the reference layout points in the reference layout point group can be determined by those skilled in the art according to actual needs. Since it is necessary to obtain a smaller number of target layout points (the number of reference layout points selected at the beginning should not be too large) and to improve the efficiency of adding candidate layout points later (the more candidate layout points, the longer it takes to add candidate layout points, that is, the number of reference layout points selected at the beginning should not be too small), it is necessary to control the number of reference layout points within a reasonable range.

For example, if there are overlapping layout points among the layout points corresponding to various road conditions, the overlapping layout points are the layout points that can simultaneously reflect the stress conditions of the target road under multiple road conditions, and this point is likely to belong to the final target layout point; therefore, the overlapping layout points can be used as reference layout points.

Alternatively, the reference layout points may be selected according to the distribution patterns of the layout points under different road conditions (e.g., distribution positions, distances between distribution centers, etc., where the distribution center may refer to a point with the smallest sum of distances from all layout points). When there are road conditions with greatly different distribution patterns, some points may be selected from the layout points corresponding to the above road conditions as reference layout points; when there are road conditions with relatively similar distribution patterns, reference layout points may be selected based on the overlapping layout points in the two road conditions.

The above process is explained with a specific example, as shown in FIG7 , if there are road conditions A, B, C, D, and E, where the layout points corresponding to road conditions A and D are located on the left side of the road, the layout points corresponding to road conditions B and C are located on the right side of the road, and the layout points corresponding to road condition E are located in the center of the road, then it can be determined that the distribution patterns of A and D, and B and C are relatively close. A candidate benchmark layout point group can be first determined based on road conditions with relatively close distribution patterns. Considering that when the distribution patterns of the layout points are relatively close, the layout points corresponding to one road condition can roughly reflect the road stress conditions in another road condition, the layout points that overlap in the two road conditions can be first selected as candidate benchmark layout points.

Then, the final reference layout point group is obtained based on the candidate reference layout points determined based on A and D, B and C, and the layout points corresponding to E. Considering that the layout points corresponding to one road condition may not reflect the road stress conditions in another road condition when the distribution patterns of the layout points are quite different, if only the layout points corresponding to one road condition are used as reference layout points, when the candidate layout points of another road condition are subsequently matched with the reference layout points, it is highly likely that the candidate layout points still need to be added to the reference layout point group. Therefore, when determining the reference layout points, some layout points can be selected from the layout points corresponding to the above road conditions in advance as reference layout points to speed up the subsequent acquisition of the target layout points.

For example, taking the above example, since B and C correspond to only one candidate benchmark layout point, the candidate benchmark layout point can be added as the final benchmark layout point. A and D correspond to two candidate benchmark layout points. In order to minimize the number of final benchmark layout points, one point (for example, a point with a larger average value of the working condition data under two road conditions) can be selected from the two candidate benchmark layout points according to the force conditions of the target road at the two candidate benchmark layout points under two road conditions (which can be represented by pressure data, acceleration signal energy data, etc. collected at the layout points, hereinafter referred to as working condition data) as the benchmark layout point. Similarly, one point can also be selected from the layout points corresponding to E as the benchmark layout point to obtain the final benchmark layout point group.

After the reference layout point group is determined, the layout points under each road condition that do not belong to the reference layout point group are taken as candidate layout points, as shown in FIG7 .

Step 106, for each candidate layout point, determine the target road condition corresponding to the candidate layout point, and determine the first condition data corresponding to each benchmark layout point under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition, and determine the correlation coefficient between the candidate layout point and the benchmark layout point group based on each first condition data and the second condition data, wherein the condition data is used to characterize the stress condition of the road at the layout point.

In the embodiment of the present application, the target road condition corresponding to the candidate layout point is the road condition that includes the candidate layout point among the layout points. If there are multiple layout points that include the road conditions of the candidate layout point, there will be multiple target road conditions.

Working condition data refers to any data that can characterize the force conditions of the road at a point, such as the force exerted on the road at that point, the acceleration signal energy data collected at that point over a period of time, the frequency of vehicles passing by that point, and the pressure exerted on the road when vehicles pass by, etc. The embodiments of the present application do not make specific limitations on this.

The corresponding working condition data of each reference layout point and candidate layout point in the reference layout point group under the target road working condition can be calculated respectively. According to each first working condition data corresponding to each reference layout point and the second working condition data corresponding to the candidate layout point, the correlation coefficient between the reference layout point group and the candidate layout point can be determined. For example, in the case where the working condition data is a numerical value (such as a pressure value), the correlation coefficient can be made inversely proportional to the smallest difference between the first working condition data and the second working condition data, that is, if the difference between the second working condition data and any first working condition data is small, the correlation coefficient between the candidate layout point and the reference layout point group is large. In the case where the working condition data is a sequence (such as the energy of the acceleration signal collected within a period of time), the correlation coefficient between each first working condition data and the second working condition data (such as the Pearson correlation coefficient, the Spearman correlation coefficient, etc.) can be calculated according to any algorithm for calculating the correlation coefficient between two sequences, and then the largest correlation coefficient among the correlation coefficients is used as the correlation coefficient between the candidate layout point and the reference layout point group. The correlation coefficient may also be calculated in other ways, which are not specifically limited in the embodiments of the present application.

If a candidate layout point corresponds to multiple target road conditions, the correlation coefficients corresponding to the candidate layout point under multiple target road conditions can be calculated separately, and then the largest correlation coefficient, or the smallest correlation coefficient, or the average of the correlation coefficients, or other numerical values that can characterize the characteristics of multiple correlation coefficients are used as the correlation coefficient corresponding to the candidate layout point. The embodiment of the present application does not make any specific limitations on this.

Step 108 : selecting a target layout point from each candidate layout point according to the correlation coefficient corresponding to each candidate layout point.

In the embodiment of the present application, a candidate layout point with a small correlation coefficient, that is, based on the original reference layout point, cannot accurately reflect the road stress condition at the candidate layout point, can be used as a target layout point. For example, a correlation coefficient threshold can be set in advance, and the candidate layout point with a correlation coefficient less than the correlation coefficient threshold can be used as the target layout point. Alternatively, the candidate layout points can be sorted from small to large according to the correlation coefficient, and the candidate layout points with a preset number of candidates in the front can be used as the target layout point, which is not specifically limited in the embodiment of the present application.

In addition to the correlation coefficient, the working condition data can also be used as a criterion for selecting the target layout point. For example, for the target layout point that has been screened out based on the correlation coefficient, all layout points under the target road working condition corresponding to the target layout point can be obtained, the sum of the working condition data of each layout point can be calculated, and then the first ratio of the working condition data of the target layout point to the sum of the working condition data of each layout point can be calculated, as well as the first ratio of the working condition data of the target layout point to the sum of the working condition data of each layout point. The second ratio of the sum of the working condition data of the layout points of the reference layout point to the sum of the working condition data of each layout point. If the first ratio is less than the second ratio and the difference between the first ratio and the second ratio is large, it means that the target layout point is not important for characterizing the stress condition of the target road under the target road working condition, and the target layout point can be deleted. If the first ratio is greater than or equal to the second ratio, or the first ratio is less than the second ratio and the difference between the first ratio and the second ratio is not large, it means that the target layout point is more important, and the target layout point can be retained.

It should be noted that the above is only an example of selecting target layout points according to the correlation coefficient and the operating condition data. In fact, those skilled in the art can also select the target layout points according to the correlation coefficient and other data, or can also select the target layout points according to the correlation coefficient and the operating condition data in other ways, and the embodiments of the present application do not specifically limit this.

Step 110, adding the target layout point to the reference layout point group to obtain a final target layout point group.

In the embodiment of the present application, each target deployment point is added to the reference deployment point group to obtain the final target deployment point group, as shown in Figure 8. After obtaining the target deployment point group, sensors can be deployed at corresponding positions of the target road according to each point in the target deployment point group to monitor the health status of the target road.

The method for determining the road sensor deployment position provided by the embodiment of the present application selects a reference deployment point group, and then respectively calculates the correlation coefficients between the remaining candidate deployment points and the reference deployment point group. When the correlation coefficients meet the requirements, it is determined that the road conditions at the candidate deployment point cannot be accurately monitored based on each reference deployment point in the reference deployment point group, and the candidate deployment point is added to the reference deployment point group, thereby obtaining the final target deployment point group. Therefore, it is possible to detect a variety of typical road diseases with fewer sensors, reducing the detection cost.

In one embodiment, as shown in FIG. 9 , in step 102 , layout points corresponding to each road condition are determined from a plurality of preset layout points of the target road, including steps 502 and 504 .

Step 502: construct road models corresponding to the target road under various road conditions.

Step 504, for each road condition, obtain the vehicle driving data corresponding to the target road under the road condition, and determine the third condition data corresponding to multiple preset layout points of the target road under the road condition based on the vehicle driving data and the road model corresponding to the target road under the road condition, and determine the layout point corresponding to the road condition from each preset layout point based on each third condition data.

In the embodiment of the present application, the working condition data can be obtained based on the road model of the target road, and the layout points corresponding to each road working condition can be determined from each preset layout point based on the working condition data. First, a road model of the target road in a normal state can be constructed. The road model should consider the structural parameters and material parameters of each layer of the road (such as the surface layer, base layer, subbase layer, etc.), the boundary conditions of each layer, the size and layout of the force transmission rods and pull rods in the road, etc., and then on this basis, the road model in the normal state is adjusted for different road working conditions to obtain road models corresponding to different road working conditions. For example, when the road working condition is that the bottom of the road plate corner is hollow, the bottom of the plate corner can be simulated at the base of the road model in the normal state to obtain the road model under the road working condition.

The vehicle driving data corresponding to the road condition refers to the data that can characterize the vehicle conditions on the road, such as the road roughness, vehicle speed, vehicle type, vehicle load, wheel track distribution probability, etc. under the road condition. The above data can be obtained by monitoring the target road in reality, or the typical data obtained by monitoring multiple roads under the road condition can be used as vehicle driving data. According to the vehicle driving data and the road model, the stress conditions of the road model at the preset layout points can be obtained, so as to simulate the stress conditions of the target road at the preset layout points and obtain the working condition data. For example, the vehicle moving load parameters can be obtained according to the above vehicle driving data. The vehicle moving load parameters are used to characterize the stress conditions of any point on the road at any time under the road condition; the vehicle moving load parameters are applied to the road model, and the road model is analyzed by the finite element method, so as to obtain the third working condition data at each preset layout point.

Since it is necessary to select points that can reflect the stress conditions of the road under the current road condition from the preset layout points, the preset layout points with larger third working condition data can be used as layout points. For example, the third working condition data threshold can be set in advance, and the preset layout points with third working condition data greater than the third working condition data threshold can be used as layout points. Alternatively, the preset layout points can be sorted from large to small according to the third working condition data, and the preset layout points with a preset number of rows in front can be used as layout points. The embodiments of the present application do not specifically limit this.

The method for determining the layout position of the road sensor provided in the embodiment of the present application constructs a road model under the road working condition, obtains the third working condition data of each preset layout point according to the vehicle driving data corresponding to the road working condition, and then selects the layout point according to the third working condition data. It is possible to determine which points among the preset layout points can better reflect the stress conditions of the road under the road working condition according to the model, and then select the reference layout point and the final target layout point from these points, which can improve the accuracy of the final target layout point.

In one embodiment, as shown in FIG. 10 , in step 504, the road working condition is determined from each preset layout point according to each third working condition data. The method further comprises steps 602 and 604.

Step 602: sort the preset layout points from large to small according to the third working condition data to obtain a preset layout point queue.

Step 604, traverse the preset layout point queue, and when the third working condition data corresponding to each preset layout point arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue, and use each preset layout point arranged before the current traversal position as the layout point corresponding to the road working condition.

In the embodiment of the present application, the preset layout points can be sorted from large to small according to the third working condition data, and the layout points corresponding to the road working condition can be determined by traversing the preset layout point queue obtained by sorting. The preset strategy refers to the termination condition for selecting the layout point, such as the third working condition data is less than the third working condition data threshold, or the number of selected layout points is equal to the number threshold, etc.

For example, when the preset strategy is that the third working condition data is less than the third working condition data threshold, the preset layout point queue can be traversed, and the third working condition data of the preset layout point arranged before the current traversal position can be compared with the third working condition data threshold. If all the third working condition data are greater than or equal to the third working condition data threshold, the next preset layout point will be traversed; if there is a third working condition data less than the third working condition data threshold, the termination condition of the traversal is triggered, and the preset layout point arranged before the current traversal position is used as the layout point corresponding to the road working condition, and the traversal process is stopped.

Alternatively, the preset strategy may also be that the sum of the third working condition data of the preset layout points arranged before the current traversal position and the sum of the third working condition data of all preset layout points are greater than the ratio threshold. The sum of the third working condition data of all preset layout points may be calculated in advance, and in the process of traversing the preset layout point queue, the sum of the third working condition data of each preset layout point arranged before the current traversal position is calculated. When the ratio between the sum and the sum of the third working condition data of all preset layout points is greater than the ratio threshold (for example, 15%), the termination condition of the traversal is triggered, the preset layout points arranged before the current traversal position are used as the layout points corresponding to the road working condition, and the traversal process is stopped.

The method for determining the layout position of the road sensor provided in the embodiment of the present application sorts each preset layout point according to the third working condition data, traverses the queue obtained after sorting, and then selects the layout point according to whether the third working condition data of each preset layout point before the current traversal position meets the preset strategy. The preset layout point that can better reflect the stress condition of the road under the current road working condition can be selected as the layout point under the current road working condition, and then the reference layout point and the final target layout point are selected from these points, which can improve the accuracy of the final target layout point.

In one embodiment, as shown in FIG. 11 , in step 104 , determining a reference layout point from the layout points corresponding to each road condition includes: step 1041 and step 1042 .

Step 1041, according to the layout points corresponding to each road condition, determine the number of layout points corresponding to each road condition.

Step 1042: taking the layout points corresponding to the largest number of layout points among the layout points as the reference layout point group.

In an embodiment of the present application, the layout points with the largest number of layout points among the corresponding layout points in each road condition can be used as the reference layout point group to reduce the number of subsequent matching of candidate layout points and the reference layout point group.

If there are multiple road conditions with the largest number of layout points, the solution of selecting the reference layout points according to the distribution law of the layout points in the aforementioned embodiment can be referred to, and the similarity between the distribution law of the layout points in these road conditions and the distribution law of the layout points in other road conditions can be calculated respectively, and the layout points corresponding to the road conditions with the highest similarity to the distribution law of the layout points in other road conditions can be selected as the reference layout points. The highest similarity to the distribution law of the layout points in other road conditions can refer to the highest sum of the similarities between the layout point distribution law of the road condition and the layout point distribution law of other road conditions, or the highest average value, etc., which is not specifically limited in the embodiments of the present application.

The method for determining the road sensor deployment position provided in the embodiment of the present application uses a group of deployment points with the largest number of deployment points as a reference deployment point group, thereby reducing the number of times that candidate deployment points and reference deployment point groups need to be matched, and improving the efficiency of obtaining a target deployment point group.

In one embodiment, as shown in FIG. 12 , in step 106 , the correlation coefficient between the candidate layout points and the reference layout point group is determined according to each of the first operating condition data and the second operating condition data, including: step 1061 and step 1062 .

Step 1061 : for any reference layout point, determine the correlation coefficient between the candidate layout point and the reference layout point according to the first operating condition data and the second operating condition data corresponding to the reference layout point.

Step 1062: Determine the correlation coefficient between the candidate layout points and the reference layout point group according to the correlation coefficient corresponding to each reference layout point.

In the embodiment of the present application, the correlation coefficient between the second working condition data and the reference layout point group can be determined according to the correlation coefficient between each first working condition data and the second working condition data. Taking the first working condition data and the second working condition data as time series data, which contain all acceleration signal energy data collected in one collection cycle, the first working condition data and the second working condition data can be calculated according to the Pearson correlation coefficient. The correlation coefficient (see formula (1)):

Where X refers to the first working condition data, Y refers to the second working condition data, r(X,Y) is the Pearson correlation coefficient, Cov(X,Y) is the covariance of X and Y, Var[X] is the variance of the first working condition data, and Var[Y] is the variance of the second working condition data.

The correlation coefficient between the second operating condition data and the reference layout point group can be determined based on all the calculated correlation coefficients. For example, the smallest correlation coefficient is used as the correlation coefficient between the second operating condition data and the reference layout point group, the largest correlation coefficient is used as the correlation coefficient between the second operating condition data and the reference layout point group, the average value of the correlation coefficient is used as the correlation coefficient between the second operating condition data and the reference layout point group, etc., which is not specifically limited in the embodiments of the present application.

The method for determining the road sensor layout position provided in the embodiment of the present application determines the correlation coefficient between the candidate layout point and the benchmark layout point group based on the correlation coefficient between each first operating condition data and the second operating condition data. The target layout point can be selected based on the correlation coefficient between the candidate layout point and all the benchmark layout points, that is, the target layout point is a point that meets the correlation coefficient requirements of all the benchmark layout points, which can improve the selection accuracy of the target layout point.

In one embodiment, in step 108, a target layout point is selected from each candidate layout point according to the correlation coefficient corresponding to each candidate layout point, including: for any candidate layout point, when the correlation coefficient corresponding to the candidate layout point is less than the correlation coefficient threshold, the candidate layout point is selected as the target layout point.

In an embodiment of the present application, when the correlation coefficient corresponding to the candidate layout point is less than the correlation coefficient threshold (a pre-set value, the value of which can be pre-set by a person skilled in the art), it means that each benchmark layout point in the benchmark layout point group cannot accurately reflect the stress condition of the target road at the candidate layout point, so the candidate layout point can be added to the benchmark layout point group as a target layout point.

Depending on the different ways of calculating the correlation coefficient of the candidate layout points, when the correlation coefficient corresponding to the candidate layout point is less than the correlation coefficient threshold, taking the candidate layout point as the target layout point also has different corresponding meanings. For example, when the correlation coefficient corresponding to the candidate layout point is the minimum value of the correlation coefficients between each benchmark layout point and the candidate layout point, taking the candidate layout point as the target layout point means that when there is a benchmark layout point that cannot accurately reflect the stress condition at the candidate layout point, the candidate layout point is added to the benchmark layout point; when the correlation coefficient corresponding to the candidate layout point is the minimum value of the correlation coefficients between each benchmark layout point and the candidate layout point, taking the candidate layout point as the target layout point means that when all benchmark layout points cannot accurately reflect the stress condition at the candidate layout point, the candidate layout point is added to the benchmark layout point. Those skilled in the art can adaptively adjust the standard for calculating the correlation coefficient in the aforementioned embodiment according to the desired effect, and the embodiments of the present application do not specifically limit this.

The method for determining the road sensor deployment position provided in the embodiment of the present application takes the candidate deployment point whose correlation coefficient is less than the correlation coefficient threshold as the target deployment point. That is, when the road stress condition at the candidate deployment point cannot be accurately detected based on the reference deployment point, the candidate deployment point is taken as the target deployment point. This can improve the monitoring accuracy of road health when the sensors are subsequently deployed according to the target deployment point.

In one embodiment, the method further includes: determining a plurality of preset layout points on the target road according to a preset layout point arrangement strategy, wherein the plurality of preset layout points are evenly distributed on the target road.

In the embodiment of the present application, in order to make the preset layout points cover the points that can characterize the stress conditions of the road under all road conditions, the preset layout points can be evenly distributed. For example, as shown in FIG13, the stress conditions of any point on the target road at each time can be calculated in advance for each road condition using the road model and vehicle driving data under the road condition, and a candidate point (point 1, point 2 and point 3 on the left side of FIG13) that best characterizes the stress conditions of the road under the road condition is selected; after obtaining the candidate points under multiple road conditions, new points can be inserted between the candidate points according to the distribution of the candidate points and the distance between the candidate points to obtain the final preset layout points and make the preset layout points evenly distributed. For example, the distance between any two candidate points in the direction of road travel can be calculated, and the smallest distance can be selected as the longitudinal distance between each preset layout point (d3 in Figure 13); the distance between any two candidate points perpendicular to the direction of road travel can be calculated, and the smallest distance can be selected as the lateral distance between each preset layout point (s2 in Figure 13); according to the longitudinal distance and the lateral distance, new points can be uniformly inserted between each candidate point, as shown on the right side of Figure 13. The points filled with shades on the right side of Figure 13 are the newly inserted points.

It should be noted that, as shown in FIG13 , since s3 cannot divide s2 and d1 cannot divide d3, the spacing between some points may be different from the spacing between other points. In this case, these points can be moved so that the spacing between these points and other points is The spacing between the points is consistent with that between other points; or these points may not be moved, and this embodiment of the present application does not specifically limit this.

Alternatively, the longitudinal distance and the lateral distance between each preset layout point, as well as the reserved distance between the preset layout point and the road edge may be preset. When it is necessary to determine the preset layout points for the target road, a row of preset layout points is first laid out according to the longitudinal distance at the reserved distance from the road edge, and then a row of preset layout points is laid out at each lateral distance, so as to complete the uniform arrangement of the preset layout points.

The method for determining the layout position of road sensors provided in the embodiment of the present application makes each preset layout point evenly distributed when presetting each preset layout point, so that each preset layout point can contain points that can reflect the road stress conditions under various road conditions, thereby improving the determination accuracy of the preset layout points, and further improving the accuracy of the final target layout points.

In order to enable those skilled in the art to better understand the embodiments of the present application, the embodiments of the present application are described below through specific examples.

14 , a flow chart of a method for determining the deployment position of a road sensor is shown.

In the embodiment of the present application, the target deployment points to be selected are the sensor deployment points for various road diseases. The deployment position of the road sensor can be determined for each cement slab of the target road. 4×5 preset deployment points can be selected on the road in advance, and then the target deployment point can be determined from the preset deployment points.

A road model without damage can be established for the target road, and the structural parameters, material parameters and boundary conditions of each layer of the road can be considered when establishing the road model. Then, based on the road model without damage, road models with different road damages can be constructed for various road damages that need to be detected. The vehicle moving load parameters are calculated by vehicle speed, vehicle load, wheel track distribution, etc., and the vehicle moving load parameters are applied to the road model to obtain the acceleration signal energy at each target layout point.

The acceleration signal energy is |a _i,j (n)| ² in FIG14 . By adding the acceleration signal energies at a preset layout point, the sum of the acceleration signal energies corresponding to the preset layout point can be obtained (E _i,j in FIG14 , N in FIG14 is the total number of acceleration signal energies). The preset layout points are sorted from large to small according to the sum of the acceleration signal energies, and the queue obtained after traversal is summed up for the sum of the acceleration signal energies corresponding to the preset layout points before the currently traversed position, and the ratio of the sum to the sum of the acceleration signal energies corresponding to all the preset layout points is calculated. When the ratio is greater than 15% for the first time, the queue is stopped, and the preset layout point before the currently traversed position is used as the layout point under this road condition.

The layout points corresponding to the road conditions with the largest number of layout points in each road condition are taken as the benchmark layout points, and the layout points that do not belong to the benchmark layout points are taken as candidate layout points. The Pearson correlation coefficient between the candidate layout points and each benchmark layout point is calculated. When the correlation coefficients between all benchmark layout points and the candidate layout points are less than 0.8, the candidate layout points are added to the benchmark layout point group as target layout points to obtain the final target layout point group.

The method for determining the deployment position of road sensors provided in the embodiment of the present application uses as few sensors as possible to maximize monitoring, thereby improving detection efficiency and reducing economic costs.

It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application also provides a device for determining the road sensor layout position for implementing the method for determining the road sensor layout position involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more road sensor layout position determination devices provided below can refer to the limitations of the road sensor layout position determination method above, and will not be repeated here.

In one embodiment, as shown in FIG. 15 , a device 1100 for determining a road sensor deployment location is provided, comprising: a first determination module 1102 , a second determination module 1104 , a third determination module 1106 , a selection module 1108 , and an adding module 1100 .

The first determining module 1102 is used to determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road.

The second determination module 1104 is used to determine the reference layout points from the layout points corresponding to each of the road conditions, construct a reference layout point group based on the reference layout points, and use the layout points that do not belong to the reference layout point group among the layout points corresponding to each of the road conditions as candidate layout points.

The third determination module 1106 is used to determine the target road condition corresponding to any of the candidate layout points, and determine the first condition data corresponding to each of the benchmark layout points under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition, and determine the correlation coefficient between the candidate layout point and the benchmark layout point group based on each of the first condition data and the second condition data, wherein the condition data is used to characterize the stress condition of the road at the layout point.

The selection module 1108 is used to select a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points.

The adding module 1110 is used to add the target layout point to the reference layout point group to obtain a final target layout point group.

The device for determining the layout position of road sensors provided in the embodiment of the present application selects a reference layout point group, and then respectively calculates the correlation coefficients between the remaining candidate layout points and the reference layout point group. When the correlation coefficients meet the requirements, it is determined that the road conditions at the candidate layout point cannot be accurately monitored based on each reference layout point in the reference layout point group, and the candidate layout point is added to the reference layout point group, thereby obtaining the final target layout point group. Therefore, it is possible to detect a variety of typical road diseases with fewer sensors, reducing the detection cost.

In one of the embodiments, the first determination module 1102 is further used to: respectively construct road models corresponding to the target road under various road conditions; for each of the road conditions, obtain vehicle driving data corresponding to the target road under the road condition, and determine third condition data corresponding to multiple preset layout points of the target road under the road condition based on the vehicle driving data and the road model corresponding to the target road under the road condition, and determine the layout point corresponding to the road condition from each of the preset layout points based on the third condition data.

In one of the embodiments, the first determination module 1102 is further used to: sort each of the preset layout points from large to small according to the third operating condition data to obtain a preset layout point queue; traverse the preset layout point queue, and when the third operating condition data corresponding to each of the preset layout points arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue, and use each of the preset layout points arranged before the current traversal position as the layout points corresponding to the road operating condition.

In one of the embodiments, the second determination module 1104 is further used to: determine the number of layout points corresponding to each of the road conditions according to the layout points corresponding to each of the road conditions; and use the layout points corresponding to the largest number of layout points among the layout points as the reference layout point group.

In one embodiment, the third determination module 1106 is further used to: for any of the benchmark layout points, determine the correlation coefficient between the candidate layout point and the benchmark layout point according to the first operating condition data and the second operating condition data corresponding to the benchmark layout point; and determine the correlation coefficient between the candidate layout point and the benchmark layout point group according to the correlation coefficient corresponding to each of the benchmark layout points.

In one of the embodiments, the selection module 1108 is further used to: for any of the candidate layout points, if the correlation coefficient corresponding to the candidate layout point is less than a correlation coefficient threshold, use the candidate layout point as a target layout point.

In one embodiment, the device further comprises a fourth determination module. The fourth determination module is used to determine a plurality of preset layout points on the target road according to the preset layout point arrangement strategy, wherein the plurality of preset layout points are evenly distributed on the target road.

Each module in the above-mentioned device for determining the layout position of the road sensor can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

In one embodiment, a computer device is provided, which may be a server or a cloud platform, and its internal structure diagram may be shown in FIG25. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external cloud platform in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a road damage detection method and a method for determining the location of a road sensor are implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

Those skilled in the art will understand that the structure shown in FIG. 25 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of any one of the above methods are implemented.

In one embodiment, a non-volatile computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

In one embodiment, a computer program product is provided, comprising executable instructions, which, when executed by a processor, implement the steps of any of the above methods.

One embodiment of the present application provides a road damage detection method, which can be applied in an application environment as shown in FIG16. It includes an implanted sensor 102 buried inside the road, a roadside acquisition device 104 and a terminal (the remote terminal is not shown in FIG16), and the implanted sensor 102, the acquisition device 104 and the terminal can communicate through a network. The terminal obtains an acceleration data set and an additional attribute feature data set of the road to be detected within a preset detection period. The acceleration data set is obtained by detecting the vibration acceleration signal generated by the vehicle load by the implanted sensor 102 set inside the road to be detected. Then, the terminal extracts features from the acceleration data set according to a recursive neural network to obtain acceleration features. The acceleration features and the additional attribute features in the additional attribute feature data set are feature spliced to obtain a fused feature vector. Then, the terminal inputs the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected. Thus, damage detection of the road interior of the road to be detected is realized, and the accuracy of road damage detection is improved.

The terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and IoT devices, and the IoT devices may be smart vehicle-mounted devices, etc.

It is understandable that, in addition to being applicable to the terminals in the above-mentioned application scenarios, the method can also be applied to servers, and can also be applied to systems including terminals and servers, and implemented through the interaction between the terminals and servers. Alternatively, the present application can also be applied to cloud platform systems. Therefore, in addition to the need for data collection with the help of implanted sensors and roadside collection equipment, the specific execution end of the road damage detection method can be any computer device with a memory and a processor that can realize data processing functions. The present application does not limit the type of the execution end of the road damage detection method.

In one embodiment, as shown in FIG. 17 , this embodiment uses the road damage detection method applied to a terminal as an example. In this embodiment, the method includes the following steps S202 to S208 .

Step S202, obtaining an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period.

The acceleration data set is obtained by detecting the vibration acceleration signal generated by the vehicle load through an implanted sensor arranged inside the road to be detected.

As shown in FIG16 , a concrete road is used as an example for explanation. In the road section to be detected on the concrete road, multiple implanted sensors are pre-arranged inside the road to be detected. The implanted sensors detect the vibration acceleration signal generated by the vehicle load when the vehicle passes through the detection area of the road to be detected. Among them, the number and arrangement of the implanted sensors are not limited. For example, k implanted sensors can be arranged based on the size of the road panel. These k implanted sensors are laid at different positions of the same road panel inside the road, for example, at the corner of the panel, the longitudinal edge of the road panel along the driving direction, the center of the road panel, etc. Furthermore, due to the high speed of the vehicle, when the vehicle passes through the same road panel of the road to be detected, the k implanted sensors can obtain the vibration acceleration signal generated by the internal structure response of the road when the vehicle passes at the same time, and process and analyze the vibration acceleration signal to obtain the acceleration data at the time when the vehicle passes. Furthermore, within the preset detection cycle, an acceleration data set is constructed.

In implementation, within a preset detection period, the terminal obtains the acceleration data set detected by the implanted sensor. In addition, when the vehicle passes through the detection area of the road to be detected, the terminal can also collect additional attribute feature data related to the road to be detected through a variety of other types of sensors, so that the terminal can obtain an additional attribute feature data set containing multi-dimensional data. Based on the acceleration data set and the additional attribute feature data set, the road structure of the road to be detected can be detected.

Step S204: extract features from the acceleration data set using a recursive neural network to obtain acceleration features.

In implementation, a road damage detection model is pre-deployed in the terminal, and the road damage detection model at least includes a recursive neural network and a classification Specifically, after obtaining the acceleration data set of the road to be detected, the terminal extracts features of the acceleration data set according to the recursive neural network in the road damage detection model to obtain acceleration features.

Optionally, the recursive neural network may select a long short-term memory recursive neural network (Long Short-Term Memory, LSTM), or may select a multi-layer recursive neural network, a bidirectional recurrent neural network, etc. The embodiment of the present application does not limit the type of recursive neural network.

In the embodiment of the present application, the LSTM network is used as an example of the recursive neural network in the road damage detection model. The LSTM network can well extract the timing signal characteristics of the timing signal (the vibration acceleration signal contained in the acceleration data set) according to the context information, thereby obtaining the acceleration characteristics corresponding to the acceleration data set, so as to detect road damage more accurately.

Step S206, concatenating the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector.

In implementation, the terminal performs a feature splicing operation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector. Specifically, in the road damage detection model, a feature fusion layer may also be included, and the feature fusion layer may be a weighted function with a weight, through which the acceleration feature and the additional attribute feature are feature spliced to obtain a fused feature vector.

Step S208: input the fused feature vector into a preset classification prediction network to obtain a road damage result of the road to be detected.

During implementation, the terminal inputs the fused feature vector obtained after feature splicing into the classification prediction network of the road damage detection model, performs data analysis and processing on the fused feature vector through the classification prediction network, and determines the road damage result of the current road to be detected. The road damage result is used to reflect the specific road damage category.

Specifically, the classification prediction network included in the road damage detection model can be a multilayer perceptron (MLP), which is a supervised learning model based on a feedforward artificial neural network. The multilayer perceptron performs prediction processing on the fused feature vector to obtain the probability of each road damage classification category. Then, the terminal can use the road damage classification category with the highest probability as the road damage result based on the probability of each road damage classification result.

In the above road damage detection method, the acceleration data set and the additional attribute feature data set of the road to be detected within a preset detection period are obtained; the acceleration data set is obtained by collecting the vibration acceleration signal generated by the vehicle load through an implanted sensor set inside the road to be detected, and the acceleration data set is feature extracted through a recursive neural network to obtain acceleration features, and the acceleration features and the additional attribute features in the additional attribute feature data set are spliced to obtain a fused feature vector, and then the fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected. Using this method, the vibration acceleration signal is the vibration response generated by the road to be detected when the vehicle load passes through the road to be detected. Therefore, by processing and analyzing the acceleration data, the condition of the internal structure of the road can be detected, and the road damage result corresponding to the road to be detected can be obtained, thereby improving the accuracy of damage detection of the road to be detected.

In one embodiment, as shown in FIG. 18 , the implantable sensor may be a vibration acceleration sensor pre-set inside the road to be detected. Step S202 acquires an acceleration data set of the road to be detected within a preset detection period, including the following steps S2021-S2023.

Step S2021, based on a preset sampling frequency and signal length, collecting vibration acceleration signals collected by each vibration acceleration sensor when a vehicle passes through a detection area of a road to be detected within a preset detection period.

The vibration acceleration signal is generated by the road panel response generated by a vehicle passing through the road to be detected.

Specifically, based on a preset sampling frequency and signal length, the terminal collects vibration acceleration signals collected by each vibration acceleration sensor when the vehicle passes through a detection area of the road to be detected within a preset detection period.

Specifically, because the response frequency of the road structure generally does not exceed 100Hz, we configure the acquisition frequency of the vibration acceleration sensor to be below 200Hz to reduce the amount of collected data and improve data processing efficiency. In addition, for general highways, the time for a vehicle to pass through each road panel is less than 2 seconds, so the vibration acceleration sensor cuts the signal length to 10s, which can ensure that the road panel fully responds.

Step S2022: perform data preprocessing on each vibration acceleration signal, and construct an acceleration vector based on the vibration acceleration signals collected at the same time.

In implementation, the terminal performs data preprocessing on each vibration acceleration signal, and constructs an acceleration vector based on the vibration acceleration signals collected at the same time. For example, the internal road panel of the road to be detected is equipped with k sensors, and the acceleration data processed by different vibration acceleration sensors at the same time are combined into a k-dimensional acceleration vector.

Step S2023, obtaining an acceleration data set based on each acceleration vector.

In implementation, the terminal constructs an acceleration data set based on each acceleration vector generated within a preset detection period.

In this embodiment, the vibration acceleration signal generated by the road panel response when a vehicle passes by the road to be detected is collected through a preset sampling frequency and signal length. Thus, an acceleration data set is constructed based on the vibration acceleration signal. The processing and analysis of the acceleration data set can realize the detection of the internal structure of the road.

In one embodiment, as shown in FIG. 19 , in addition to collecting vibration acceleration signals through the vibration acceleration sensor, multiple types of sensors may be used to collect additional attribute feature data from multiple sources. Specifically, the additional attribute feature data set of the road to be detected within a preset detection period is obtained in step S202, including steps S402 and S404:

Step S402, acquiring attribute feature data of the road to be detected, attribute feature data of each vehicle passing through the road to be detected within a preset detection period, and internal monitoring environment data of the implanted sensor within the preset detection period.

The additional attribute feature data may include, but is not limited to, attribute feature data of the road to be detected, attribute feature data of the vehicle, and internal monitoring environment data of the implanted sensor.

During implementation, the terminal obtains various types of attribute characteristic data, for example, the attribute characteristic data of the road to be inspected: road structure dimension information (for example, roadbed thickness, base thickness, roadbed width, etc.), joint form, etc.; the attribute characteristic data of each vehicle passing through the road to be inspected within a preset detection period: axle weight, vehicle type, vehicle speed, etc., and the internal monitoring environment data of the implanted sensor within the preset detection period: temperature, humidity, etc.

Among them, the acquisition method of various types of attribute feature data is specifically as follows: the attribute feature data of the road to be detected can be directly queried and obtained in the road attribute feature record, that is, the terminal can query the road structure size information, joint form, etc. of the current road to be detected in the road attribute feature record based on the current location of the road to be detected. The attribute feature data of each vehicle passing on the road to be detected and the internal monitoring environment data of the implanted sensor can be collected based on other types of sensors or directly reported based on the implanted sensor. For example, an ultrasonic radar can be set in the collection device on the side of the road to be detected, and the speed of the vehicle passing through the road to be detected can be collected by the ultrasonic radar. A weighing instrument can also be buried in the detection area of the road to be detected, and the vehicle load can be collected by the weighing instrument, and a temperature sensor and a humidity sensor can be set inside the collection device and the implanted sensor, and the internal temperature and humidity of the collection device and the implanted sensor can be monitored by the temperature sensor and the humidity sensor.

Optionally, the attribute characteristic data of the vehicle are not limited to including: vehicle speed, axle weight, vehicle model, vehicle load, etc., the attribute characteristic data of the road to be inspected are not limited to including road structure dimension information, joint form, material, etc., and the internal monitoring environment data of the implanted sensor are not limited to including temperature, humidity, etc. The embodiment of the present application does not limit the data type of the additional attribute characteristic data.

Step S404, performing data cleaning and normalization processing on the attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor to obtain an additional attribute feature data set.

In implementation, the terminal cleans various types of attribute feature data, such as the attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor, eliminates missing values and abnormal values in each attribute feature data, and normalizes each attribute feature data after cleaning to obtain normalized attribute feature data. Thus, based on each attribute feature data after data cleaning and normalization, an additional attribute feature data set is constructed.

Among them, the embodiment of the present application adopts the mean variance normalization method to normalize each attribute feature data. The formula of the mean variance normalization method is as follows:

Where _Xi represents each data in each type of attribute feature data, μ and σ represent the mean and standard deviation of the attribute feature data of this type, respectively.

In this embodiment, an additional attribute feature data set is constructed by collecting multi-source data such as the attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor. The additional attribute feature data set can include relevant attribute features for detecting road damage, thereby combining with the acceleration features to achieve multi-dimensional road damage detection.

In one embodiment, the hidden layer of the recursive neural network in the road damage detection model includes multiple hidden layer units, and step S204 performs feature extraction on the acceleration data set according to the recursive neural network to obtain acceleration features, which specifically includes step S2041.

Step S2041, inputting the acceleration data set into a pre-trained recursive neural network, performing feature extraction on the acceleration vector in the acceleration data set through a plurality of hidden layer units included in a hidden layer of the recursive neural network, and obtaining acceleration features.

In practice, a recursive neural network is a neural network with time series synapses. In this application, an LSTM network is selected to process the acceleration data set. Specifically, the acceleration data set is input into the recursive neural network model in the form of a time series. For example, The n k-dimensional acceleration vectors x _t on the time series t are used as input data of the LSTM network. The LSTM network includes multiple hidden layer units. As shown in FIG20 , the LSTM network uses the output result h _n output by the last hidden layer unit as the feature extraction result of the acceleration data, i.e., the acceleration feature. The number of hidden layer units is adjusted according to the actual training effect during the model training process, and is not limited in the embodiment of the present application.

In this embodiment, a recursive neural network is used to extract features from the acceleration data set, and the timing signals contained in the acceleration data set are learned, thereby obtaining acceleration features to better analyze the timing changes contained in the acceleration data set, thereby more accurately detecting road damage.

In one embodiment, as shown in FIG. 21 , the method further includes steps S602 and S604 .

Step S602: Based on the road damage result, a target road management strategy is determined in the corresponding relationship between the road damage result and the road management strategy.

During implementation, a list containing correspondences between road damage results and road management strategies is pre-configured in the terminal. Within a preset detection period, the terminal determines the current road damage results of the road to be detected. Then, based on the road damage results, the terminal determines the target road management strategy in the correspondences between each road damage result and the road management strategy.

Step S604: Based on the target road management strategy, instruct to perform maintenance management on the road to be inspected.

In implementation, the terminal instructs maintenance management of the road to be detected based on the target road management strategy. Optionally, the target road management strategy includes generating alarm information and providing road maintenance management advice information. For example, if the road damage result is road debonding damage, the target road management strategy includes: generating alarm information characterizing road debonding damage, and at the same time, providing road maintenance management advice information for maintaining road debonding damage (for example, repairing and filling, removing damaged road surface, maintaining roadbed, etc.). The alarm information characterizing road debonding damage is used to remind the user that there is road debonding damage in the target detection section of the current road to be detected, and the road maintenance management advice information is used to guide the user to complete the corresponding road maintenance.

In this embodiment, by pre-configuring the correspondence between road damage results and road management strategies, after determining the road damage results of the road to be detected, the target road management strategy for the current road to be detected can be automatically recommended, and then, based on the instructions of the target road management strategy, timely maintenance of the current road to be detected can be achieved.

In one embodiment, the road damage detection model includes a recursive neural network layer and a classification prediction network layer. Before the road damage detection model is applied, it is necessary to perform model training in advance to ensure the accuracy of the model output results. As shown in FIG22, the method also includes:

Step S702, obtaining training data samples.

The training data samples include training acceleration data sets, additional attribute feature data sets, and road damage category labels.

In implementation, the terminal obtains a training data sample. The additional attribute feature data set in the training data sample may include, but is not limited to, a vehicle attribute feature data set, a road attribute feature data set, and internal monitoring environment data of an implanted sensor. When constructing the training data sample, the training data is divided into a training set, a validation set, and a test set and annotated, and the specific division ratio may be 0.9:0.09:0.01.

Optionally, the various types of training data contained in the training data sample are similar to the data acquisition process in step 202 in the above embodiment. For example, training acceleration data and internal monitoring environment data of the implanted sensor can be collected by an implanted sensor, vehicle attribute feature data can be collected by a collection device, and road attribute feature data can be obtained by query, etc. Then, the terminal can construct a training data sample based on the acquired training acceleration data, vehicle attribute feature data, road attribute feature data, and internal environment monitoring data of the implanted sensor, etc. The embodiments of the present application will not be described in detail here.

Optionally, after acquiring each type of training data, each type of training data may be cleaned and normalized, and the processing process is similar to step S404 in the above embodiment, and the present embodiment will not be described in detail here. Thus, a training data sample is constructed based on the training data after data cleaning and normalization.

Step S704: input the training acceleration data set into the recursive neural network, perform feature extraction on the training acceleration data set, and obtain acceleration features.

In the implementation, the terminal inputs the training acceleration data set in the training data sample into the recursive neural network, and extracts the features of the training acceleration data set through each hidden layer unit of the hidden layer in the recursive neural network to obtain the acceleration features. Among them, the number of hidden layer units is adjusted according to the actual training effect of the road damage detection model.

Specifically, when the recursive neural network processes the training acceleration data, it adaptively learns the time dependency of the sequence data to model and predict the acceleration data sequence. During the model training process, supervised learning can be used to use known annotated acceleration data. The model is trained based on the acceleration data to extract the characteristics of the acceleration data, such as the change trend, peak value, and duration of the acceleration value.

Step S706: Concatenate the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector.

In implementation, the terminal performs feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector.

Specifically, the road damage detection model may also include a feature fusion layer, which is used to fuse acceleration features and additional attribute features. The feature fusion layer may be a weighted function of a preset weight coefficient, through which feature splicing of multi-source features is achieved to obtain a fused feature vector.

Step S708: input the fused feature vector and the road damage category label into the classification prediction network, and perform data processing on the fused feature vector through the classification prediction network to obtain a classification prediction result.

In implementation, during the training of the classification prediction network, the terminal inputs the fused feature vector and the road damage category label into the classification prediction network, performs data processing on the fused feature vector through the classification prediction network, and outputs the classification prediction result representing the road damage category, which can be the probability of the corresponding road damage category.

Specifically, the classification prediction network can be a multilayer perceptron, which is a feedforward neural network with multiple hidden layers. It can be used for supervised learning tasks. In classification tasks, MLP can learn to map input training data samples to predefined category labels through training. In the training process of this application, except for the activation function of the last classification layer of the multilayer perceptron, the activation function of the remaining layers uses the Relu activation function, and the activation function is used to calculate the probability that each input training data belongs to a different road damage category.

Optionally, the Adam method may be selected to train the classification prediction network. The Adam method may be used to adaptively adjust the learning rate in the gradient descent process to avoid local convergence of the road damage detection model during the model training process.

Step S710, determining the loss result of the road damage detection model according to the classification prediction result, the road damage category label and the preset loss function, until the loss result meets the preset model loss condition, and determining that the road damage detection model training is completed.

In implementation, the terminal determines the loss result of the road damage detection model based on the classification prediction result, the road damage category label and the preset loss function. Furthermore, the terminal determines whether the road damage detection model is trained based on the loss result and the preset model loss condition.

Specifically, the final loss function of the road damage detection model may be a cross entropy loss function, and the calculation formula is as follows:

Where M is the total number of classifications, c is a different classification category (road damage category), i represents a different sample, _pic is the predicted probability that training data i belongs to classification category c, and _yic itself has only two values, 0 and 1. When the training data i is actually labeled as c, it is 1, and otherwise it is 0.

The preset model loss condition may be less than or equal to a preset model loss threshold. If the model loss result does not meet the preset model loss condition, the above steps S702 to S710 are repeatedly performed until the loss result meets the preset model loss condition, and it is determined that the road damage detection model training is completed.

In this embodiment, the road damage detection model is trained using training data samples containing multidimensional training data to obtain a trained road damage detection model. Through the trained road damage detection model, road damage detection based on multidimensional detection data can be implemented.

In one embodiment, as shown in FIG. 23 , an example of a road damage detection method applied to a concrete pavement is given, specifically including steps 801 to 805 .

Step 801, obtaining an additional attribute feature data set of a road to be detected within a preset detection period.

Step 802: Acquire an acceleration data set of the road to be detected within a preset detection period.

Step 803: extract features from the acceleration data set using a recursive neural network to obtain acceleration features.

Step 804 , concatenating the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector.

Step 805: input the fused feature vector into a preset classification prediction network to obtain the road damage result of the concrete pavement.

As shown in FIG. 23 , the execution order of step 801 and step 802 may be synchronous.

It should be understood that, although the steps in the flowcharts of the embodiments described above are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least one of the flowcharts of the embodiments described above is shown in the order indicated by the arrows. A part of the steps may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequentially, but can be executed in rotation or alternation with other steps or at least part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application also provides a road damage detection device for implementing the road damage detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more road damage detection device embodiments provided below can refer to the limitations of the road damage detection method above, and will not be repeated here.

In one embodiment, as shown in FIG. 24 , a road damage detection device 900 is provided, including: an acquisition module, a feature extraction module, a splicing module and a detection and discrimination module.

The acquisition module 901 is used to acquire the acceleration data set and the additional attribute feature data set of the road to be detected within a preset detection period; the acceleration data set is obtained by detecting the vibration acceleration signal generated by the vehicle load through an implanted sensor set inside the road to be detected.

The feature extraction module 902 is used to extract features from the acceleration data set according to a recursive neural network to obtain acceleration features.

The splicing module 903 is used to perform feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector.

The detection and discrimination module 904 is used to input the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected.

In one embodiment, the implantable sensor is a vibration acceleration sensor pre-installed inside the road to be detected, and the acquisition module 901 is specifically used to collect the vibration acceleration signals collected by each vibration acceleration sensor when the vehicle passes through the detection area of the road to be detected within a preset detection period based on the preset sampling frequency and signal length. The vibration acceleration signal is generated by the road panel response generated by the vehicle passing through the road to be detected.

The acquisition module 901 is used to perform data preprocessing on each vibration acceleration signal and construct an acceleration vector based on the vibration acceleration signals collected at the same time.

The acquisition module 901 is used to obtain an acceleration data set based on each acceleration vector.

In one of the embodiments, the acquisition module 901 is specifically used to obtain the attribute feature data of the road to be detected, the attribute feature data of each vehicle passing through the road to be detected within a preset detection period, and the internal monitoring environment data of the implanted sensor within the preset detection period; the attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor are cleaned and normalized to obtain an additional attribute feature data set.

In one of the embodiments, the hidden layer of the recursive neural network includes multiple hidden layer units, and the feature extraction module 902 is specifically used to input the acceleration data set into the pre-trained recursive neural network, and extract features of the acceleration vector in the acceleration data set through the multiple hidden layer units included in the hidden layer of the recursive neural network to obtain acceleration features.

In one embodiment, the apparatus 900 further includes: a determination module and an indication module.

The determination module is used to determine the target road management strategy based on the road damage result and in the corresponding relationship between the road damage result and the road management strategy.

The instruction module is used to instruct maintenance management of the road to be inspected based on the target road management strategy.

In one embodiment, the device 900 further includes: a training acquisition module, a feature extraction module, a splicing module, a detection and discrimination module and a training and discrimination module.

The training acquisition module is used to obtain training data samples. The training data samples include training acceleration data sets, additional attribute feature data sets, and road damage category labels.

The feature extraction module is used to input the training acceleration data set into the recursive neural network, perform feature extraction on the training acceleration data set, and obtain acceleration features.

The splicing module is used to perform feature splicing on the acceleration feature and the additional attribute features in the additional attribute feature data set to obtain a fused feature vector.

The detection and discrimination module is used to input the fused feature vector and the road damage category label into the classification prediction network, and perform data processing on the fused feature vector through the classification prediction network to obtain the classification prediction result.

The training discriminant module is used to determine the road damage detection model based on the classification prediction results, road damage category labels and preset loss functions. The loss results of the model are calculated until the loss results meet the preset model loss conditions, and the road damage detection model training is determined to be completed.

Each module in the above-mentioned road damage detection device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG25. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a road damage detection method is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, a trackball or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

Those skilled in the art will understand that the structure shown in FIG. 25 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

Acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by collecting a vibration acceleration signal generated by a vehicle load through an implanted sensor arranged inside the road to be detected;

Extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features;

Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector;

The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

Based on a preset sampling frequency and signal length, the vibration acceleration signals collected by each vibration acceleration sensor when a vehicle passes through a detection area of the road to be detected within a preset detection period are collected; the vibration acceleration signals are generated by a road panel response generated by a vehicle passing through the road to be detected;

Performing data preprocessing on each of the vibration acceleration signals, and constructing an acceleration vector based on the vibration acceleration signals collected at the same time;

Based on each of the acceleration vectors, an acceleration data set is obtained.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

Acquire attribute characteristic data of the road to be detected, attribute characteristic data of each vehicle passing through the road to be detected within a preset detection period, and internal monitoring environment data of the implanted sensor within the preset detection period;

The attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor are cleaned and normalized to obtain an additional attribute feature data set.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

The acceleration data set is input into a pre-trained recursive neural network, and the acceleration vector in the acceleration data set is feature extracted through a plurality of hidden layer units included in a hidden layer of the recursive neural network to obtain acceleration features.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

Based on the road damage result, determining a target road management strategy in a corresponding relationship between the road damage result and the road management strategy;

Based on the target road management strategy, an instruction is given to perform maintenance management on the road to be inspected.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

Acquire training data samples; the training data samples include a training acceleration data set, an additional attribute feature data set, and a road damage category label;

Inputting the training acceleration data set into a recursive neural network, performing feature extraction on the training acceleration data set to obtain acceleration features;

Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector;

Inputting the fused feature vector and the road damage category label into a classification prediction network, performing data processing on the fused feature vector through the classification prediction network to obtain a classification prediction result;

According to the classification prediction result, the road damage category label and the preset loss function, the loss result of the road damage detection model is determined until the loss result meets the preset model loss condition, and it is determined that the road damage detection model training is completed.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to this.

The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims (28)

A road damage detection method, characterized by comprising: Acquire sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road; the sample damage range includes damage ranges of different damage types; Based on the image information of the road and the current damage range of the road, a target damage area of the road is identified through a damage location identification strategy, and based on the damage sensor map of the road, a damage identification network is used to identify the damage type of the road; Acquire damage sensor data of the target damage area, and calculate the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type; The target damage area, the damage level of the target damage area, and the damage type of the target damage area are used as target damage information of the road. The method according to claim 1 is characterized in that the sensor data information of the road includes sensor information of multiple sensors of the road and position information of each of the sensors; and the determining of the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road comprises: Based on the sensing information of each sensor and the position information of each sensor, three-dimensional sensing map data of the road is established, and damage map data that meets the road damage condition is screened in the three-dimensional sensing map data of the road, and the range of all damage map data included in the three-dimensional sensing map data is used as the current damage range of the road; the damage map data includes the position information of the corresponding pixel points of the damage map data and the sensing information of the corresponding pixel points of the damage map data; Based on the position information of each image data within the current damage range and the sensor information of each image data, a damage sensor map corresponding to each image data within the current damage range is established. The method according to claim 1 or 2, characterized in that the identifying the target damaged area of the road through a damage location identification strategy based on the image information of the road and the current damage range of the road comprises: Establishing three-dimensional image data of the image information of the road, and identifying the damaged location area of the image information of the road through a damaged image recognition network; Establishing a correspondence between the three-dimensional sensing image data and the three-dimensional image data, and identifying a sub-damage range corresponding to a damage position area within the current damage range based on the correspondence; Clustering the damage map data within the current damage range according to the distance between each damage map data and the sub-damage range to obtain multiple damage map data groups, and calculating the average distance between each damage map data in each damage map data group and the sub-damage range; Filter each target damage map data in the damage map data group corresponding to the average distance below the preset distance threshold, and use the sub-damage range and the range included in the target damage map data corresponding to the sub-damage range as the sub-damage area, and use all the sub-damage areas as the target damage areas of the road. The method according to any one of claims 1 to 3 is characterized in that the identifying the damage type of the road through a damage identification network based on the damage sensor map of the road comprises: Extracting damage feature data of each sub-damage area of the damage sensing map, and inputting each damage feature data into a damage identification network to obtain a sub-damage type corresponding to each damage feature data; The sub-damage type corresponding to each damage feature data is used as the damage type of the road. The method according to any one of claims 1 to 4, characterized in that the step of calculating the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type comprises: For each sub-damage region, based on the sample damage ranges of the degrees corresponding to the sub-damage types of the sub-damage region, the sample damage range to which the damage sensing data of the sub-damage region belongs is identified, and the degree of the sub-damage type corresponding to the sub-damage region is obtained; Based on the sub-damage type corresponding to the sub-damage area and the degree of the sub-damage type corresponding to the sub-damage area, the damage A level division strategy is used to determine the damage level corresponding to the sub-damage area. The method according to any one of claims 1 to 5 is characterized in that after taking the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road, the method further comprises: For each sub-damage region, determining a damage repair strategy for the sub-damage region based on the sub-damage type of the sub-damage region and the degree of the sub-damage type of the sub-damage region; According to the damage level of each sub-damage area from high to low, the maintenance order of each sub-damage area is arranged to obtain the repair sequence of each sub-damage area; and the damage repair strategy of each sub-damage area is filled into the repair sequence to obtain the repair task information of the target damage area, and the early warning information including the repair task information and the damage information of the target damage area is sent to the display module. The method according to any one of claims 1 to 6, further comprising: In response to the user's sensor acquisition system update operation, the acquisition system update information of each sensor is obtained, and the acquisition system update data information is sent to the acquisition box; the acquisition system update data information is used to update the current acquisition system data information of each sensor to the acquisition system update data information. The method according to any one of claims 1 to 7, further comprising: In response to the user's sensor acquisition task upload operation, an acquisition instruction for each sensor is generated and sent to the acquisition box; the acquisition instruction includes the acquisition task of each sensor, and the acquisition instruction is used to instruct each sensor to execute the acquisition task in the acquisition instruction. A road damage detection system includes a cloud platform and a collection box: The collection box is communicatively connected with the cloud platform; The collection box is used to collect sensor data information of the road and image information of the road; The cloud platform is used to obtain multiple different sample damage ranges, and based on the sensor data information of the road, determine the current damage range of the road and the damage sensor map of the road; the sample damage range includes damage ranges of different damage types; based on the image information of the road and the current damage range of the road, identify the target damage area of the road through a damage location recognition strategy, and based on the damage sensor map of the road, identify the damage type of the road through a damage identification network; obtain the damage sensor data of the target damage area, and based on the damage type of the road and the sample damage ranges of each degree of the damage type, calculate the damage level corresponding to the damage sensor data of the target damage area; use the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road. A road damage detection device, comprising: an acquisition module, used to acquire sensor data information of the road sent by the collection box, image information of the road sent by the collection box, and a plurality of different sample damage ranges, and determine the current damage range of the road and the damage sensor map of the road based on the sensor data information of the road; the sample damage range includes damage ranges of different damage types; an identification module, configured to identify a target damaged area of the road through a damage location identification strategy based on the image information of the road and the current damage range of the road, and to identify a damage type of the road through a damage identification network based on a damage sensor map of the road; a re-acquisition module, configured to acquire the damage sensor data of the target damage area, and calculate the damage level corresponding to the damage sensor data of the target damage area based on the damage type of the road and the sample damage range of each degree of the damage type; The determination module is used to use the target damage area, the damage level of the target damage area, and the damage type of the target damage area as the target damage information of the road. A method for determining a road sensor deployment position, characterized in that the method comprises: Determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road; Determine a reference layout point from the layout points corresponding to each of the road conditions, construct a reference layout point group based on the reference layout point, and The layout points corresponding to the road condition that do not belong to the reference layout point group are used as candidate layout points; For each candidate layout point, determine the target road condition corresponding to the candidate layout point, and determine the first condition data corresponding to each reference layout point under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition, and determine the correlation coefficient between the candidate layout point and the reference layout point group according to each of the first condition data and the second condition data, wherein the condition data is used to characterize the stress condition of the road at the layout point; Selecting a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points; The target layout point is added to the reference layout point group to obtain a final target layout point group. The method according to claim 11, characterized in that the step of determining the layout points corresponding to each road condition from a plurality of preset layout points of the target road comprises: Constructing road models corresponding to target roads under various road conditions respectively; For each of the road conditions, the vehicle driving data corresponding to the target road under the road condition is obtained, and based on the vehicle driving data and the road model corresponding to the target road under the road condition, the third condition data corresponding to multiple preset layout points of the target road under the road condition are respectively determined, and based on each of the third condition data, the layout point corresponding to the road condition is determined from each of the preset layout points. The method according to claim 12, characterized in that the step of determining the layout point corresponding to the road working condition from the preset layout points according to the third working condition data comprises: Sorting the preset layout points from large to small according to the third working condition data to obtain a preset layout point queue; Traverse the preset layout point queue, and when the third operating condition data corresponding to each of the preset layout points arranged before the current traversal position meets the preset strategy, stop traversing the preset layout point queue, and use each of the preset layout points arranged before the current traversal position as the layout points corresponding to the road operating condition. The method according to any one of claims 11 to 13, characterized in that the step of determining the reference layout point from the layout points corresponding to each of the road conditions comprises: According to the layout points corresponding to the road conditions, respectively determine the number of layout points corresponding to the road conditions; The layout points corresponding to the largest number of layout points among the numbers of layout points are taken as the reference layout point group. The method according to any one of claims 11 to 14, characterized in that the step of determining the correlation coefficient between the candidate layout points and the reference layout point group according to each of the first operating condition data and the second operating condition data comprises: For any of the reference layout points, determining a correlation coefficient between the candidate layout point and the reference layout point according to the first operating condition data and the second operating condition data corresponding to the reference layout point; The correlation coefficient between the candidate layout point and the reference layout point group is determined according to the correlation coefficient corresponding to each of the reference layout points. The method according to any one of claims 11 to 15, characterized in that the step of selecting a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points comprises: For any of the candidate layout points, when the correlation coefficient corresponding to the candidate layout point is less than a correlation coefficient threshold, the candidate layout point is used as a target layout point. The method according to any one of claims 11 to 16, characterized in that the method further comprises: According to the preset layout point arrangement strategy, a plurality of preset layout points are determined on the target road, and the plurality of preset layout points are evenly distributed on the target road. A device for determining a road sensor deployment position, characterized in that the device comprises: A first determination module is used to determine the layout points corresponding to each road condition from a plurality of preset layout points of the target road; The second determination module is used to determine a reference layout point from the layout points corresponding to each of the road conditions, construct a reference layout point group based on the reference layout point, and use the layout points that do not belong to the reference layout point group among the layout points corresponding to each of the road conditions as candidate layout points. point; A third determination module is used to determine, for any of the candidate layout points, the target road condition corresponding to the candidate layout point, and determine the first condition data corresponding to each of the reference layout points under the target road condition, and the second condition data corresponding to the candidate layout point under the target road condition, and determine the correlation coefficient between the candidate layout point and the reference layout point group according to each of the first condition data and the second condition data, wherein the condition data is used to characterize the stress condition of the road at the layout point; A selection module, configured to select a target layout point from each of the candidate layout points according to the correlation coefficient corresponding to each of the candidate layout points; An adding module is used to add the target layout point to the reference layout point group to obtain a final target layout point group. A road damage detection method, characterized in that the method comprises: Acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be detected; Extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features; Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector; The fused feature vector is input into a preset classification prediction network to obtain the road damage result of the road to be detected. The method according to claim 19, characterized in that the implantable sensor is a vibration acceleration sensor pre-installed inside the road to be detected, and the acquisition of the acceleration data set of the road to be detected within a preset detection period comprises: Based on a preset sampling frequency and signal length, the vibration acceleration signals collected by each vibration acceleration sensor when a vehicle passes through a detection area of the road to be detected within a preset detection period are collected; the vibration acceleration signals are generated by a road panel response generated by a vehicle passing through the road to be detected; Performing data preprocessing on each of the vibration acceleration signals, and constructing an acceleration vector based on the vibration acceleration signals collected at the same time; Based on each of the acceleration vectors, an acceleration data set is obtained. The method according to claim 19 or 20 is characterized in that the step of obtaining an additional attribute feature data set of the road to be detected within a preset detection period comprises: Acquire attribute characteristic data of the road to be detected, attribute characteristic data of each vehicle passing through the road to be detected within a preset detection period, and internal monitoring environment data of the implanted sensor within the preset detection period; The attribute feature data of the road to be detected, the attribute feature data of each vehicle, and the internal monitoring environment data of the implanted sensor are cleaned and normalized to obtain an additional attribute feature data set. The method according to any one of claims 19 to 21, characterized in that the hidden layer of the recursive neural network includes a plurality of hidden layer units, and the extracting features of the acceleration data set according to the recursive neural network to obtain acceleration features comprises: The acceleration data set is input into a pre-trained recursive neural network, and the acceleration vector in the acceleration data set is feature extracted through a plurality of hidden layer units included in a hidden layer of the recursive neural network to obtain acceleration features. The method according to any one of claims 19 to 22 is characterized in that, after inputting the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected, the method further comprises: Based on the road damage result, determining a target road management strategy in a corresponding relationship between the road damage result and the road management strategy; Based on the target road management strategy, an instruction is given to perform maintenance management on the road to be inspected. The method according to any one of claims 19 to 23, characterized in that the method further comprises: Acquire training data samples; the training data samples include a training acceleration data set, an additional attribute feature data set, and a road damage category label; Inputting the training acceleration data set into a recursive neural network, performing feature extraction on the training acceleration data set to obtain acceleration features; Performing feature concatenation on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector; Inputting the fused feature vector and the road damage category label into a classification prediction network, performing data processing on the fused feature vector through the classification prediction network to obtain a classification prediction result; According to the classification prediction result, the road damage category label and the preset loss function, the loss result of the road damage detection model is determined until the loss result meets the preset model loss condition, and it is determined that the road damage detection model training is completed. A road damage detection device, characterized in that the device comprises: An acquisition module is used to acquire an acceleration data set and an additional attribute feature data set of a road to be detected within a preset detection period; the acceleration data set is obtained by detecting a vibration acceleration signal generated by a vehicle load using an implanted sensor disposed inside the road to be detected; A feature extraction module, used for extracting features from the acceleration data set according to a recursive neural network to obtain acceleration features; A splicing module, used for performing feature splicing on the acceleration feature and the additional attribute feature in the additional attribute feature data set to obtain a fused feature vector; The detection and discrimination module is used to input the fused feature vector into a preset classification prediction network to obtain the road damage result of the road to be detected. A computer device comprises a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the method described in any one of claims 1 to 8, 11 to 17, and 19 to 24 when executing the computer program. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the method described in any one of claims 1-8, 11-17, 19 to 24 are implemented. A computer program product, comprising executable instructions, characterized in that when the executable instructions are executed by a processor, the steps of the method described in any one of claims 1-8, 11-17, and 19 to 24 are implemented.