HK1247374B - Image-based vehicle loss assessment method and apparatus, and electronic device - Google Patents

Description

Image-based vehicle damage assessment method, device, and electronic equipment

Technical Field

The present application belongs to the field of computer image data processing technology, and in particular relates to an image-based vehicle damage assessment method, device and electronic equipment.

Background Art

After a traffic accident, it's often necessary to wait for the insurance company's claims adjuster to arrive and process the claim, taking photos and other procedures to obtain the required evidence. With the recent increase in motor vehicle ownership, the number of traffic accidents has remained high annually. However, vehicle claims assessment often relies on on-site manual processing by professional insurance personnel, which is costly, time-consuming, and inefficient.

Currently, some approaches in the industry use traffic accident scene images for automatic analysis to categorize pre-set vehicle damage locations. For example, application publication number "CN105678622A," entitled "Auto Insurance Claim Photo Analysis Method and System," discloses an algorithm that uses a conventional convolutional neural network (CNN) to analyze claim photos uploaded by mobile terminals, identify damage location categories, and generate reminder information based on the analysis results. However, this approach simply identifies vehicle damage location categories, such as front, side, and rear, without identifying specific damage types. The identified damage location reminder information is primarily used by insurance company staff for manual comparison with manual damage assessment results, serving as reference information to assist insurance company staff in damage assessment calculations. Furthermore, this algorithm only utilizes a general CNN object recognition algorithm, leaving the final vehicle damage assessment result to rely on manual verification, which is labor-intensive and time-consuming. Furthermore, different insurance companies have inconsistent standards for damage assessment, and coupled with human subjective factors, vehicle damage assessment results vary widely, resulting in low reliability.

Summary of the Invention

The purpose of this application is to provide an image-based vehicle damage assessment method, device and electronic equipment that can quickly, accurately and reliably detect specific information such as the location and degree of damage to vehicle components, and the damage assessment results are more accurate and reliable, providing users with repair plan information, quickly and efficiently performing vehicle damage assessment processing, and greatly improving the user service experience.

The present application provides an image-based vehicle damage assessment method, device, and electronic device that are implemented as follows:

A vehicle damage assessment method based on an image, the method comprising:

Obtaining component images corresponding to vehicle components;

Detecting the component image using a constructed component damage recognition model to determine the damage location and damage type of the vehicle component;

A repair plan result for the vehicle component is generated based on information including a damage location and a damage type of the vehicle component.

A vehicle damage assessment device based on an image, comprising:

An image acquisition module, used to acquire component images corresponding to vehicle components;

a detection module, configured to store the constructed component damage recognition model, and detect the component image using the component damage recognition model to determine the damage location and damage type of the vehicle component;

A damage assessment processing module is used to generate a repair plan result for the vehicle component based on information including the damage location and damage type of the vehicle component.

An image-based vehicle damage assessment device includes a processor and a memory for storing processor-executable instructions, wherein when the processor executes the instructions, the following are achieved:

Obtaining component images corresponding to vehicle components;

Detecting the component image using a constructed component damage recognition model to determine the damage location and damage type of the vehicle component;

A repair plan result for the vehicle component is generated based on information including a damage location and a damage type of the vehicle component.

A computer-readable storage medium having computer instructions stored thereon, which, when executed, implement the following steps:

Obtaining component images corresponding to vehicle components;

Detecting the component image using a constructed component damage recognition model to determine the damage location and damage type of the vehicle component;

A repair plan result for the vehicle component is generated based on information including a damage location and a damage type of the vehicle component.

An electronic device includes a processor and a memory for storing processor-executable instructions, wherein when the processor executes the instructions, the following is achieved:

Obtaining component images corresponding to vehicle components;

Detecting the component image using a constructed component damage recognition model to determine the damage location and damage type of the vehicle component;

A repair plan result for the vehicle component is generated based on information including a damage location and a damage type of the vehicle component.

The present application provides an image-based vehicle damage assessment method, device, and electronic device that can construct a training model that can more comprehensively identify information such as damage locations and types, and produce more reliable results. This model can detect the damaged locations of vehicle components and the types of damage corresponding to the damaged locations, thereby obtaining the information required for accurate and comprehensive damage assessment of vehicle components. Furthermore, the implementation scheme of the present application utilizes information including the identified damage locations and types of damage to generate repair plans for vehicle components, providing insurance personnel and vehicle owners with more accurate, reliable, and practical damage assessment information. The implementation scheme of the present application can automatically generate repair plans based on specific vehicle damage information, a vehicle component price database, and repair processing methods, which can meet the needs of insurance companies or vehicle owners for fast, comprehensive, accurate, and reliable vehicle damage assessment processing, improve the accuracy and reliability of vehicle damage assessment processing results, and enhance user service experience.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a schematic diagram of a method flow of an embodiment of an image-based vehicle damage assessment method described in the present application;

FIG2 is a schematic diagram of a method flow of another embodiment of an image-based vehicle damage assessment method described in the present application;

FIG3 is a schematic diagram of the network structure of a neural network for recognizing component images provided in one embodiment of the present application;

FIG4 is a schematic diagram of the module structure of an embodiment of an image-based vehicle damage assessment device provided by the present application;

FIG5 is a schematic structural diagram of an embodiment of an electronic device provided by the present application;

FIG6 is a schematic diagram of a processing scenario for vehicle damage assessment using the implementation scheme of the present application.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

FIG1 is a method flow chart of an embodiment of an image-based vehicle damage assessment method described in the present application. Although the present application provides method operation steps or device structures as shown in the following embodiments or drawings, the method or device may include more or fewer operation steps or module units after partial merger based on routine or no creative labor. In the steps or structures where there is no necessary causal relationship logically, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure shown in the embodiments or drawings of the present application. When the method or module structure is applied to an actual device, server or terminal product, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or drawings (for example, a parallel processor or multi-threaded processing environment, or even a distributed processing, server cluster implementation environment).

In the existing actual traffic accident handling, such as scratch accidents, it is often necessary to wait for the insurance company's claims adjuster to arrive at the scene to take pictures before leaving the scene, which often leads to traffic jams, wastes a lot of time, and takes a long time to obtain information on the damage assessment results. However, when a traffic accident occurs using the implementation scheme of this application, the owner of the vehicle involved often wants to know the loss or claim status of his or the other party's vehicle. At this time, you can take pictures of the traffic accident scene yourself. On the one hand, it can be used as on-site evidence. On the other hand, you can use the pictures to estimate the loss and claim status of the vehicle through the terminal APP. It meets the needs of the car owners involved in the accident to obtain vehicle damage assessment quickly, comprehensively, accurately and reliably.

For clarity, the following embodiment will be described using a specific application scenario in which a vehicle owner requests a vehicle damage assessment service using a mobile terminal app (application). In this embodiment, the vehicle owner can use a mobile terminal (e.g., a cell phone) to take a photo of the damaged area of their vehicle at the scene of a traffic accident and then upload the captured photo (image) through the terminal app. When uploading the photo, the vehicle owner can select or enter information about the vehicle component being photographed, such as the front bumper, left front door, or rear taillight. After receiving the uploaded image of a single vehicle component, the cloud server uses a machine learning algorithm to identify the location and type of damage. A rule engine can then be designed to access different price libraries based on repair policy information such as vehicle model, location, and repair shop, ultimately generating a repair plan for at least one vehicle component. This repair plan is then returned to the vehicle owner, who can then quickly obtain the vehicle damage assessment results. Of course, if the user is an insurance company employee, the repair plan results can be returned to the insurance company or displayed directly. However, those skilled in the art will appreciate that the essence of this solution can be applied to other implementation scenarios of vehicle damage assessment, such as automatic vehicle damage assessment by insurance companies or repair shops, or self-service vehicle damage assessment services provided by 4S stores or other servers.

A specific embodiment is shown in FIG1 . In one embodiment of an image-based vehicle damage assessment method provided by the present application, the method may include:

S1: Obtain component images corresponding to vehicle components.

The server can obtain component images of single components of the vehicle from the client or a third-party server (such as an insurance company's server). In this embodiment, these component images can be considered to be images of the vehicle components. The component images described in this embodiment can be a general term for various graphics and images, usually referring to pictures with visual effects, which can generally include pictures on paper media, negatives or photos, televisions, projectors or computer screens, etc. In the application scenario of this embodiment, when the owner uploads the component image, he can specify the component information to which the component image belongs, such as the component image of the front bumper or the left front door or the rear taillight. Of course, the client or server side can mark the corresponding vehicle component marking information for the obtained component image. The vehicle components usually refer to components on the vehicle, such as the front bumper, the left front door, the rear taillight, etc.

In an optional embodiment, it is also possible to determine whether the image quality of the component image meets the set processing requirements. If the image quality is poor, such as the photo is blurred and cannot be recognized, the component image can be discarded and feedback is given to the mobile terminal APP to prompt the user to pay attention to focus, lighting and other factors that affect clarity when taking images.

S2: Detecting the component image using the constructed component damage recognition model to determine the damage location and damage type of the vehicle component.

After the cloud server obtains the component image, it can use the pre-built component damage recognition model to detect the component image and identify the damaged part and damage type of the vehicle component in the image. The damaged part described in this embodiment generally refers to a damaged part on the vehicle. A damaged vehicle component may contain multiple damaged parts, and each damaged part corresponds to a damage type (such as severe scratches, mild deformation, etc.). This embodiment can detect the location area of the damaged part in the image to be processed, and can detect the location area to identify the damage type. The damage types described in this embodiment may include mild scratches, severe scratches, mild deformation, moderate deformation, severe deformation, breakage, and need for disassembly inspection.

In this embodiment, a designed machine learning algorithm can be used in advance to construct a component damage recognition model for identifying the damaged parts and damage types of vehicle parts in an image. After sample training, the component damage recognition model can identify the damaged parts and damage types of vehicle parts in the component image. In this embodiment, the component damage recognition model can be constructed and generated using some network models or variant network models of deep neural networks after sample image training. In another embodiment of the method provided in the present application, the component damage recognition model can be constructed based on a convolutional neural network (CNN) and a region proposal network (RPN), combined with a pooling layer, a fully connected layer, and the like. Therefore, in another embodiment of the method described in the present application, the component damage recognition model includes:

S201: A network model based on convolutional layers and region proposal layers is trained with sample data to build a generated deep neural network.

Convolutional neural networks generally refer to neural networks with convolutional layers (CNN) as the main structure and combined with other components such as activation layers, which are mainly used for image recognition. The deep neural network described in this embodiment may include convolutional layers and other important layers (such as pooling layers, data normalization layers, activation layers, etc.), and be jointly constructed and generated in combination with a region proposal network (RPN). Convolutional neural networks usually combine two-dimensional discrete convolution operations in image processing with artificial neural networks. This convolution operation can be used to automatically extract features. The region proposal network (RPN) can take features extracted from an image (of any size) as input (two-dimensional features extracted using a convolutional neural network) and output a set of rectangular target proposal boxes, each box having an object score. To avoid confusion, in this embodiment, the convolutional neural network (CNN) used can be referred to as a convolutional layer (CNN), and the region proposal network (RPN) can be referred to as a region proposal layer (RPN). In other embodiments of the present application, the component damage recognition model may also include a deep convolutional neural network constructed and generated after training with sample data based on an improved variant network model of the convolutional neural network or an improved variant network model of the region proposal network.

The models and algorithms used in the above embodiments can be selected from the same type of models or algorithms. Specifically, for example, in the component recognition model, various models and variants based on convolutional neural networks and region proposal networks can be used, such as Faster R-CNN, YOLO, Mask-FCN, etc. The convolutional neural network (CNN) can be any CNN model, such as ResNet, Inception, VGG, etc. and their variants. Generally, the convolutional network (CNN) portion of the neural network can use a mature network structure that has achieved good results in object recognition, such as Inception, ResNet, etc. For example, the ResNet network takes an image as input and outputs multiple image regions containing damage parts, and the corresponding damage classification (damage classification is used to determine the damage type) and confidence (confidence is a parameter indicating the degree of authenticity of the damage type). Faster R-CNN, YOLO, Mask-FCN, etc. are all deep neural networks containing convolutional layers that can be used in this embodiment. The deep neural network used in this embodiment, combined with the region proposal layer and CNN layer, can detect the damaged part, damage type, and the location area of the damaged part in the component image.

It should be noted that in one embodiment of the present application, a separate algorithm server can be used to detect the component image and identify the damaged location and damage type of the vehicle component in the component image. For example, a business server is set up to obtain the component image of the vehicle component and output the repair solution result. At the same time, an algorithm server can also be set up to store the constructed component damage recognition model to detect and identify the component image of the business server to determine the damaged location and damage type of the vehicle component. Of course, the above-mentioned processing can also be performed by the same server.

S3: Generate a maintenance plan result for the vehicle component based on the information including the damage location and damage type of the vehicle component.

After determining the damage location and type of the vehicle component, the algorithm server can generate a repair plan for the vehicle component based on this information and preset processing rules. For example, the left front fender of the 2016 B1 model car from manufacturer A1 is slightly deformed and requires sheet metal repair; the left front door of the 2010 B2 model car from manufacturer A2 is severely scratched and severely deformed and requires replacement; the front bumper of the 2013 B3 model car from manufacturer A3 is slightly scratched and requires painting; the left headlight requires disassembly and inspection, etc.

In another embodiment of the method described herein, to meet the user's need for information on cost and price in vehicle damage assessment, the repair plan results may also include information on estimated revised prices for vehicle component repairs. This allows the user to be informed of the revised cost information and select a more appropriate repair method, thereby meeting the user's needs and improving the user experience. Therefore, in another embodiment of the method described herein, the method may further include:

S300: Acquiring information on the maintenance strategy of the vehicle component;

Correspondingly, the maintenance plan result may also include an estimated maintenance price corresponding to the maintenance strategy; the estimated maintenance price is the estimated maintenance price of the vehicle component calculated based on information including the damaged location, damage type, and maintenance strategy of the vehicle component, and after querying the price data of the products and/or maintenance services corresponding to the vehicle component in the maintenance strategy.

Figure 2 is a flowchart of another embodiment of the method described in the present application. In terms of specific implementation, a calculation rule can be designed to call different price libraries based on the information on the vehicle model to which the vehicle component belongs, the selected vehicle component maintenance location, the repair shop (4S point or ordinary comprehensive repair shop), etc., to generate a maintenance plan result for the vehicle component including a pre-repair processing method and a corresponding estimated maintenance price. The maintenance strategy information can be determined by user selection. For example, the user can select the maintenance location (such as city-level or district-level division), whether it is a 4S shop or a comprehensive repair shop, and input the car brand and model. Then, the algorithm server can obtain a maintenance plan result similar to the following based on the maintenance strategy information of the vehicle component and the identified damage location and damage type:

The front bumper of the A3 manufacturer's 2013 B3 model has slight scratches and needs to be painted; the estimated repair cost at the local 4S shop is: 600 yuan.

Of course, other implementations could also leverage traditional auto insurance companies' claims experience to compile information such as damaged parts, damage type, and extent at the accident scene, combined with repair time and costs at the 4S dealership to create an engine module. Once the actual processing application identifies the damaged part and type, the engine module can be invoked to output the damage assessment results.

The repair strategy information described above can be modified. For example, if the user chooses to have the car repaired at a 4S dealership, a repair strategy will be generated, with a corresponding repair plan result. If the user chooses to have the car repaired at a comprehensive repair shop, a different repair strategy will be generated, with a different repair plan result.

It should be understood that damage assessment typically includes both damage assessment and price assessment information. In the embodiments of this application, if the output repair plan results for a vehicle component do not include repair cost information, they can be classified as damage assessment. If they include repair cost information, it can be considered that both damage assessment and price assessment have been calculated. Both of the above repair plan results are considered part of the vehicle damage assessment process.

By using an image-based vehicle damage assessment method provided in an embodiment of the present application, a training model can be constructed that can more comprehensively identify information such as damage locations and types and produce more reliable results. This model can detect the damaged locations of vehicle components and the damage types corresponding to each damaged location, thereby obtaining the information required for accurate and comprehensive damage assessment of vehicle components. Furthermore, the implementation scheme of the present application utilizes information including the identified damaged locations and damage types to generate repair plans for vehicle components, providing insurance personnel and vehicle owners with more accurate, reliable, and practical damage assessment information. Some implementation schemes of the present application can automatically generate repair plans and estimated repair costs based on more specific vehicle damage information, a vehicle component price database, and repair processing methods, which can meet the needs of insurance companies or vehicle owners for fast, comprehensive, accurate, and reliable vehicle damage assessment processing, improve the accuracy and reliability of vehicle damage assessment processing results, and enhance user service experience.

In another implementation scenario, the acquired component image may include multiple damaged parts. In another embodiment of the method described in this application, a deep neural network model can be configured to separately identify each damaged part and the damage degree (damage type) of each damaged part. Specifically, in one embodiment, when it is determined that the currently processed component image includes at least two damaged parts, the component damage recognition model is configured to include:

S200: Identifying a damaged portion of the vehicle component and a damage type corresponding to the damaged portion based on the component image;

S210: Selecting a damage location corresponding to a damage type indicating the highest damage degree among the damage types as the damage location of the vehicle component. Accordingly, the damage type indicating the highest damage degree is the determined damage type of the vehicle component.

Usually, each damage type corresponds to a maintenance plan, such as severe deformation corresponds to replacement parts, mild deformation requires sheet metal, and minor scratches require painting. For users, the final output of a component can be a maintenance plan. When there are multiple damages, the repair method of the most severely damaged part is used as the final treatment method for the entire component. Usually, a component on a vehicle is a whole. If there are multiple damages, it is more reasonable to treat the most severe damage. In this embodiment, a maintenance plan can be selected to solve all damages. For example, if the damage type of one damaged part is severe damage and requires replacement parts, and the damage type of another damaged part is moderate deformation and requires sheet metal, then the part can be replaced without sheet metal treatment.

The above-mentioned implementation method can identify multiple damaged parts on a single component image during model training. Specifically, during sample training, the input is a picture, and the output is multiple picture areas and corresponding damage classifications. The parameters of the selected neural network can be obtained by using labeled data for mini-batch gradient descent training. For example, when mini-batch = 32, 32 training pictures are used as input for training at the same time. The labeled data marks the areas and corresponding types of pictures, and can be obtained by manually labeling real car damage pictures. The input of this neural network is a picture, and the output area is related to the number of damaged parts contained in the picture. Specifically, for example, if there is one damaged part, one picture area is output; if there are k damaged parts, k picture areas can be output; if there is no damaged part, 0 picture areas are output.

During the damage assessment process, the component image is input into this neural network. If there are multiple damaged parts, multiple image areas containing the damaged parts are detected, the image areas are detected, the damage types of the image areas are determined, and the damaged parts and damage types corresponding to each image area are output respectively. Furthermore, in this embodiment, the damaged part corresponding to the damage type with the highest degree of damage among the damage types can be selected as the damaged part of the vehicle component. Accordingly, the damage type with the highest degree of damage is the determined damage type of the vehicle component. Figure 3 is a schematic diagram of the network structure of a neural network for detecting component images provided in one embodiment of the present application.

In another implementation scenario, if multiple photos of a single component are acquired, the damage location and damage classification can be detected from each image separately, and the detection results of multiple images can be combined to obtain a more reliable result. Therefore, in another embodiment of the method described in the present application, if at least two component images of the vehicle component are acquired, the component damage recognition model constructed is used to detect the component images to determine the damage location and damage type of the vehicle component, including:

S20: Detecting the component images respectively using the component damage recognition model to determine the damage location and damage type of each component image;

S21: The damage location and damage type corresponding to each component image are combined and screened according to a predetermined rule to determine the damage location and damage type results of the vehicle component.

The merge and filter method described above can be customized. For example, for the results of the damage location and damage type of each component image, the damage type with the highest degree of damage is selected as the damage type of the vehicle component and the corresponding damage location that are finally output. For example, two pictures of the rear bumper are obtained, one is a side picture, and the other is a front picture. The result obtained by identifying the front picture is that the right side of the rear bumper is slightly deformed, but the result obtained from the side picture is that the rear bumper is damaged, which is a damage type more serious than slight deformation. Since the front picture is not fully photographed, the side picture just reflects a more realistic damage condition of the rear bumper. Therefore, at this time, the results of the two pictures are merged using the solution of this embodiment to obtain the result of the rear bumper damage, so that the output result is more reliable.

Based on the above-mentioned image-based vehicle damage assessment method, the present application also provides an image-based vehicle damage assessment device. The device may include a system (including a distributed system), software (application), modules, components, servers, clients, etc. using the method described in the present application and a device combined with necessary implementation hardware. Based on the same innovative concept, the device in an embodiment provided by the present application is as described in the following embodiment. Since the implementation scheme of the device to solve the problem is similar to the method, the implementation of the specific device of the present application can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and/or hardware for predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived. Specifically, Figure 4 is a schematic diagram of the module structure of an embodiment of an image-based vehicle damage assessment device provided by the present application. As shown in Figure 4, the device may include:

The image acquisition module 101 can be used to acquire component images corresponding to vehicle components;

The detection module 102 may be used to store the constructed component damage recognition model and detect the component image using the component damage recognition model to determine the damage location and damage type of the vehicle component;

The damage assessment processing module 103 can be used to generate a repair plan result for the vehicle component based on information including the damage location and damage type of the vehicle component.

With reference to the aforementioned method, the device may also include other implementations. For example, the component damage identification model may be a network model based on a convolutional layer and a region proposal layer, and a deep neural network may be constructed and generated after training with sample data. Alternatively, the device may also include a maintenance strategy acquisition module or directly obtain maintenance strategy information for the vehicle components through the damage assessment processing module 103, and generate a maintenance plan result including an estimated repair price. For details, please refer to the relevant description of the aforementioned method embodiment, and a detailed description will not be given here.

The method or apparatus described above in this application can be implemented through a computer program combined with necessary hardware and can be installed in an application of a device to quickly and reliably output image-based vehicle damage assessment results. Therefore, this application also provides an image-based vehicle damage assessment device that can be used on a server and can include a processor and a memory for storing processor-executable instructions. When the processor executes the instructions, it achieves the following:

Obtaining component images corresponding to vehicle components;

Detecting the component image using a constructed component damage recognition model to determine the damage location and damage type of the vehicle component;

A repair plan result for the vehicle component is generated based on information including a damage location and a damage type of the vehicle component.

The specific actual processing of the above-mentioned device may also include other processing hardware, such as a GPU (Graphics Processing Unit). As described in the above method, in another embodiment of the device, when the processor executes the instructions, it can also achieve:

obtaining information on a maintenance strategy for the vehicle component;

Correspondingly, the maintenance plan result may also include an estimated maintenance price corresponding to the maintenance strategy; the estimated maintenance price is the estimated maintenance price of the vehicle component calculated based on information including the damaged location, damage type, and maintenance strategy of the vehicle component, and after querying the price data of the products and/or maintenance services corresponding to the vehicle component in the maintenance strategy.

In another embodiment of the device, the instructions for constructing the component damage identification model may include algorithm processing instructions for constructing a deep neural network based on a network model of a convolutional layer and a region proposal layer after training with sample data.

In another embodiment of the device, when the processor executes the instructions, if it is determined that the currently processed component image includes at least two damaged locations, then executing the instructions of the component damage recognition model includes:

Identifying a damaged portion of the vehicle component and a damage type corresponding to the damaged portion based on the component image;

The damage location corresponding to the damage type indicating the highest damage degree among the damage types is selected as the damage location of the vehicle component. Accordingly, the damage type indicating the highest damage degree is the determined damage type of the vehicle component.

In another embodiment of the device, when the processor executes the instruction, if at least two component images of the vehicle component are acquired, the processor detects the component images using the constructed component damage recognition model to determine the damage location and damage type of the vehicle component, including:

Detecting the component images respectively using the component damage recognition model to determine the damage location and damage type of each component image;

The damage location and damage type corresponding to each component image are combined and screened according to predetermined rules to determine the damage location and damage type results of the vehicle component.

By using an image-based vehicle damage assessment device provided in an embodiment of the present application, a training model can be constructed that can identify information such as damage locations and types more comprehensively and produce more reliable results. This model can detect the damaged locations of vehicle parts and the damage types corresponding to each damaged location, thereby obtaining the information required for accurate and comprehensive damage assessment of vehicle parts. Furthermore, the implementation scheme of the present application uses information including the identified damage locations and damage types to generate repair plans for vehicle parts, providing insurance personnel and car owners with more accurate, reliable, and practical damage assessment information. Some implementation schemes of the present application can automatically generate repair plans and estimated repair costs based on more specific vehicle damage information, a vehicle parts price library, and repair processing methods, which can meet the needs of insurance companies or car owners for fast, comprehensive, accurate, and reliable vehicle damage assessment processing, improve the accuracy and reliability of vehicle damage assessment processing results, and enhance user service experience.

The methods or devices described in the above embodiments of the present application can implement business logic through computer programs and record them on a storage medium, which can be read and executed by a computer to achieve the effects of the solutions described in the embodiments of the present application. Therefore, the present application also provides a computer-readable storage medium on which computer instructions are stored, which, when executed, can implement the following steps:

Obtaining component images corresponding to vehicle components;

Detecting the component image using a constructed component damage recognition model to determine the damage location and damage type of the vehicle component;

A repair plan result for the vehicle component is generated based on information including a damage location and a damage type of the vehicle component.

The computer-readable storage medium may include a physical device for storing information, typically digitizing the information and then storing it in a medium utilizing electrical, magnetic, or optical means. The computer-readable storage medium described in this embodiment may include: devices that store information using electrical energy, such as various types of memory, such as RAM and ROM; devices that store information using magnetic energy, such as hard disks, floppy disks, magnetic tapes, magnetic core memories, bubble memories, and USB flash drives; and devices that store information optically, such as CDs or DVDs. Of course, there are other types of computer-readable storage media, such as quantum memories, graphene memories, and so on.

The device or method described above can be used in electronic devices for image processing to achieve rapid processing of vehicle damage assessment based on images. The electronic device can be a separate server, a system cluster composed of multiple application servers, or a server in a distributed system. Figure 5 is a structural diagram of an embodiment of an electronic device provided by the present application. In one embodiment, the electronic device may include a processor and a memory for storing processor-executable instructions, and when the processor executes the instructions, it implements:

Obtaining component images corresponding to vehicle components;

Detecting the component image using a constructed component damage recognition model to determine the damage location and damage type of the vehicle component;

A repair plan result for the vehicle component is generated based on information including a damage location and a damage type of the vehicle component.

Figure 6 is a schematic diagram of a processing scenario for vehicle damage assessment using the implementation scheme of the present application. The client in Figure 6 is the user's mobile terminal, and it can also be a PC or other terminal device in other implementation scenarios. The present application provides an image-based vehicle damage assessment method, device and electronic device, which uses deep learning technology to detect damaged parts and damage types, and uses image matching methods to accurately locate damaged parts, and can use multi-image detection results to improve the accuracy of damage assessment. Image detection technology is combined with a vehicle parts price library and maintenance rules to automatically generate repair plans and estimates. Some implementation schemes of the present application can automatically generate repair plans and estimated repair costs based on more specific vehicle damage information, vehicle parts price libraries, and repair processing methods, which can meet the needs of insurance companies or car owners for fast, comprehensive, accurate and reliable vehicle damage assessment processing, improve the accuracy and reliability of vehicle damage assessment processing results, and improve user service experience.

It should be noted that although the above embodiments provide descriptions of embodiments of some devices, electronic devices, and computer-readable storage media, based on the descriptions of the aforementioned related methods or device embodiments, the devices, electronic devices, and computer-readable storage media may also include other implementation methods. For details, please refer to the descriptions of the related methods or device embodiments, and no further examples will be given here.

Although the content of this application mentions image quality processing, convolutional neural networks and region proposal networks and deep neural networks generated by their combination, calculation methods for estimating repair prices, processing methods for multiple component images of the same component, etc., such data model construction, data acquisition, interaction, calculation, judgment, etc., however, this application is not limited to situations that must comply with industry communication standards, standard data models, computer processing and storage rules or the embodiments of this application. Certain industry standards or slightly modified implementation plans based on the implementation described in the custom method or embodiment can also achieve the same, equivalent or similar, or predictable implementation effects as the above-mentioned embodiments. The embodiments obtained by applying these modified or deformed data acquisition, storage, judgment, processing methods, etc. can still fall within the scope of the optional implementation plans of this application.

In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, an in-vehicle human-computer interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

Although the application provides the method operation steps as described in the embodiment or flow chart, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiment is only a way in the order of many step executions and does not represent a unique execution order. When the device or terminal product in practice is executed, it can be performed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (such as a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "comprise", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements not only include those elements, but also include other elements not clearly listed, or also include elements inherent to such process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements.

For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and/or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

Those skilled in the art will also appreciate that, in addition to implementing the controller in pure computer-readable program code, it is entirely possible to implement the same functionality by logically programming the method steps in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and the like. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered structures within the hardware component. Alternatively, the devices for implementing various functions can be considered both software modules implementing the method and structures within the hardware component.

The present invention is described with reference to the flowcharts and/or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of processes and/or boxes in the flowchart and/or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and/or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

The various embodiments in this specification are described in a progressive manner. Similar or identical parts between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments. In particular, since the system embodiments are generally similar to the method embodiments, the description is relatively simple. For relevant parts, reference can be made to the partial description of the method embodiments. In the description of this specification, reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as features of different embodiments or examples, without conflict.

The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims (11)

1. An image-based vehicle damage assessment method, characterized in that the method comprises: Obtain the component image corresponding to the vehicle component, wherein the component to which the component image belongs is specified when the user uploads the component image; The component damage identification model is used to detect the component image and determine the damage location and type of the vehicle component. A repair plan for the vehicle component is generated based on information including the location and type of damage. Wherein, if it is confirmed that a damaged vehicle component being processed currently includes at least two damaged areas, the component damage identification model is configured to include: Based on the component image, identify the damaged parts of the vehicle component and the corresponding damage type; The damaged part corresponding to the damage type with the highest degree of damage among the damage types is selected as the damaged part of the vehicle component. Accordingly, the damage type with the highest degree of damage is the determined damage type of the vehicle component. 2. The image-based vehicle damage assessment method as described in claim 1, characterized in that the method further comprises: Obtain information on the maintenance strategy for the vehicle components; Accordingly, the repair plan result may also include an estimated repair price corresponding to the repair strategy; the estimated repair price is calculated based on information including the damaged part, damage type, and repair strategy of the vehicle component, and after querying the price data of the product and/or repair service corresponding to the vehicle component in the repair strategy. 3. The image-based vehicle damage assessment method as described in claim 1, characterized in that the component damage identification model includes: A deep neural network is generated by training a network model based on convolutional layers and region proposal layers using sample data. 4. The image-based vehicle damage assessment method as described in claim 1, characterized in that, if at least two component images of the vehicle component are obtained, the step of detecting the component images through a constructed component damage recognition model to determine the damage location and damage type of the vehicle component includes: The component damage recognition model is used to detect the component images respectively, and the damage location and damage type of each component image are determined; According to predetermined rules, the damage location and damage type corresponding to each component image are merged and filtered to determine the damage location and damage type of the vehicle component. 5. An image-based vehicle damage assessment device, characterized in that the device comprises: The image acquisition module is used to acquire component images corresponding to vehicle components, wherein the component to which the component image belongs is specified when the user uploads the component image; A detection module is used to store a constructed component damage identification model, and to detect component images using the component damage identification model to determine the damaged location and damage type of the vehicle component; it is also used to configure the component damage identification model to include, if it is confirmed that a currently processed damaged vehicle component includes at least two damaged locations: Based on the component image, identify the damaged parts of the vehicle component and the corresponding damage type; The damaged part corresponding to the damage type with the highest degree of damage among the damage types is selected as the damaged part of the vehicle component. Accordingly, the damage type with the highest degree of damage is the determined damage type of the vehicle component. The damage assessment module is used to generate a repair plan result for the vehicle component based on information including the location and type of damage. 6. An image-based vehicle damage assessment device, characterized in that it includes a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to achieve: Obtain the component image corresponding to the vehicle component, wherein the component to which the component image belongs is specified when the user uploads the component image; The component damage identification model is used to detect the component image and determine the damage location and type of the vehicle component. A repair plan for the vehicle component is generated based on information including the location and type of damage. Wherein, when the processor executes the instruction, if it confirms that a damaged vehicle component currently being processed includes at least two damaged areas, then when executing the instruction of the component damage identification model, it implements the following: Based on the component image, identify the damaged parts of the vehicle component and the corresponding damage type; The damaged part corresponding to the damage type with the highest degree of damage among the damage types is selected as the damaged part of the vehicle component. Accordingly, the damage type with the highest degree of damage is the determined damage type of the vehicle component. 7. The image-based vehicle damage assessment device as described in claim 6, characterized in that, when the processor executes the instruction, it further implements: Obtain information on the maintenance strategy for the vehicle components; Accordingly, the repair plan result may also include an estimated repair price corresponding to the repair strategy; the estimated repair price is calculated based on information including the damaged part, damage type, and repair strategy of the vehicle component, and after querying the price data of the product and/or repair service corresponding to the vehicle component in the repair strategy. 8. The image-based vehicle damage assessment device as described in claim 6, characterized in that the instructions of the constructed component damage identification model include algorithm processing instructions for a deep neural network constructed after training with sample data based on a network model with convolutional layers and region proposal layers. 9. The image-based vehicle damage assessment device as described in claim 6, characterized in that, when the processor executes the instruction, if at least two component images of the vehicle component are acquired, the processor detects the component images through a constructed component damage recognition model to determine the damaged location and damage type of the vehicle component, including: The component damage recognition model is used to detect the component images respectively, and the damage location and damage type of each component image are determined; According to predetermined rules, the damage location and damage type corresponding to each component image are merged and filtered to determine the damage location and damage type of the vehicle component. 10. A computer-readable storage medium storing computer instructions thereon, characterized in that, when the instructions are executed, they perform the following steps: Obtain the component image corresponding to the vehicle component, wherein the component to which the component image belongs is specified when the user uploads the component image; The component damage identification model is used to detect the component image and determine the damage location and type of the vehicle component. A repair plan for the vehicle component is generated based on information including the location and type of damage. Wherein, if it is confirmed that a damaged vehicle component being processed includes at least two damaged locations, the component damage identification model is configured to include: Based on the component image, identify the damaged parts of the vehicle component and the corresponding damage type; The damaged part corresponding to the damage type with the highest degree of damage among the damage types is selected as the damaged part of the vehicle component. Accordingly, the damage type with the highest degree of damage is the determined damage type of the vehicle component. 11. An electronic device, characterized in that it includes a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements: Obtain the component image corresponding to the vehicle component, wherein the component to which the component image belongs is specified when the user uploads the component image; The component damage identification model is used to detect the component image and determine the damage location and type of the vehicle component. A repair plan for the vehicle component is generated based on information including the location and type of damage. Wherein, when the processor executes the instruction, if it confirms that a damaged vehicle component currently being processed includes at least two damaged areas, then when executing the instruction of the component damage identification model, it implements the following: Based on the component image, identify the damaged parts of the vehicle component and the corresponding damage type; The damaged part corresponding to the damage type with the highest degree of damage among the damage types is selected as the damaged part of the vehicle component. Accordingly, the damage type with the highest degree of damage is the determined damage type of the vehicle component.