CN111476275A - Image recognition-based target detection method, server and storage medium - Google Patents

Abstract

The invention discloses a target detection method based on image recognition, which is applied to a server. The method includes receiving an image to be detected uploaded by a client, extracting a corresponding image feature map, and inputting the image feature map into a target extraction model to output first image data. , judging whether the first image data contains at least two first preset target types and a first preset number of first target frames, and if the conditions are met, judge the positional relationship between the first target frames, from Find the corresponding judgment result in the database, and if the judgment result is the first judgment result, judge whether the first target frame contains the second target frame, and if so, identify and judge whether the first preset data corresponding to the second target frame is Corresponding to the second preset data in the database, and feedback information is generated according to the analysis result and fed back to the client. The present invention can replace the manual judgment of whether the rescue situation of the photo is true, and improve the judgment efficiency.

Description

Image recognition-based target detection method, server and storage medium

technical field

The invention relates to the technical field of data processing, and in particular, to a target detection method, server and storage medium based on image recognition.

Background technique

Insurance companies will provide corresponding rescue services to users who have purchased auto insurance, and entrust rescue providers to provide rescue services for users. After the rescue provider provides rescue services for users, in order to prove the authenticity of the rescue to the insurance company, rather than counterfeiting, the rescue provider needs to take pictures of the scene to collect evidence after completing the rescue. On the trailer and upload it to the service provider, the service provider manually judges whether it is a real rescue.

However, the method of judging whether the rescue in the photo is true by the naked eye is not only inefficient, but also has large errors in manual judgment, resulting in inaccurate judgment results. Therefore, how to improve the efficiency of recognizing the authenticity of photos has become a technical problem that needs to be solved urgently.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide a target detection method, server and storage medium based on picture recognition, aiming at the problem of how to improve the efficiency of identifying the authenticity of pictures.

In order to achieve the above purpose, a target detection method based on image recognition provided by the present invention is applied to a server, and the method includes:

Receiving step: receiving the image to be detected uploaded by the client, inputting the image to be detected into a pre-trained image data feature extraction model to obtain an image feature map corresponding to the image to be detected;

The first judgment step: inputting the obtained image feature map into a pre-trained target extraction model, outputting first image data, and judging whether the first image data includes at least two first targets of the first preset target type frame, and the number of first target frames of each of the preset types is a first preset number;

The second judgment step: if the first image data includes at least two first target frames of the first preset target type, and the number of the first target frames of each of the preset types is the first preset quantity , the position coordinates of each of the first target frames are obtained respectively, and the positional relationship between each of the first target frames is determined based on a preset calculation rule;

The third judgment step: according to the positional relationship, find out the corresponding judgment result from the mapping relationship table between the positional relationship and judgment result pre-established in the database, wherein the judgment result includes the first judgment result and the second judgment result. Judgment results, the first judgment result indicates that the authenticity of the information corresponding to the first image data is to be determined, and the second judgment result indicates that the authenticity of the information corresponding to the first image data is not true; and

Feedback step: if the judgment result is the first judgment result, judge whether the first target frame includes a second target frame, and if the second target frame is included, identify the corresponding second target frame the first preset data, analyze whether the first preset data corresponds to the second preset data pre-stored in the database, and generate feedback information based on the analysis results to feed back to the client, wherein the analysis results include true and untrue.

Preferably, the target extraction model is an SSD model, and the judging whether the first image data contains at least two first target frames of the first preset target types includes:

Based on the SSD model, a corresponding default frame is generated for each pixel in the image feature map, and the position coordinates of each of the default frames in the image feature map and corresponding to different first preset target types are obtained. probability score, and the maximum value in the probability score of each of the default boxes is set as the primary confidence;

Sorting the default boxes corresponding to the primary confidence level according to the probability score from large to small, taking the default box corresponding to the maximum probability score as a starting point, and sequentially obtaining a preset number of the default boxes as target candidate boxes, Perform bounding box regression analysis based on the position coordinates of each of the target candidate frames to obtain the area size corresponding to each target candidate frame;

Perform softxmax classification on the probability score of each of the target candidate frames to obtain target confidence levels corresponding to different preset target types for each of the target candidate frames; and

Based on the non-maximum value suppression algorithm, a third preset number of target candidate frames whose iou(M, b) is higher than the preset threshold are obtained as the first target frame, where M represents the default frame corresponding to the maximum probability score, b represents other default boxes except the default box M in the image feature map, and iou(M, b) represents the degree of overlap between the default box M and the default box b.

Preferably, the training process of the target extraction model includes:

Obtaining an image feature map sample, generating a corresponding default frame sample for each pixel in the image feature map sample based on the target extraction model, and obtaining the coordinate position of each default frame sample in the image feature map sample, and probability scores corresponding to different first preset target types;

Calculate the sum of the softmax classification loss and the bounding box regression loss for each default box sample, respectively, based on the location coordinates and probability score for each default box sample; and

Sort the sum of the softmax classification loss and the bounding box regression loss in descending order, take the default box sample corresponding to the smallest sum of the softmax classification loss and the bounding box regression loss as the starting point, and obtain the preset number in turn the default frame samples of The target extraction model is obtained by training by updating the weight values of each layer network of the target extraction model.

Preferably, the loss function is calculated by the following formula:

Among them, L _conf (x, c) is the softmax classification loss, L _loc (x, l, g) is the bounding box regression loss, K=|f _k |*|f _k |*α, |f _k | is the largest image The size of the feature map, α is the weight value, x is the default box, c is the category information of the default box, l is the location information of the default box, and g is the calibration area result of the default box.

Preferably, the feedback step further includes:

If the analysis result is not true, perform histogram equalization on the image to be detected to obtain second image data, adjust the second image data to a preset angle and then re-input the image in the receiving step Feature extraction model.

In order to achieve the above object, the present invention further provides a server, the server includes a memory and a processor, and the memory stores a target detection program based on picture recognition, and the target detection program based on picture recognition is processed by the The following steps are implemented when the device is executed:

Receiving step: receiving the image to be detected uploaded by the client, inputting the image to be detected into a pre-trained image data feature extraction model to obtain an image feature map corresponding to the image to be detected;

The first judgment step: inputting the obtained image feature map into a pre-trained target extraction model, outputting first image data, and judging whether the first image data includes at least two first targets of the first preset target type frame, and the number of first target frames of each of the preset types is a first preset number;

The second judgment step: if the first image data includes at least two first target frames of the first preset target type, and the number of the first target frames of each of the preset types is the first preset quantity , the position coordinates of each of the first target frames are obtained respectively, and the positional relationship between each of the first target frames is determined based on a preset calculation rule;

The third judgment step: according to the positional relationship, find out the corresponding judgment result from the mapping relationship table between the positional relationship and judgment result pre-established in the database, wherein the judgment result includes the first judgment result and the second judgment result. Judgment results, the first judgment result indicates that the authenticity of the information corresponding to the first image data is to be determined, and the second judgment result indicates that the authenticity of the information corresponding to the first image data is not true; and

Feedback step: if the judgment result is the first judgment result, judge whether the first target frame includes a second target frame, and if the second target frame is included, identify the corresponding second target frame the first preset data, analyze whether the first preset data corresponds to the second preset data pre-stored in the database, and generate feedback information based on the analysis results to feed back to the client, wherein the analysis results include true and untrue.

Preferably, the target extraction model is an SSD model, and the judging whether the first image data contains at least two first target frames of the first preset target types includes:

Based on the SSD model, a corresponding default frame is generated for each pixel in the image feature map, and the position coordinates of each default frame in the image feature map and the probability corresponding to different first preset target types are obtained. score, and set the maximum value in the probability score of each of the default boxes as the primary confidence;

Sort the default boxes corresponding to the primary confidence levels in descending order of probability score, take the default box corresponding to the maximum probability score as the starting point, and sequentially obtain a second preset number of the default boxes as target candidates frame, and perform bounding box regression analysis based on the position coordinates of each target candidate frame to obtain the area size corresponding to each target candidate frame;

Perform softxmax classification on the probability score of each of the target candidate frames to obtain target confidence levels corresponding to different preset target types for each of the target candidate frames; and

Based on the non-maximum value suppression algorithm, a third preset number of target candidate frames whose iou(M, b) is higher than the preset threshold are obtained as the first target frame, where M represents the default frame corresponding to the maximum probability score, b represents other default boxes except the default box M in the image feature map, and iou(M, b) represents the degree of overlap between the default box M and the default box b.

Preferably, the training process of the target extraction model includes:

Obtaining an image feature map sample, generating a corresponding default frame sample for each pixel in the image feature map sample based on the target extraction model, and obtaining the coordinate position of each default frame sample in the image feature map sample, and probability scores corresponding to different first preset target types;

Calculate the sum of the softmax classification loss and the bounding box regression loss for each default box sample, respectively, based on the location coordinates and probability score for each default box sample; and

Sort the sum of the softmax classification loss and the bounding box regression loss in descending order, take the default box sample corresponding to the smallest sum of the softmax classification loss and the bounding box regression loss as the starting point, and obtain the preset number in turn the default frame samples of The target extraction model is obtained by training by updating the weight values of each layer network of the target extraction model.

Preferably, the loss function is calculated by the following formula:

Among them, L _conf (x, c) is the softmax classification loss, L _loc (x, l, g) is the bounding box regression loss, K=|f _k |*|f _k |*α, |f _k | is the largest image The size of the feature map, α is the weight value, x is the default box, c is the category information of the default box, l is the location information of the default box, and g is the calibration area result of the default box.

In order to achieve the above object, the present invention further provides a computer-readable storage medium on which a target detection program based on picture recognition is stored, and the target detection program based on picture recognition can be stored by one or more The processor executes the steps of the above-mentioned image recognition-based target detection method.

The target detection method, server and storage medium based on image recognition proposed by the present invention receive the image to be detected uploaded by the client, input the to-be-detected image into an image data feature extraction model to obtain an image feature map, and input the image feature map into the target extraction model Outputting the first image data, judging whether the first image data contains at least two first target frames of the first preset target type, and the number of each first target frame is the first preset quantity, and if the conditions are met, respectively Obtain the position coordinates of each first target frame, and judge the positional relationship between each first target frame, and find out the corresponding judgment result from the mapping relationship table in the database according to the positional relationship, if the judgment result is the first The judgment result is to judge whether the first target frame contains the second target frame, and if so, identify the first preset data corresponding to the second target frame, and judge whether the first preset data is the same as the second preset data in the database. The corresponding data is generated, and feedback information is generated and fed back to the client according to the analysis results. The present invention can replace manual judgment to determine whether the rescue situation is true by judging photos, improve judgment efficiency, and reduce errors caused by manual judgment at the same time.

Description of drawings

Fig. 1 is the application environment diagram of the preferred embodiment of the server of the present invention;

Fig. 2 is the program module schematic diagram of the preferred embodiment of the target detection program based on picture recognition in Fig. 1;

FIG. 3 is a schematic flowchart of a preferred embodiment of a target detection method based on image recognition according to the present invention.

The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Detailed ways

In order to make the purpose, technology, and advantages of this embodiment of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that the descriptions involving "first", "second", etc. in the present invention are only for the purpose of description, and should not be construed as indicating or implying their relative importance or implying the number of indicated technical features . Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In addition, the technology between the various embodiments can be combined with each other, but it must be based on the realization by those of ordinary skill in the art. When the combination of the technology in this embodiment is contradictory or cannot be realized, it should be considered that this technology is based on Combinations of embodiments do not exist and are not within the scope of protection claimed by the present invention.

The present invention provides a server 1 .

The server 1 includes, but is not limited to, a memory 11 , a processor 12 and a network interface 13 .

The memory 11 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (eg, SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, and the like. The memory 11 may be an internal storage unit of the server 1 in some embodiments, such as a hard disk of the server 1 . In other embodiments, the memory 11 may also be an external storage device of the server 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital, SD) card equipped on the server 1. , Flash card (Flash Card) and so on.

Further, the memory 11 may also include both an internal storage unit of the server 1 and an external storage device. The memory 11 can be used not only to store application software installed on the server 1 and various data, such as the code of the target detection program 10 based on image recognition, etc., but also to temporarily store data that has been output or will be output.

In some embodiments, the processor 12 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor or other data processing chip for executing program codes or processing stored in the memory 11 Data, for example, the target detection program 10 based on image recognition is executed.

Optionally, the network interface 13 may include a standard wired interface and a wireless interface (such as a WI-FI interface), which is usually used to establish a communication connection between the server and other electronic devices.

The client 14 may be a desktop computer, a notebook computer, a tablet computer, a mobile phone, or the like.

The network 15 may be the Internet, a cloud network, a wireless fidelity (Wi-Fi) network, a personal network (PAN), a local area network (LAN), and/or a metropolitan area network (MAN). Various devices in a network environment may be configured to connect to communication networks according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP/IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Optical Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocol and/or a Bluetooth (BlueTooth) communication protocol or a combination thereof.

Optionally, the server 1 may further include a user interface, and the user interface may include a display (Display), an input unit such as a keyboard (Keyboard), and an optional user interface may also include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, and the like. The display may also be referred to as a display screen or a display unit for displaying information processed in the server 1 and for displaying a visual user interface.

FIG. 1 only shows the server 1 having components 11-15 and the target detection program 10 based on image recognition. Those skilled in the art can understand that the structure shown in FIG. 1 does not constitute a limitation on the server 1, and may include Fewer or more components than shown, or some components are combined, or a different arrangement of components.

In this embodiment, when the image recognition-based target detection program 10 of FIG. 1 is executed by the processor 12, the following steps are implemented:

Receiving step: receiving the image to be detected uploaded by the client 14, inputting the image to be detected into a pre-trained image data feature extraction model to obtain an image feature map corresponding to the image to be detected;

The first judgment step: inputting the obtained image feature map into a pre-trained target extraction model, outputting first image data, and judging whether the first image data includes at least two first targets of the first preset target type frame, and the number of first target frames of each of the preset types is a first preset number;

The second judgment step: if the first image data includes at least two first target frames of the first preset target type, and the number of the first target frames of each of the preset types is the first preset quantity , the position coordinates of each of the first target frames are obtained respectively, and the positional relationship between each of the first target frames is determined based on a preset calculation rule;

The third judgment step: according to the positional relationship, find out the corresponding judgment result from the mapping relationship table between the positional relationship and judgment result pre-established in the database, wherein the judgment result includes the first judgment result and the second judgment result. Judgment results, the first judgment result indicates that the authenticity of the information corresponding to the first image data is to be determined, and the second judgment result indicates that the authenticity of the information corresponding to the first image data is not true; and

Feedback step: if the judgment result is the first judgment result, judge whether the first target frame includes a second target frame, and if the second target frame is included, identify the corresponding second target frame the first preset data, analyze whether the first preset data corresponds to the second preset data pre-stored in the database, and generate feedback information according to the analysis results to feed back to the client 14, wherein the analysis results include true and false.

In another embodiment, the feedback step further includes:

If the analysis result is not true, perform histogram equalization on the image to be detected to obtain second image data, adjust the second image data to a preset angle and then re-input the image in the receiving step Feature extraction model.

For a detailed description of the above steps, please refer to the following descriptions of FIG.

Referring to FIG. 2 , it is a schematic diagram of program modules of an embodiment of the target detection program 10 based on image recognition in FIG. 1 . The target detection program 10 based on image recognition is divided into a plurality of modules, and the plurality of modules are stored in the memory 11 and executed by the processor 12 to complete the present invention. A module referred to in the present invention refers to a series of computer program instruction segments capable of accomplishing specific functions.

In this embodiment, the target detection program 10 based on image recognition includes a receiving module 110 , a first judging module 120 , a second judging module 130 , a third judging module 140 and a feedback module 150 .

The receiving module 110 is configured to receive the to-be-detected image uploaded by the client 14, input the to-be-detected image into a pre-trained image data feature extraction model, and obtain an image feature map corresponding to the to-be-detected image.

In this embodiment, the server 1 receives the image to be detected uploaded by the client 14 (for example, a camera or other shooting terminal with shooting function, or a device with shooting function and image transmission function), and uses a pre-trained image feature extraction model to extract images from the image. The image feature map is extracted from the image to be detected. In this embodiment, the image feature extraction model is trained by the MobileNetV2 network model. The MobileNetV2 network model is a lightweight convolutional neural network structure model. The MobileNetV2 network model can efficiently identify images with low resolution. , and has the characteristics of small operation bandwidth, which can be used on mobile devices. The MobileNetV2 network model includes 53 layers of convolutional layers, 1 layer of pooling layers and 1 layer of fully connected layers connected in sequence, among which the 53 layers of convolutional layers include 1 layer of input layer, 17 bottleneck building blocks and 1 layer of output layer connected in sequence Each bottleneck building block includes 3 convolutional layers, and the convolution kernels of the 53 convolutional layers are all 3×3.

In other embodiments, when training the MobileNetV2 network model, a loss function can be set for the MobileNetV2 network model in advance, the training samples are input into the MobileNetV2 network model, and the input training samples are forward propagated to obtain the actual output, Substitute the preset target output and the actual output into the loss function, calculate the loss value of the loss function, perform backpropagation and use the loss value to optimize the parameters of the MobileNetV2 network model to obtain the optimized MobileNetV2 network model. Then select a training sample and input it into the optimized MobileNetV2 network model. Referring to the above operation, train the optimized MobileNetV2 network model again until the condition for stopping training is reached.

The first judgment module 120 is used for inputting the obtained image feature map into a pre-trained target extraction model, outputting first image data, and judging whether the first image data contains at least two types of first preset targets. A first target frame, and the number of the first target frames of each preset type is a first preset quantity.

In this embodiment, when it is determined that the first image data includes first target frames of at least two first preset target types, and the number of first target frames of each preset type is the first preset number (in In this embodiment, when the first preset number is 1), it means that the image to be detected uploaded by the client 14 meets the requirements, and the subsequent steps are executed; otherwise, it means that the image to be detected uploaded by the client 14 does not meet the requirements, and a feedback is generated The information is fed back to the client 14 .

The target frame is drawn based on a third-party marking tool (eg, RectLabel), and each target frame corresponds to a first preset target type.

The following is further explained with specific examples:

For example, insurance companies provide corresponding rescue services to users who have purchased auto insurance, and entrust rescue providers to provide rescue services for users. When rescue providers provide rescue services for users, in order to prove the authenticity of the rescue to the insurance company, rather than counterfeiting, the rescue providers need to take pictures of the scene to collect evidence and feedback to the service providers after completing the rescue.

Therefore, through this scheme, after the photos taken by rescuers on site are input into the image feature extraction model, the image feature map of the photos is obtained, and then the image feature maps are input into the target extraction model to obtain the first image data of the photos. An image data is analyzed and judged. When it is judged that the first image data includes at least two first target types of first preset target types (eg, a broken car, a trailer), a first target frame (at least the frame of the broken car and the image of the trailer) are included. frame), and when the number of the first target frames of each preset type is the first preset number, it means that the image to be detected uploaded by the client 14 meets the requirements, and the subsequent steps are executed; otherwise, it is explained that the image to be detected uploaded by the client 14 meets the requirements; It is detected that the image does not meet the requirements, there may be fraudulent behavior, or the captured picture does not meet the requirements, and feedback information is generated and fed back to the client 14 .

After that, input the image feature map into a pre-trained target extraction model to obtain first image data corresponding to the image to be monitored. The target extraction model is an SSD model. In the above step, judging whether the first image data contains at least two first target frames of the first preset target type includes:

Based on the SSD model, a corresponding default frame is generated for each pixel in the image feature map, and the position coordinates of each of the default frames in the image feature map and corresponding to different first preset target types are obtained. probability score, and the maximum value in the probability score of each of the default boxes is set as the primary confidence;

Sort the default boxes corresponding to the primary confidence levels in descending order of probability score, take the default box corresponding to the maximum probability score as the starting point, and sequentially obtain a second preset number of the default boxes as target candidates frame, and perform bounding box regression analysis based on the position coordinates of each target candidate frame to obtain the area size corresponding to each target candidate frame;

Perform softxmax classification on the probability score of each of the target candidate frames to obtain target confidence levels corresponding to different preset target types for each of the target candidate frames; and

Based on the non-maximum value suppression algorithm, a third preset number of target candidate frames whose iou(M, b) is higher than the preset threshold are obtained as the first target frame, where M represents the default frame corresponding to the maximum probability score, b represents other default boxes except the default box M in the image feature map, and iou(M, b) represents the degree of overlap between the default box M and the default box b.

The training process of the target extraction model includes:

The acquired image feature map samples, based on the target extraction model, generate corresponding default frame samples for each pixel in the image feature map samples respectively, and obtain the coordinate position of each default frame sample in the image feature map sample , and the probability scores corresponding to different first preset target types;

Calculate the sum of the softmax classification loss and the bounding box regression loss for each default box sample, respectively, based on the location coordinates and probability score for each default box sample; and

Sort the sum of the softmax classification loss and the bounding box regression loss in descending order, take the default box sample corresponding to the smallest sum of the softmax classification loss and the bounding box regression loss as the starting point, and obtain the preset number in turn the default frame samples of Update the weight value of each layer network of the target extraction model, and train to obtain the target extraction model;

The loss function is calculated by the following formula:

Among them, L _conf (x, c) is the softmax classification loss, L _loc (x, l, g) is the bounding box regression loss, K=|f _k |*|f _k |*α, |f _k | is the largest image The size of the feature map, α is the weight value, x is the default box, c is the category information of the default box, l is the location information of the default box, and g is the calibration area result of the default box.

The second judging module 130 is configured to, if the first image data includes at least two first target frames of the first preset target type, and the number of the first target frames of each preset type is the first The preset number (in this embodiment, the first preset number is 1), the position coordinates of each of the first target frames are obtained respectively, and the distance between each of the first target frames is determined based on the preset calculation rule. Positional relationship.

In this embodiment, when identifying the first target frame containing at least two first preset target types in the first image data, the position coordinates of the four vertices of the first target frame output by the target extraction model are obtained respectively, And determine the positional relationship between the first target frames based on the preset calculation rules. Taking the faulty vehicle and the trailer as an example, the positional relationship can be that the faulty vehicle is above the trailer, or the faulty vehicle is below the trailer, or the faulty vehicle is in the trailer Left, or the faulty car is on the right of the trailer.

Wherein, the calculation rule is: respectively take the coordinates of each vertex of the first target frame represented by the faulty car and the first target frame represented by the trailer, that is, the upper left coordinate of the faulty vehicle (x1, y1), and the upper right coordinate of the faulty vehicle (x2, y2 ), the lower left coordinate of the faulty vehicle (x4, y4), the lower right coordinate of the faulty vehicle (x3, y3), the upper left coordinate of the trailer (a1, b1), the upper right coordinate of the trailer (a2, b2), the lower left coordinate of the trailer (a4, b4), The lower right coordinates of the trailer (a3, b3), subtract a1 from x1, and subtract a2 from x2. If the result is one positive and one negative, it means that the faulty vehicle is in the middle of the trailer. At the same time, subtract b1 from y1 and subtract b2 from y2. If the results are all positive, it means that the faulty vehicle is located above the trailer. When the above two conditions are met, it means that the faulty vehicle is located directly above the trailer. It can be preliminarily determined that the photos uploaded by the rescuer meet the requirements of the insurer for on-site evidence collection.

The third judging module 140 is configured to, according to the positional relationship, find out the corresponding judgment result from the mapping relationship table between the positional relationship and judgment result pre-established in the database, wherein the judgment result includes the first judgment result and a second judgment result, the first judgment result indicates that the authenticity of the information corresponding to the first image data is to be determined, and the second judgment result indicates that the authenticity of the information corresponding to the first image data is not true.

In order to further illustrate the specific solution of this step, in this embodiment, the above-mentioned example is continued to be described. The corresponding judgment result is found from the mapping relationship table between the position relationship and the judgment result pre-established in the database, wherein the judgment result includes the first judgment result and the second judgment result. When the first image data shows that the faulty car is above the trailer, it means that the photos uploaded by the rescuer meet the requirements, but because the authenticity of the faulty car or the trailer may be uncertain, that is, to be determined, the client 14 will be sent the first Feedback information of the judgment result (to be determined, that is, the real identity of the faulty vehicle and/or the faulty vehicle is to be determined), the feedback information indicates that the authenticity of the information corresponding to the first image data is to be determined. When the first image data does not show that the faulty vehicle is placed above the trailer, it means that the photos uploaded by the rescue provider do not meet the requirements, that is, a feedback message of the second judgment result (not true) is sent to the client 14 .

The feedback module 150 is configured to judge whether the first target frame includes a second target frame if the judgment result is the first judgment result, and identify the second target frame if the second target frame is included. For the first preset data corresponding to the target frame, analyze whether the first preset data corresponds to the second preset data pre-stored in the database, and generate feedback information to feed back to the client 14 according to the analysis result, wherein the Analysis results include true and false.

In order to avoid the situation where the authenticity of the faulty car or trailer may be uncertain, for example, the rescue company may upload fake photos, and the faulty car or trailer in the photo was not taken by the rescue company on the spot. Therefore, in this embodiment, after judging the After the positional relationship between the different first target frames, the first target frame still needs to be analyzed, and the specific solution of this step is described with the above example. If the judgment result is the first judgment result, that is, the photo uploaded by the rescue provider shows that the faulty vehicle is above the trailer, then it is determined whether the first target frame contains the second target frame, such as the license plate of the faulty vehicle or the trailer. Wherein, if the first target frame contains the second target frame, the first preset data (for example, the license plate number) corresponding to the second target frame are respectively identified, and the first preset data and the second preset data pre-stored in the database are analyzed. Set whether the data (for example, the name of the car owner) corresponds, and generate feedback information according to the analysis result and feed it back to the client 14 .

In another embodiment, the feedback module 150 is further configured to perform histogram equalization processing on the to-be-detected image to obtain second image data if the analysis result is false, and the second image data After adjusting to the preset angle, re-input the image feature extraction model in the receiving step.

In this embodiment, if the analysis result is false, it may be that the photos uploaded by the rescue provider are fake photos, or the photos uploaded by the rescue provider do not meet the server identification requirements, for example, the photo is dark or the angle is not well placed. of. Therefore, when the analysis result is not real-time, the to-be-detected image can be processed by histogram equalization to obtain the second image data, and the second image data can be adjusted to a preset angle (for example, 270°, that is, symmetrically flipped) and then input again. Receive the image feature extraction model in the step, and repeat the above steps.

In addition, the present invention also provides a target detection method based on image recognition. Referring to FIG. 3 , it is a schematic flowchart of an embodiment of a target detection method based on image recognition according to the present invention. When the processor 12 of the server 1 executes the image recognition-based target detection program 10 stored in the memory 11, the following steps are implemented to realize the image recognition-based target detection method:

S110: Receive the to-be-detected image uploaded by the client 14, input the to-be-detected image into a pre-trained image data feature extraction model, and obtain an image feature map corresponding to the to-be-detected image.

In this embodiment, the server 1 receives the image to be detected uploaded by the client 14 (for example, a camera or other shooting terminal with shooting function, or a device with shooting function and image transmission function), and uses a pre-trained image feature extraction model to extract images from the image. The image feature map is extracted from the image to be detected. In this embodiment, the image feature extraction model is trained by the MobileNetV2 network model. The MobileNetV2 network model is a lightweight convolutional neural network structure model. The MobileNetV2 network model can efficiently identify images with low resolution. , and has the characteristics of small operation bandwidth, which can be used on mobile devices. The MobileNetV2 network model includes 53 layers of convolutional layers, 1 layer of pooling layers and 1 layer of fully connected layers connected in sequence, among which the 53 layers of convolutional layers include 1 layer of input layer, 17 bottleneck building blocks and 1 layer of output layer connected in sequence Each bottleneck building block includes 3 convolutional layers, and the convolution kernels of the 53 convolutional layers are all 3×3.

In other embodiments, when training the MobileNetV2 network model, a loss function can be set for the MobileNetV2 network model in advance, the training samples are input into the MobileNetV2 network model, and the input training samples are forward propagated to obtain the actual output, Substitute the preset target output and the actual output into the loss function, calculate the loss value of the loss function, perform backpropagation and use the loss value to optimize the parameters of the MobileNetV2 network model to obtain the optimized MobileNetV2 network model. Then select a training sample and input it into the optimized MobileNetV2 network model. Referring to the above operation, train the optimized MobileNetV2 network model again until the condition for stopping training is reached.

S120: Input the obtained image feature map into a pre-trained target extraction model, output first image data, and determine whether the first image data includes first target frames of at least two first preset target types, and The number of the first target frames of each preset type is the first preset number.

In this embodiment, when it is determined that the first image data includes first target frames of at least two first preset target types, and the number of first target frames of each preset type is the first preset number (in In this embodiment, when the first preset number is 1), it means that the image to be detected uploaded by the client 14 meets the requirements, and the subsequent steps are executed; otherwise, it means that the image to be detected uploaded by the client 14 does not meet the requirements, and a feedback is generated The information is fed back to the client 14 .

The target frame is drawn based on a third-party marking tool (eg, RectLabel), and each target frame corresponds to a first preset target type.

The following is further explained with specific examples:

For example, insurance companies provide corresponding rescue services to users who have purchased auto insurance, and entrust rescue providers to provide rescue services for users. When rescue providers provide rescue services for users, in order to prove the authenticity of the rescue to the insurance company, rather than counterfeiting, the rescue providers need to take pictures of the scene to collect evidence and feedback to the service providers after completing the rescue.

Therefore, through this scheme, after the photos taken by rescuers on site are input into the image feature extraction model, the image feature map of the photos is obtained, and then the image feature maps are input into the target extraction model to obtain the first image data of the photos. An image data is analyzed and judged, when it is judged that the first image data includes at least two first target types of the first preset target types (such as a faulty car and a trailer) in a first target frame (at least the frame of the faulty car and the image of the trailer). frame), and when the number of the first target frames of each preset type is the first preset number, it means that the image to be detected uploaded by the client 14 meets the requirements, and the subsequent steps are executed; otherwise, it is explained that the image to be detected uploaded by the client 14 meets the requirements; It is detected that the image does not meet the requirements, there may be fraudulent behavior, or the captured picture does not meet the requirements, and feedback information is generated and fed back to the client 14 .

After that, input the image feature map into a pre-trained target extraction model to obtain first image data corresponding to the image to be monitored. The target extraction model is an SSD model. In the above step, judging whether the first image data contains at least two first target frames of the first preset target type includes:

Based on the SSD model, a corresponding default frame is generated for each pixel in the image feature map, and the position coordinates of each of the default frames in the image feature map and corresponding to different first preset target types are obtained. probability score, and the maximum value in the probability score of each of the default boxes is set as the primary confidence;

Sort the default boxes corresponding to the primary confidence levels in descending order of probability score, take the default box corresponding to the maximum probability score as the starting point, and sequentially obtain a second preset number of the default boxes as target candidates frame, and perform bounding box regression analysis based on the position coordinates of each target candidate frame to obtain the area size corresponding to each target candidate frame;

Perform softxmax classification on the probability score of each of the target candidate frames to obtain target confidence levels corresponding to different preset target types for each of the target candidate frames; and

Based on the non-maximum value suppression algorithm, a third preset number of target candidate frames whose iou(M, b) is higher than the preset threshold are obtained as the first target frame, where M represents the default frame corresponding to the maximum probability score, b represents other default boxes except the default box M in the image feature map, and iou(M, b) represents the degree of overlap between the default box M and the default box b.

The training process of the target extraction model includes:

The acquired image feature map samples, based on the target extraction model, generate corresponding default frame samples for each pixel in the image feature map samples respectively, and obtain the coordinate position of each default frame sample in the image feature map sample , and the probability scores corresponding to different first preset target types;

Calculate the sum of the softmax classification loss and the bounding box regression loss for each default box sample, respectively, based on the location coordinates and probability score for each default box sample; and

Sort the sum of the softmax classification loss and the bounding box regression loss in descending order, take the default box sample corresponding to the smallest sum of the softmax classification loss and the bounding box regression loss as the starting point, and obtain the preset number in turn the default frame samples of Update the weight value of each layer network of the target extraction model, and train to obtain the target extraction model;

The loss function is calculated by the following formula:

Among them, L _conf (x, c) is the softmax classification loss, L _loc (x, l, g) is the bounding box regression loss, K=|f _k |*|f _k |*α, |f _k | is the largest image The size of the feature map, α is the weight value, x is the default box, c is the category information of the default box, l is the location information of the default box, and g is the calibration area result of the default box.

S130, if the first image data includes at least two first target frames of a first preset target type, and the number of first target frames of each of the preset types is a first preset quantity (in this case In the embodiment, if the first preset number is 1), the position coordinates of each of the first target frames are obtained respectively, and the positional relationship between each of the first target frames is determined based on a preset calculation rule.

In this embodiment, when identifying the first target frame containing at least two first preset target types in the first image data, the position coordinates of the four vertices of the first target frame output by the target extraction model are obtained respectively, And determine the positional relationship between the first target frames based on the preset calculation rules. Taking the faulty vehicle and the trailer as an example, the positional relationship can be that the faulty vehicle is above the trailer, or the faulty vehicle is below the trailer, or the faulty vehicle is in the trailer Left, or the faulty car is on the right of the trailer.

Wherein, the calculation rule is: respectively take the coordinates of each vertex of the first target frame represented by the faulty car and the first target frame represented by the trailer, that is, the upper left coordinate of the faulty vehicle (x1, y1), and the upper right coordinate of the faulty vehicle (x2, y2 ), the lower left coordinate of the faulty vehicle (x4, y4), the lower right coordinate of the faulty vehicle (x3, y3), the upper left coordinate of the trailer (a1, b1), the upper right coordinate of the trailer (a2, b2), the lower left coordinate of the trailer (a4, b4), The lower right coordinates of the trailer (a3, b3), subtract a1 from x1, and subtract a2 from x2. If the result is one positive and one negative, it means that the faulty vehicle is in the middle of the trailer. At the same time, subtract b1 from y1 and subtract b2 from y2. If the results are all positive, it means that the faulty vehicle is located above the trailer. When the above two conditions are met, it means that the faulty vehicle is located directly above the trailer. It can be preliminarily determined that the photos uploaded by the rescuer meet the requirements of the insurer for on-site evidence collection.

S140, according to the positional relationship, find out the corresponding judgment result from the mapping relationship table between the positional relationship and the judgment result pre-established in the database, wherein the judgment result includes a first judgment result and a second judgment result, The first judgment result indicates that the authenticity of the information corresponding to the first image data is to be determined, and the second judgment result indicates that the authenticity of the information corresponding to the first image data is not true.

In order to further illustrate the specific solution of this step, in this embodiment, the above-mentioned example is continued to be described. The corresponding judgment result is found from the mapping relationship table between the position relationship and the judgment result pre-established in the database, wherein the judgment result includes the first judgment result and the second judgment result. When the first image data shows that the faulty car is above the trailer, it means that the photos uploaded by the rescuer meet the requirements, but because the authenticity of the faulty car or the trailer may be uncertain, that is, to be determined, the client 14 will be sent the first Feedback information of the judgment result (to be determined, that is, the real identity of the faulty vehicle and/or the faulty vehicle is to be determined), the feedback information indicates that the authenticity of the information corresponding to the first image data is to be determined. When the first image data does not show that the faulty vehicle is placed above the trailer, it means that the photos uploaded by the rescue provider do not meet the requirements, that is, a feedback message of the second judgment result (not true) is sent to the client 14 .

S150: If the judgment result is the first judgment result, judge whether the first target frame includes a second target frame, and if the second target frame is included, identify the corresponding frame of the second target frame. For the first preset data, analyze whether the first preset data corresponds to the second preset data pre-stored in the database, and generate feedback information according to the analysis result to feed back to the client 14, wherein the analysis result includes true and untrue.

In order to avoid the situation where the authenticity of the faulty car or trailer may be uncertain, for example, the rescue company may upload fake photos, and the faulty car or trailer in the photo was not taken by the rescue company on the spot. Therefore, in this embodiment, after judging the After the positional relationship between the different first target frames, the first target frame still needs to be analyzed, and the specific solution of this step is described with the above example. If the judgment result is the first judgment result, that is, the photo uploaded by the rescue provider shows that the faulty vehicle is above the trailer, then it is determined whether the first target frame contains the second target frame, such as the license plate of the faulty vehicle or the trailer. Wherein, if the first target frame includes a second target frame, the first preset data (for example, the license plate number) corresponding to the second target frame are respectively identified, and the first preset data and the second preset data pre-stored in the database are analyzed. Set whether the data (for example, the name of the car owner) corresponds, and generate feedback information according to the analysis result and feed it back to the client 14 .

In another embodiment, the method further includes the following steps:

If the analysis result is not true, perform histogram equalization on the image to be detected to obtain second image data, adjust the second image data to a preset angle and then re-input the image in the receiving step Feature extraction model.

In this embodiment, if the analysis result is false, it may be that the photos uploaded by the rescue provider are fake photos, or the photos uploaded by the rescue provider do not meet the server identification requirements, for example, the photo is dark or the angle is not well placed. of. Therefore, when the analysis result is not real-time, the to-be-detected image can be processed by histogram equalization to obtain the second image data, and the second image data can be adjusted to a preset angle (for example, 270°, that is, symmetrically flipped) and then input again. Receive the image feature extraction model in the step, and repeat the above steps.

In addition, an embodiment of the present invention also provides a computer-readable storage medium, where the computer-readable storage medium may be a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory Memory (EPROM), portable compact disk read only memory (CD-ROM), USB memory, etc. any one or any combination of several. The computer-readable storage medium includes a target detection program 10 based on image recognition. The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the specific implementation of the above-mentioned image recognition-based target detection method and server 1. Repeat.

It should be noted that the above-mentioned preambles of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments. And the terms "comprising", "comprising" or any other variation thereof herein are intended to encompass a non-exclusive inclusion such that a process, device, article or method comprising a list of elements includes not only those elements, but also includes no explicit Other elements listed, or those inherent to such a process, apparatus, article, or method are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, apparatus, article, or method that includes the element.

The above-mentioned preambles of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments. From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on such understanding, the technical aspects of the present invention can be embodied in the form of software products that are essentially or contribute to the prior art, and the computer software products are stored in a storage medium (such as a ROM) as described above. /RAM, magnetic disk, optical disk), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in the various embodiments of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.

Claims (10)

1. A target detection method based on picture recognition is applied to a server, and is characterized by comprising the following steps:

a receiving step: receiving an image to be detected uploaded by a client, inputting the image to be detected into a pre-trained image data feature extraction model, and obtaining an image feature map corresponding to the image to be detected;

a first judgment step: inputting the obtained image feature map into a pre-trained target extraction model, outputting first image data, and judging whether the first image data contains first target frames of at least two first preset target types, wherein the number of the first target frames of each preset type is a first preset number;

a second judgment step: if the first image data comprises first target frames of at least two first preset target types and the number of the first target frames of each preset type is a first preset number, respectively acquiring the position coordinates of the first target frames, and judging the position relation between the first target frames based on a preset calculation rule;

a third judging step: according to the position relation, finding out a corresponding judgment result from a mapping relation table between the position relation and the judgment result which are pre-established in a database, wherein the judgment result comprises a first judgment result and a second judgment result, the first judgment result shows that the authenticity of the information corresponding to the first image data is to be determined, and the second judgment result shows that the authenticity of the information corresponding to the first image data is not true; and

a feedback step: if the judgment result is a first judgment result, respectively judging whether the first target frame comprises a second target frame, if so, respectively identifying first preset data corresponding to the second target frame, analyzing whether the first preset data correspond to second preset data stored in a database in advance, and generating feedback information according to the analysis result to feed back the feedback information to the client, wherein the analysis result comprises an actual fact and a non-actual fact.

2. The method as claimed in claim 1, wherein the object extraction model is an SSD model, and the determining whether the first image data includes a first object frame of at least two first preset object types comprises:

respectively generating corresponding default frames for each pixel point in the image feature map based on the SSD model, acquiring position coordinates of each default frame in the image feature map and probability scores corresponding to different first preset target types, and setting the maximum value in the probability scores of each default frame as a primary confidence coefficient;

sorting the default frames corresponding to the primary confidence degrees from large to small according to probability scores, sequentially obtaining a second preset number of default frames as target candidate frames by taking the default frame corresponding to the maximum value of the probability scores as a starting point, and performing bounding box regression analysis based on the position coordinates of each target candidate frame to obtain the area size corresponding to each target candidate frame;

performing softxmax classification on the probability score of each target candidate frame to obtain target confidence coefficients of each target candidate frame corresponding to different preset target type classifications; and

and acquiring a third preset number of target candidate frames with iou (M, b) higher than a preset threshold as the first target frame based on a non-maximum suppression algorithm, wherein M represents a default frame corresponding to the maximum probability score, b represents other default frames except the default frame M in the image feature map, and iou (M, b) represents the overlapping degree between the default frame M and the default frame b.

3. The image recognition-based target detection method of claim 2, wherein the training process of the target extraction model comprises:

acquiring an image feature map sample, respectively generating corresponding default frame samples for each pixel point in the image feature map sample based on the target extraction model, and acquiring the coordinate position of each default frame sample in the image feature map sample and probability scores corresponding to different first preset target types;

respectively calculating the sum of softmax classification loss and bounding box regression loss of each default frame sample based on the position coordinate and probability score of each default frame sample; and

sequencing the sum of the softmax classification loss and the bounding box regression loss from large to small, sequentially acquiring a preset number of default frame samples by taking the default frame sample corresponding to the minimum softmax classification loss and the bounding box regression loss as a starting point, calculating the loss functions of the default frame samples of the preset number, performing back propagation on the calculated loss functions of the default frame samples of the preset number in the target extraction model, updating the weight values of each layer of network of the target extraction model, and training to obtain the target extraction model.

4. The image recognition-based target detection method of claim 3, wherein the loss function is calculated by the following formula:

wherein, L_conf(x, c) Softmax classification loss, L_loc(x, l, g) is bounding box regression loss, K ═ f_k|*|f_k|*α，|f_kThe method comprises the steps of |, weighting value α, default frame c, default frame position information l and calibration area result g.

5. The target detection method based on picture recognition according to any one of claims 1-4, wherein the feedback step further comprises:

and if the analysis result is not true, performing histogram equalization processing on the image to be detected to obtain second image data, adjusting the second image data to a preset angle, and inputting the second image data into the image feature extraction model in the receiving step again.

6. A server, comprising a memory and a processor, wherein the memory stores a picture recognition-based object detection program, and the picture recognition-based object detection program when executed by the processor implements the steps of:

a receiving step: receiving an image to be detected uploaded by a client, inputting the image to be detected into a pre-trained image data feature extraction model, and obtaining an image feature map corresponding to the image to be detected;

a first judgment step: inputting the obtained image feature map into a pre-trained target extraction model, outputting first image data, and judging whether the first image data contains first target frames of at least two first preset target types, wherein the number of the first target frames of each preset type is a first preset number;

a second judgment step: if the first image data comprises first target frames of at least two first preset target types and the number of the first target frames of each preset type is a first preset number, respectively acquiring the position coordinates of the first target frames, and judging the position relation between the first target frames based on a preset calculation rule;

a third judging step: according to the position relation, finding out a corresponding judgment result from a mapping relation table between the position relation and the judgment result which are pre-established in a database, wherein the judgment result comprises a first judgment result and a second judgment result, the first judgment result shows that the authenticity of the information corresponding to the first image data is to be determined, and the second judgment result shows that the authenticity of the information corresponding to the first image data is not true; and

a feedback step: if the judgment result is a first judgment result, respectively judging whether the first target frame comprises a second target frame, if so, respectively identifying first preset data corresponding to the second target frame, analyzing whether the first preset data correspond to second preset data stored in a database in advance, and generating feedback information according to the analysis result to feed back the feedback information to the client, wherein the analysis result comprises an actual fact and a non-actual fact.

7. The server according to claim 6, wherein the object extraction model is an SSD model, and the determining whether the first image data includes a first object box of at least two first preset object types includes:

respectively generating corresponding default frames for each pixel point in the image feature map based on the SSD model, acquiring position coordinates of each default frame in the image feature map and probability scores corresponding to different first preset target types, and setting the maximum value in the probability scores of each default frame as a primary confidence coefficient;

sorting the default frames corresponding to the primary confidence degrees from large to small according to probability scores, sequentially obtaining a second preset number of default frames as target candidate frames by taking the default frame corresponding to the maximum value of the probability scores as a starting point, and performing bounding box regression analysis based on the position coordinates of each target candidate frame to obtain the area size corresponding to each target candidate frame;

performing softxmax classification on the probability score of each target candidate frame to obtain target confidence coefficients of each target candidate frame corresponding to different preset target type classifications; and

and acquiring a third preset number of target candidate frames with iou (M, b) higher than a preset threshold as the first target frame based on a non-maximum suppression algorithm, wherein M represents a default frame corresponding to the maximum probability score, b represents other default frames except the default frame M in the image feature map, and iou (M, b) represents the overlapping degree between the default frame M and the default frame b.

8. The server of claim 7, wherein the training process of the target extraction model comprises:

acquiring an image feature map sample, respectively generating corresponding default frame samples for each pixel point in the image feature map sample based on the target extraction model, and acquiring the coordinate position of each default frame sample in the image feature map sample and probability scores corresponding to different first preset target types;

respectively calculating the sum of softmax classification loss and bounding box regression loss of each default frame sample based on the position coordinate and probability score of each default frame sample; and

sequencing the sum of the softmax classification loss and the bounding box regression loss from large to small, sequentially acquiring a preset number of default frame samples by taking the default frame sample corresponding to the minimum softmax classification loss and the bounding box regression loss as a starting point, calculating the loss functions of the default frame samples of the preset number, performing back propagation on the calculated loss functions of the default frame samples of the preset number in the target extraction model, updating the weight values of each layer of network of the target extraction model, and training to obtain the target extraction model.

9. The server of claim 8, wherein the loss function is calculated by the formula:

wherein, L_conf(x, c) Softmax classification loss, L_loc(x, l, g) is bounding box regression loss, K ═ f_k|*|f_k|*α，|f_kThe method comprises the steps of |, weighting value α, default frame c, default frame position information l and calibration area result g.

10. A computer-readable storage medium, wherein a picture recognition-based object detection program is stored on the computer-readable storage medium, and the picture recognition-based object detection program is executable by one or more processors to implement the steps of the picture recognition-based object detection method according to any one of claims 1 to 5.

Images