CN111046886A

CN111046886A - Number plate automatic identification method, device, device and computer readable storage medium

Info

Publication number: CN111046886A
Application number: CN201911276472.9A
Authority: CN
Inventors: 王欣; 杨竣翔
Original assignee: Jilin University
Current assignee: Jilin University
Priority date: 2019-12-12
Filing date: 2019-12-12
Publication date: 2020-04-21
Anticipated expiration: 2039-12-12
Also published as: CN111046886B

Abstract

The present application discloses a method, device, device and computer-readable storage medium for automatic identification of a number plate. Wherein, the method includes extracting the athlete target area from the to-be-processed image containing at least one athlete block by using a target detection model whose detection target is a person, and then extracting the text area from the target area by using the text detection model, and based on the text recognition model for the text area The Chinese character part is identified to obtain the initial number information. Finally, the tree filter algorithm is used to filter the initial number information to obtain a set of number values corresponding to each athlete in the image; the text detection model is based on the CTPN algorithm, using the training sample set training loop It is obtained from the end-to-end network model combining neural network and convolutional neural network; the text recognition model is obtained by training the convolutional recurrent network model using the number plate training sample set. The application can efficiently and accurately identify the number plate of the athlete, which is beneficial to improve the recognition accuracy of the photos of the marathon runners.

Description

Automatic identification method, device and equipment for number plate and computer readable storage medium

Technical Field

The present application relates to the field of deep learning based target detection and identification technologies, and in particular, to a method and an apparatus for automatically identifying a number plate, and a computer-readable storage medium.

Background

In recent years, the number of marathon events in the world has increased in a blowout manner, and only 10 months in 2018 have led to 81 marathon events in china. Due to its low threshold, light equipment and no limitations in place, time, climate, etc., many professional players as well as amateurs are attracted to participate, which presents a great challenge to the service and dissemination of the event. Accurately sorting and pushing photographs taken for athletes during an event is a challenging event service task.

The number of participants in the 32 nd Hangzhou marathon which is held in 11 months in 2018 reaches 35000; the number of 2018 New York marathon participants, which were held in New York, U.S. at the same time, reached 50000. Taking the Hangzhou Malason as an example, if the host takes more than 4 photos for each participant, the number of photos will exceed 140000. At present, most marathon games are formed by manually classifying shot pictures and then pushing the pictures for athletes. This is inefficient and costly and requires the athlete to obtain their own picture of the event several days after the event is completed. In addition, the marathon race is long, and if the athlete asks the photographer to take a full shot alone, it also costs a high amount. Therefore, an efficient and accurate image recognition method is needed to classify and push a large number of photos.

The identification and classification of the captured photographs by identifying player number plates is one of the methods commonly used in some marathon games today. This approach is largely divided into two categories. One is to detect and identify the player number plate by using a conventional scene character recognition detection. The other type is detection and identification of the number plate of the player based on a deep learning method.

The traditional scene character recognition detection method firstly needs to detect a human body, and usually adopts algorithms such as DPM (differential pulse width modulation), most of the algorithms firstly calculate a gradient direction histogram, then use SVM (support vector machine) training to obtain a gradient model of an object, and use the model to recognize and classify the human body; then, the number plate of the athlete needs to be positioned, and a method of k-means clustering (k-means clustering), AdaBoost algorithm, joint probability and the like are generally used, wherein the method basically trains a classifier, and the number plate is positioned through the classifier; after positioning, the characters on the number plate are segmented by using a character segmentation Method, mainly using a Projection Method (Projection Method), a connected region analysis Method and other methods; finally, the segmented characters are identified by using an OCR (optical Character recognition) technology, template matching and other methods. Although the traditional scene character recognition detection method is superior to a manual classification method in speed, the traditional scene character recognition detection method is easily influenced by factors such as illumination, number plate distortion and the like, the problems of high calculation complexity, low precision and the like exist, the classification effect is poor, and a sportsman may receive more pictures which do not belong to the traditional scene character recognition detection method when pushing the pictures.

The current deep learning-based method is superior to the traditional computer vision technology in many aspects, but has a plurality of disadvantages: in the scene character detection method, TextBoxes are a text detection model in a natural scene based on SSD (Single Shot MultiBoxDetector), and the model directly detects a word as an object. However, since the model uses the default box, the detection effect is not good when problems such as the text area is too long are encountered. In addition, because the photos shot in the marathon competition process have the problems of weather, light and the like, the characters of the number plate part can have the problems of light reflection, blurring and the like, and the model cannot be effectively detected. For example, the phoocr method is to detect a single character first, and then recognize the detected character using a deep convolutional neural network. However, this method needs to be able to accurately detect and crop each character to accurately recognize the text in the image. Another related art method needs to consider scene text recognition as an image classification problem, i.e., each word is assigned a class label. This makes the number of classes classified by the model very large, resulting in a problem of poor generalization of the model. Furthermore, the more popular depth models like DCNN require operations on inputs and outputs with fixed dimensions and therefore cannot be directly applied to sequence prediction.

In view of this, how to efficiently and accurately identify the number plate of the player for identifying and classifying the marathon player photos is a technical problem to be solved by those skilled in the art.

Disclosure of Invention

The application provides a number plate automatic identification method, a number plate automatic identification device and a computer readable storage medium, which can efficiently and accurately identify the number plate of a sportsman and are beneficial to improving the identification accuracy of the photos of the marathon sportsman.

In order to solve the above technical problems, embodiments of the present invention provide the following technical solutions:

the embodiment of the invention provides an automatic identification method of a number plate on one hand, which comprises the following steps:

extracting a target area from an image to be processed by utilizing a pre-constructed target detection model, wherein the target area comprises at least one athlete image block;

extracting a character area from the target area through a pre-constructed character detection model;

recognizing the character part in the character area based on a pre-constructed character recognition model to obtain initial number information;

screening the initial number information by using a tree filtering algorithm to obtain at least one group of number values as the number plate identification result of the athlete contained in the image to be processed; the number of the number value groups is consistent with the number of the player patterns contained in the target area;

the detection target of the target detection model is person, the character detection model is obtained by training an end-to-end network model by utilizing a training sample set based on a CTPN algorithm, and the end-to-end network model is a training model generated by combining a cyclic neural network and a convolutional neural network; the character recognition model is obtained by training a convolution cyclic neural network model by using a number plate training sample set.

Optionally, the target detection model is generated by training a YOLOv3 network model with a training sample set, and a detection target of the YOLOv3 network model is person.

Optionally, after extracting the target region from the image to be processed by using the pre-constructed target detection model, the method further includes:

and carrying out normalization processing on the target area according to a preset first image standard size value so as to keep the size of the target area after normalization processing consistent with the first image standard size value.

Optionally, the first image standard size value is 150 × 371.

Optionally, after extracting the text region from the target region through the pre-constructed text detection model, the method further includes:

and carrying out normalization processing on the character area according to a preset second image standard size value so as to keep the size of the character area after normalization processing consistent with the second image standard size value.

Optionally, the second image standard size value is 280 × 83.

Optionally, the similarity threshold of the target detection model is 98.7, and the target region extracted by the target detection model contains at most 4 player segments.

Another aspect of the embodiments of the present invention provides an automatic number plate recognition apparatus, including:

the target extraction module is used for extracting a target area from the image to be processed by utilizing a pre-constructed target detection model, and the target area comprises at least one athlete image block;

the character detection module is used for extracting a character area from the target area through a pre-constructed character detection model; the character detection model is obtained by training an end-to-end network model by utilizing a training sample set based on a CTPN algorithm, and the end-to-end network model is a training model generated by combining a cyclic neural network and a convolutional neural network;

the character recognition module is used for recognizing the character part in the character area based on a pre-constructed character recognition model to obtain initial number information; the character recognition model is obtained by training a convolution cyclic neural network model by using a number plate training sample set;

the result screening module is used for screening the initial number information by utilizing a tree-shaped filtering algorithm to obtain at least one group of number values as the number plate identification result of the athlete contained in the image to be processed; the number of the number value groups is consistent with the number of the player patterns contained in the target area.

The embodiment of the invention also provides a number plate automatic identification device, which comprises a processor, wherein the processor is used for realizing the steps of the number plate automatic identification method when executing the computer program stored in the memory.

Finally, an embodiment of the present invention provides a computer-readable storage medium, where a number plate automatic identification program is stored on the computer-readable storage medium, and when the number plate automatic identification program is executed by a processor, the steps of the number plate automatic identification method are implemented as in any one of the foregoing.

The technical scheme provided by the application has the advantages that the athlete image blocks of each picture are taken as the target area for extraction, so that the data volume of subsequent image processing can be reduced, the interference of irrelevant image blocks can be eliminated, and the accuracy of subsequent model identification can be improved; the image blocks of the screened athletes are subjected to character recognition based on the CTPN algorithm, characters existing in the external environment are effectively eliminated, the character recognition efficiency is greatly improved, and the difficulty in screening the final number plate is reduced; the CRNN is adopted to carry out transfer learning aiming at the problem which needs to be solved actually, and the relevant characters can be identified quickly and efficiently; because other character parts may exist on the sportsman, in order to eliminate interference, the tree-shaped filtering algorithm is used for screening the initial recognition result, and finally the character part of the sportsman number plate with high accuracy can be obtained, so that the sportsman number plate can be efficiently and accurately recognized, and the accuracy rate of recognizing the photos of the marathon sportsman can be improved.

In addition, the embodiment of the invention also provides a corresponding implementation device and a computer readable storage medium for the number plate automatic identification method, so that the method has higher practicability, and the device and the computer readable storage medium have corresponding advantages.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the related art, the drawings required to be used in the description of the embodiments or the related art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a schematic flow chart of an automatic number plate identification method according to an embodiment of the present invention;

fig. 2 is a schematic diagram of statistical results of the collected images including the number of athletes and the images meeting the requirements according to the embodiment of the present invention;

FIG. 3 is a diagram illustrating a loss value statistical result of the character recognition model according to the embodiment of the present invention;

fig. 4 is a structural diagram of an embodiment of an automatic identification device for a number plate according to an embodiment of the present invention;

fig. 5 is a structural diagram of another specific embodiment of the automatic number plate recognition apparatus according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The terms "first," "second," "third," "fourth," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may include other steps or elements not expressly listed.

Having described the technical solutions of the embodiments of the present invention, various non-limiting embodiments of the present application are described in detail below.

Referring to fig. 1, fig. 1 is a schematic flow chart of an automatic number plate identification method according to an embodiment of the present invention, where the embodiment of the present invention includes the following:

s101: and extracting a target region from the image to be processed by utilizing a pre-constructed target detection model.

In this application, at least one player in the image to be processed, i.e. the application is directed to the identification of a plurality of players in the same image. The detection target of the target detection model is person, and is used for identifying a portrait in an image and extracting an identified image block, and any image identification algorithm and image block extraction algorithm capable of identifying the target can be adopted, which is not limited in this application. And inputting the image to be processed into a target detection model, wherein the output of the target detection model is the athlete image block for extracting the image to be processed, and if the image to be processed comprises a plurality of athletes, the target area correspondingly comprises at least one athlete image block.

S102: and extracting a character area from the target area through a pre-constructed character detection model.

It can be understood that, in the process of identifying the characters on the number plate of the player, the characters on the number plate can be interfered by light, distortion and the like. When the traditional method is used for detection, firstly, the number plate needs to be positioned, then, the number plate area is subjected to binarization processing, and finally, the number plate area is segmented through a reasonable segmentation method. The detection process of the method is complex, and if the problems of illumination, distortion and the like are encountered, accurate detection cannot be realized. In order to solve the problems, an end-to-end character detection model based on deep learning can be adopted, the character detection model which can be adopted in the application is obtained by training an end-to-end network model by utilizing a training sample set based on a CTPN algorithm, and the end-to-end network model is a training model generated by combining a cyclic neural network and a convolutional neural network. The CTPN (detecting Text in Natural Image with connecting Text property network) algorithm combines a cyclic neural network and a convolutional neural network to form an end-to-end trainable model. The network introduces a vertical anchor point mechanism and provides a network inner loop mechanism for connecting a sequence text proposal in convolution feature mapping, which can accurately position text lines in natural images, including extremely fuzzy texts in the images. Characters interfered by illumination, distortion and the like in a natural scene can be effectively detected. Because the CTPN algorithm can accurately detect all characters in the image, except the characters in the number plate, the characters in other areas can be detected. The problem that the detection quantity of the model is large and the overall efficiency of the model is low is caused. The designed preprocessing module is used for extracting the athletes in the image, so that the CTPN only carries out character detection on the athletes in the image, and the detection amount is effectively reduced.

S103: and identifying the character part in the character area based on a pre-constructed character identification model to obtain initial number information.

In the identification of marathon player number cards, the number on the number cards may be different in length in different marathon matches, and the identification may be affected by factors such as illumination, shading, and distortion. So that the general character recognition method is difficult to have higher accuracy in the problem. The character recognition model can be obtained by training a convolution cyclic neural network model by using a number plate training sample set.

CRNN (Convolutional Recurrent Network) is a currently excellent general framework for character recognition, and can be applied to various fields and problems related to image sequence prediction. The conditional probabilities defined in the Connection Time Classification (CTC) layer may be used in CRNN. The conditional probability is defined as being composed of

The sum of the probabilities of all π mapped onto l, can be expressed as:

where the probability of π is defined as:

where L is given as a tag sequence, y is an input sequence, and L 'is a character set LU { blank }, and L'^TThe elements in (a) are considered paths and are denoted by pi,

there is a label pi at time t_tThe probability of (c). CTC methods support predicting tags directly on an undivided sequence, thereby enabling CRNN to run directly on coarse-grained tags (e.g., words), without the need to detail each individual element (e.g., character) during the training phase.

S104: and screening the initial number information by using a tree filtering algorithm to obtain at least one group of number values as the number plate identification result of the athlete contained in the image to be processed.

It is understood that the character detection model CRNN used in S103 recognizes all characters in the image, and even though the detection amount of characters is reduced by the preprocessing module, characters may exist in the clothes worn by the athlete, which may result in a plurality of recognition results, and the photos cannot be classified correctly. That is, the initial number information may include some english letters, number information and text information on a non-number plate, and in order to obtain accurate number plate information, the initial number information may be filtered, the non-number letters or text may be detected and deleted, and the number information on the non-number plate may be removed based on the attribute information of the number plate, where the attribute information includes, for example, a number plate with 4 digits, and if the initial number information includes a set of 3-digit or 5-digit numerical information, the set of numerical information does not necessarily include information on the number plate. In addition, if the number of number value groups matches the number of player tiles included in the target area, that is, if the target area in S101 includes 4 players, the number plate recognition result output in S104 is 4 groups of number information.

The tree filtering algorithm takes the input image as a root, and each identified athlete is taken as a child node. The text identified for each player serves as a child node. And counting the number of the same rules of the sub-nodes, taking the most number as a number plate identification result, and deleting other nodes. Based on this, the flow of the tree filtering algorithm of the present application may be as follows:

in the technical scheme provided by the embodiment of the invention, the athlete image block of each picture is taken as the target area for extraction, so that the data volume of subsequent image processing can be reduced, the interference of irrelevant image blocks can be eliminated, and the accuracy of subsequent model identification can be improved; the image blocks of the screened athletes are subjected to character recognition based on the CTPN algorithm, characters existing in the external environment are effectively eliminated, the character recognition efficiency is greatly improved, and the difficulty in screening the final number plate is reduced; the CRNN is adopted to carry out transfer learning aiming at the problem which needs to be solved actually, and the relevant characters can be identified quickly and efficiently; because other character parts may exist on the sportsman, in order to eliminate interference, the tree-shaped filtering algorithm is used for screening the initial recognition result, and finally the character part of the sportsman number plate with high accuracy can be obtained, so that the sportsman number plate can be efficiently and accurately recognized, and the accuracy rate of recognizing the photos of the marathon sportsman can be improved.

As an alternative embodiment, the target detection model may be generated by training a YOLOv3 network model with a training sample set, and the detection target of the YOLOv3 network model is person. To improve the accuracy and efficiency of the target detection model, the similarity threshold of the target detection model may be set to 98.7, and the target detection model extracts at most 4 player patches in the target region. The YOLOv3 basic network structure is Darknet-53, full convolution is adopted, a residual error structure is introduced, multi-scale prediction can be carried out, and each scale predicts three boxes.

In order to improve the processing speed of subsequent images, in S101, after the athlete subgraph meeting the requirements in the image is extracted, there may be a problem of different image sizes, and in order to facilitate the subsequent processing and detection of the model, the extracted athlete subgraph may be normalized, for example, the target area may be normalized according to a preset first image standard size value, so that the size of the target area after the normalization processing is consistent with the first image standard size value, for example, the sizes of the extracted athlete subgraphs may be unified to 150 × 371, that is, the first image standard size value is 150 × 371.

As another alternative, after extracting relevant characters on the player through CTPN, a normalization process needs to be performed on the extracted result for the convenience of subsequent character recognition and other processes. The text area can be normalized according to a preset second image standard size value, so that the size of the text area after the normalization processing is consistent with the second image standard size value. For example, the size of the extracted text image may be unified to 280 × 83, that is, the standard size value of the second image is 280 × 83.

In order to verify the effectiveness of the technical scheme of the application, the application also provides a verification experiment, which can comprise the following contents:

aiming at the marathon athlete number plate recognition model designed by the invention, a certain amount of data is collected to carry out experiments on the marathon athlete number plate recognition model. 3000 photographs of players of different sessions of marathon games, including photographs at the end and during the game, were taken as the original data set MARD of the model. In addition, 1000 photographs were randomly selected from the original data set, divided into 800 and 200 two sets of figures, each player's number plate was manually cut, and the cut figures were scaled to 280 × 32. These two sets of drawings are made as a number plate dataset MANUMD and as a training set and a test set of text recognition models.

In the preprocessing module, two thresholds are determined experimentally: a similarity threshold and a detection threshold.

Similarity threshold: if the similarity threshold is set too low, workers and audiences with certain shelters at two sides of the runway can be extracted as subsequent detection targets; if the similarity threshold is set too high, some players may not be detected. Therefore, the application sets the similarity threshold to 98.7 through actual test statistics. And extracting the target for subsequent detection and identification only when the similarity of the target is higher than 98.7. Although the threshold value can cause that a few athletes with shielding and the like cannot be detected, audiences and staff on two sides of the runway can be effectively removed.

Detection threshold value: (1) 3000 pictures of athletes taken during different marathon competitions were analyzed by mathematical statistics. Firstly, 3000 photos are randomly sequenced and then numbered; then, 300 samples were taken therefrom, by the formula

The segmentation interval k of the samples is calculated, N is the total number of samples, e.g. 3000, N is the total number of samples, e.g. 300, and k is 10.

The number 6 image is extracted from the first group by random sampling, and the numbers 16, 26, …, 2986 and 2996 are sequentially extracted, so as to obtain a sample with the sample capacity of 300. According to actual requirements, the classified photos need to be pushed to corresponding athletes, so that the athletes are main bodies in the photos and do not have serious shielding. Provision is made here for: the athlete needs to focus clearly and without occlusion in the picture. The athlete is detected when this condition is met. Through counting the 300 samples extracted, the proportion of the photos which have the number of the athletes more than 4 and meet the regulation in each photo is found to be less than 1%. The statistical analysis results are shown in fig. 2, the abscissa is the number of athletes included in the image, and the ordinate is the number of images satisfying the requirements. Therefore, according to the data and the actual demand obtained after the analysis, the detection threshold value is set to be 4, namely, 4 people are extracted at most to detect and identify the number plate, so that the model is more in line with the actual demand.

According to the practical problem to be solved, the detection target of YOLOv3 is set to person, i.e. only human is detected. The CPU used in the experiment was Intel (R) core (TM) i7-6700k, the GPU was NVIDIA GeForce GTX 1080, and the memory was 16G. The character recognition model is first trained on a character data set (containing chinese, english, numbers) at 3600000. After training on the character data set, the recognition accuracy of the character recognition model reaches 98.37%. And then, the pre-trained character recognition model is subjected to transfer learning by using the number plate data set manufactured by the application. Fig. 3 shows a result diagram of loss values obtained by performing fine tuning on the character recognition model and iterating 300 times, where the abscissa is the number of iterations and the ordinate is the loss value, and the accuracy of the final model on the test set is 0.935.

Comparative experiments with SSD and YOLOv3 using raw data sets collected from marathon athletes found: (1) on this data set, the SSD detected the athlete with a precision of 96.89%; the detection accuracy of YOLOv3 for the athlete was 95.69%. Both SSD and YOLOv3 are capable of more accurately detecting athletes in images. (2) Both take 49ms for SSD and 21ms for YOLOv3 when detecting the same image in the dataset. Therefore, YOLOv3 is 2 times faster than SSD in detection speed under the same conditions.

The comparison experiment of the number plate data set obtained by cutting from the original data set is used for finding that: tesseract can not accurately identify the number plate influenced by illumination, distortion, fuzziness and the like, can accurately identify the number plate not influenced by the conditions, and can better handle various conditions based on the character identification model of CRNN. Under a natural scene, the number plate is almost affected by factors such as illumination, distortion and the like, so that the recognition accuracy of Tesseract on the number plate data set obtained by cutting is only 0.15, and the recognition accuracy of the character recognition model after transfer learning on the data set is 0.935.

The embodiment of the invention also provides a corresponding device for the automatic number plate identification method, so that the method has higher practicability. Wherein the means can be described separately from the functional module point of view and the hardware point of view. In the following, the automatic number plate recognition device according to the embodiment of the present invention is introduced, and the automatic number plate recognition device described below and the automatic number plate recognition method described above may be referred to each other.

Based on the angle of the function module, referring to fig. 4, fig. 4 is a structural diagram of an automatic identification device for a number plate according to an embodiment of the present invention, in a specific implementation manner, the device may include:

and the target extraction module 401 is configured to extract a target region from the image to be processed by using a pre-constructed target detection model in which a detection target is person, where the target region includes at least one athlete segment.

A text detection module 402, configured to extract a text region from the target region through a pre-constructed text detection model; the character detection model is obtained by training an end-to-end network model by utilizing a training sample set based on a CTPN algorithm, and the end-to-end network model is a training model generated by combining a cyclic neural network and a convolutional neural network.

A character recognition module 403, configured to recognize a character part in a character region based on a pre-constructed character recognition model, so as to obtain initial number information; the character recognition model is obtained by training a convolution cyclic neural network model by using a number plate training sample set.

A result screening module 404, configured to screen the initial number information by using a tree filtering algorithm to obtain at least one group of number values, where the group of number values is used as a number plate identification result of the athlete included in the image to be processed; the number of code value groups is consistent with the number of player segments contained in the target zone.

Optionally, in some embodiments of this embodiment, the apparatus may further include a normalization processing module, for example, the normalization processing module is configured to perform normalization processing on the target area according to a preset first image standard size value, so that the size of the target area after the normalization processing is consistent with the first image standard size value; and carrying out normalization processing on the character area according to a preset second image standard size value so as to keep the size of the character area after the normalization processing consistent with the second image standard size value.

The functions of the functional modules of the automatic number plate identification device according to the embodiments of the present invention can be specifically implemented according to the method in the above method embodiments, and the specific implementation process may refer to the related description of the above method embodiments, which is not described herein again.

Therefore, the embodiment of the invention can efficiently and accurately identify the number plate of the athlete, and is beneficial to improving the identification accuracy of the photos of the marathon athlete.

The above-mentioned automatic identification device for number plate is described from the perspective of functional module, and further, the present application also provides an automatic identification device for number plate, which is described from the perspective of hardware. Fig. 5 is a structural diagram of another automatic number plate recognition device according to an embodiment of the present application. As shown in fig. 5, the apparatus comprises a memory 50 for storing a computer program;

a processor 51 for implementing the steps of the automatic number plate identification method as mentioned in the above embodiments when executing the computer program.

The processor 51 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 51 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 51 may also include a main processor and a coprocessor, the main processor is a processor for processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 51 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 51 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 50 may include one or more computer-readable storage media, which may be non-transitory. Memory 50 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 50 is at least used for storing the following computer program 501, wherein after being loaded and executed by the processor 51, the computer program can implement the relevant steps of the testing method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 50 may also include an operating system 502, data 503, and the like, and the storage manner may be a transient storage manner or a permanent storage manner. Operating system 502 may include Windows, Unix, Linux, etc. Data 203 may include, but is not limited to, data corresponding to test results, and the like.

In some embodiments, the automatic number plate recognition device may further include a display screen 52, an input/output interface 53, a communication interface 54, a power source 55, and a communication bus 56.

Those skilled in the art will appreciate that the configuration shown in figure 5 does not constitute a limitation of the automatic number plate identification device and may include more or fewer components than those shown.

It is to be understood that, if the number plate automatic identification method in the above embodiments is implemented in the form of a software functional unit and sold or used as a separate product, it may be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application may be substantially or partially implemented in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the methods of the embodiments of the present application, or all or part of the technical solutions. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a CD-ROM, a magnetic or optical disk, and other various media capable of storing program codes.

Based on this, the embodiment of the present invention further provides a computer-readable storage medium, in which a number plate automatic identification program is stored, and the number plate automatic identification program is executed by a processor, according to the steps of the number plate automatic identification method according to any one of the above embodiments.

The functions of the functional modules of the computer-readable storage medium according to the embodiment of the present invention may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the related description of the foregoing method embodiment, which is not described herein again.

The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The above description details an automatic number plate recognition method, device and computer-readable storage medium provided by the present application. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present application.

Claims

1. a number plate automatic identification method, is characterized in that, comprises:

Extracting a target area from the image to be processed using a pre-built target detection model, the target area containing at least one athlete tile;

Extract a text area from the target area by using a pre-built text detection model;

Recognizing the text portion in the text area based on a pre-built text recognition model to obtain initial number information;

Utilize tree filter algorithm to carry out screening to described initial number information, obtain at least one group of number value, as described image to be processed contains the number plate recognition result of athlete; The number of blocks remains the same;

Wherein, the detection target of the target detection model is person, and the text detection model is obtained by using the training sample set to train an end-to-end network model based on the CTPN algorithm, and the end-to-end network model is a recurrent neural network and a convolutional neural network. The training model generated by the combination of the network; the character recognition model is obtained by training the convolutional recurrent neural network model with the number plate training sample set.

2. number plate automatic identification method according to claim 1, is characterized in that, described target detection model is to utilize training sample set to train YOLOv3 network model to generate, and the detection target of described YOLOv3 network model is person.

3. number plate automatic identification method according to claim 2, is characterized in that, after described utilizing pre-built target detection model to extract target area from the image to be processed, also comprises:

The target area is normalized according to the preset first image standard size value, so that the size of the normalized target area is consistent with the first image standard size value.

4 . The method of claim 3 , wherein the standard size of the first image is 150×371. 5 .

5. The method for automatic identification of number plates according to claim 4, characterized in that, after the text region is extracted from the target region by a text detection model constructed in advance, the method further comprises:

The text area is normalized according to the preset second image standard size value, so that the size of the normalized text area is consistent with the second image standard size value.

6 . The automatic identification method for a number plate according to claim 4 , wherein the standard size value of the second image is 280×83. 7 .

7. The number plate automatic identification method according to any one of claims 1 to 6, wherein the similarity threshold of the target detection model is 98.7, and the target area extracted by the target detection model contains at most 4 athlete tiles.

8. A number plate automatic identification device, characterized in that, comprising:

A target extraction module for extracting a target area from the image to be processed by utilizing a pre-built target detection model, the target area comprising at least one player image block;

A text detection module for extracting text regions from the target region through a pre-built text detection model; the text detection model is obtained by training an end-to-end network model based on the CTPN algorithm using a training sample set, and the end-to-end network The model is a training model generated by the combination of recurrent neural network and convolutional neural network;

a character recognition module, used for recognizing the character part in the character area based on a pre-built character recognition model to obtain initial number information; the character recognition model is obtained by training a convolutional cyclic neural network model with a number plate training sample set;

The result screening module is used to screen the initial number information using a tree filtering algorithm to obtain at least one group of number values, as the image to be processed contains the number plate recognition result of the athlete; The target area contains the same number of player tiles.

9. a number plate automatic identification device is characterized in that, comprises a processor, when the processor is used to execute the computer program stored in the memory, realizes the steps of the number plate automatic identification method as described in any one of claims 1 to 7 .

10. A computer-readable storage medium, characterized in that, a number plate automatic identification program is stored on the computer-readable storage medium, and when the number plate automatic identification program is executed by the processor, any one of claims 1 to 7 is realized. A step of the automatic identification method of the number plate.