[go: up one dir, main page]

CN109711407A - A kind of method and relevant apparatus of Car license recognition - Google Patents

A kind of method and relevant apparatus of Car license recognition Download PDF

Info

Publication number
CN109711407A
CN109711407A CN201811626428.1A CN201811626428A CN109711407A CN 109711407 A CN109711407 A CN 109711407A CN 201811626428 A CN201811626428 A CN 201811626428A CN 109711407 A CN109711407 A CN 109711407A
Authority
CN
China
Prior art keywords
board information
license
license board
license plate
tracking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811626428.1A
Other languages
Chinese (zh)
Other versions
CN109711407B (en
Inventor
李锐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Jieshun Science and Technology Industry Co Ltd
Original Assignee
Shenzhen Jieshun Science and Technology Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Jieshun Science and Technology Industry Co Ltd filed Critical Shenzhen Jieshun Science and Technology Industry Co Ltd
Priority to CN201811626428.1A priority Critical patent/CN109711407B/en
Publication of CN109711407A publication Critical patent/CN109711407A/en
Application granted granted Critical
Publication of CN109711407B publication Critical patent/CN109711407B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Image Analysis (AREA)

Abstract

The embodiment of the present application discloses the method and relevant apparatus of a kind of Car license recognition, for reducing interference of the color for Car license recognition, accelerates the speed of Car license recognition.The embodiment of the present application includes: the real-time video flowing for obtaining parking lot entrance;Image in video flowing is converted into gray level image;Obtain tracking list;Whether detecting and tracking list is empty;If not empty, then the first license board information in tracking list is tracked;If tracking the confidence level for successfully detecting the first license board information;If confidence level is up to standard, the first license board information in tracking list is updated;If it is empty, then it whether there is the second license board information using vehicle plate location model detection video flowing is interior;If detecting the success of the second license board information, the confidence level of the second license board information is detected;If confidence level is up to standard, the second license board information is added to tracking list.The application is converted to gray level image by the image for taking video flowing, reduces interference of the color for Car license recognition, increases Car license recognition speed.

Description

A kind of method and relevant apparatus of Car license recognition
Technical field
This application involves field of image processings, and in particular to a kind of method and relevant apparatus of Car license recognition.
Background technique
In recent years, license plate recognition technology is quickly grown at home, and the mode that pure Car license recognition is passed in and out as parking lot is It is universal in China.However, overseas countries or the license plate recognition technology in area fall behind relatively, Car license recognition is using less at present.With Domestic license plate recognition technology increasingly mature and promote, license plate recognition technology is recognized in more and more countries and regions It is convenient, it is desirable to introduce license plate recognition technology.Domestic many producers take to overseas Car license recognition research and development.However, overseas license plate Identification technology, especially on License Plate, there are certain difficulty.
The license plate of overseas many countries includes single layer and bilayer.The ratio of width to height of the overseas license plate of single layer is single compared with domestic Layer license plate is bigger, i.e., the overseas license plate of single layer can be wider more " short ".In addition the ratio of width to height of the double-deck overseas license plate is double-deck compared with domestic License plate is smaller, i.e., overseas license plate can be narrower higher.For this means that overseas license plate compared with the country, the range meeting of the ratio of width to height It is bigger.Overseas some areas, license plate size are not fixed, i.e., big vehicle uses biggish license plate, and small vehicle use is compared with trolley Board, and there is also differences for license plate the ratio of width to height of two kinds of models.Will cause same type license plate in this way, there are more sizes, Duo Kuangao Compare the case where.The license plate of overseas many countries and regions is all oneself application license plate number, is then made again by garage, thus There can be the license plate of various material, different materials will lead to reflective inconsistent situation.Also, there is also a variety of for overseas license plate There is black in color, such as Macao, red, green license plate.In addition, font on license plate is also there are many color, for example, Macao have it is white Color, red, black and yellow font etc..Even there are also background patterns on Countries and the license plate in area, these can all give vehicle Board positioning belt carrys out very big difficulty.
Current video camera most on the market is all yuv format output, when executing Detection of License, in order to The component information for obtaining tri- channels R, G and B, after receiving video camera and being transmitted through the image come, system has to execute one The conversion of YUV to RGB.If picture size is big, this conversion can be very time-consuming, leads to the increase of recognition time, it is possible to shadow Ring the real-time of Vehicle License Plate Recognition System.
Apply for content
The embodiment of the present application provides a kind of method of Car license recognition, for realizing color is reduced for the dry of Car license recognition It disturbs, accelerates the speed of Car license recognition.
In order to achieve the above objectives, the application first aspect provides a kind of method of Car license recognition, and this method may include:
The video flowing of parking lot entrance is obtained in real time, license board information is contained in the video flowing, and the license board information is used List is tracked in generating;
Image in video flowing is converted into gray level image;
Obtain tracking list;
Detect whether the tracking list is empty;
If the tracking list is not sky, the first license board information in the tracking list is tracked;
If tracking the confidence level for successfully detecting first license board information;
If the confidence level of first license board information is up to standard, first license plate letter in the tracking list is updated Breath;
If list is empty for the tracking, believe using in vehicle plate location model detection video flowing with the presence or absence of the second license plate Breath;
If detecting the second license board information success, the confidence level of second license board information is detected;
If the confidence level of second license board information is up to standard, second license board information is added to the tracking and is arranged Table.
Optionally, with reference to the above first aspect, in the first possible implementation, it is in the detection video flowing It is no there are before the second license board information, the method also includes:
Training vehicle plate location model, the vehicle plate location model are used for car plate detection.
Optionally, shoot the video of parking lot entrance perhaps the image video or image include license plate area with Background area, generates a trained pictures, and the trained pictures include that the picture intercepted from video is obtained with field device The picture taken;
Mark the license plate area for including in the trained pictures;
N1 positive sample, n2 part license plate sample, n3 negative sample, institute are generated from the trained pictures using program Stating positive sample is with the overlapping region of tab area in the first preset ratio section, and the part license plate sample is and marked area In the second preset ratio section, the negative sample is with the overlapping region of tab area in the default ratio of third for the overlapping region in domain In example section;
Using the positive sample, the part license plate sample and negative sample training MTCNN multitask convolutional Neural net The Pnet detection model of network;
Training pictures are detected using the Pnet detection model, to obtain Pnet void inspection picture, the Pnet void inspection is Pnet detection model testing result is license plate, but is less than the detection knot of preset threshold with the lap of the tab area Fruit;
Using the positive sample, the part license plate sample and the Pnet void examine picture training multitask convolutional Neural net The Rnet detection model of network (MTCNN).
Optionally, generate input picture pyramid, the input picture is the video flowing obtained in real time, the pyramid by The scaled obtained a series of pictures of the image of each frame in video flowing;
Full figure detection, output shot chart and regressand value figure are carried out using the Pnet;
The shot chart that X score is greater than preset fraction is chosen, it is non-greatly to there are the progress of the shot chart of overlapping region Value inhibits;
If detection terminates without candidate frame, the result without license plate is exported;
The candidate frame if it exists is then adjusted the candidate frame using the regressand value figure, and carries out scale tune It is whole;
The candidate frame is detected using the Rnet, exports the shot chart and the regressand value figure;
The shot chart that Y score is greater than preset fraction is chosen, it is non-greatly to there are the progress of the shot chart of overlapping region Value inhibits;
If detection terminates without the candidate frame, the result without license plate is exported;
The candidate frame if it exists is then adjusted the candidate frame using the regressand value figure, and will test Area maps return original image.
Optionally, with reference to the above first aspect, in the fourth possible implementation, update in the tracking list First license board information includes:
First license board information of previous frame is revised as to first license board information of next frame.
The application second aspect provides a kind of system of Car license recognition, comprising:
Shooting unit, for shooting video flowing;
Converting unit, for the image in video flowing to be converted to gray level image;
Acquiring unit, for obtaining tracking list;
Detection unit, for detecting whether the tracking list is empty;
Tracking cell, for not being empty, in the tracking tracking list the first license board information when the tracking list;
First detection module, for detecting the confidence level of first license board information when tracking successfully;
Update module, it is up to standard for the confidence level when the first license board information, update described first in the tracking list License board information;
Second detection module, for when it is described tracking list is empty, using vehicle plate location model detection video flowing in whether There are the second license board informations;
Third detection module, for detecting setting for second license board information when detecting the second license board information success Reliability;
Adding unit, it is up to standard for the confidence level when second license board information, second license board information is added to The tracking list.
Optionally, the system also includes:
Training unit, for training vehicle plate location model.
Optionally, the training unit includes:
Shooting module generates training pictures for shooting the video or image of parking lot entrance;
Labeling module, for marking the license plate area for including in the trained pictures;
Sample generation module, for generating described n1 positive sample of trained pictures, n2 part license plate, n3 negative samples This;
First training module, using the positive sample, the part license plate sample and negative sample training multitask volume The Pnet detection model of product neural network (MTCNN).
4th detection module detects training pictures using the Pnet detection model, to obtain Pnet void inspection picture;
Second training module, using the positive sample, the part license plate sample and Pnet void inspection picture training are more The Rnet detection model of task convolutional neural networks (MTCNN).
The embodiment of the present application third aspect provides a kind of computer installation, comprising:
Processor, memory, input-output equipment and bus;
The processor, memory, input-output equipment are connected with the bus respectively;
The processor is for executing such as the described in any item methods of previous embodiment.
The embodiment of the present application fourth aspect provides a kind of computer readable storage medium, is stored thereon with computer journey Sequence, it is characterised in that: the step of computer program realizes method as in the foregoing embodiment when being executed by processor.
As can be seen from the above technical solutions, the embodiment of the present application has the advantage that in the present embodiment, obtains stop in real time The video flowing of parking lot entrance contains license board information in the video flowing, and the license board information is for generating tracking list;It will view Image in frequency stream is converted to gray level image;Obtain tracking list;Detect whether the tracking list is empty;If the tracking column Table is not sky, then tracks the first license board information in the tracking list;If tracking successfully, first license board information is detected Confidence level;If the confidence level of first license board information is up to standard, first license plate letter in the tracking list is updated Breath;If list is empty for the tracking, it whether there is the second license board information using vehicle plate location model detection video flowing is interior;If inspection The second license board information success is surveyed, then detects the confidence level of second license board information;If second license board information is set Reliability is up to standard, then second license board information is added to the tracking list.Therefore, the application has used video flowing, license plate It detects and tracks the scheme combined with license plate, it is compared with the prior art middle to solve doing for overseas License Plate using single technological means Method, improves the accuracy rate of overseas License Plate and the capture rate of license plate and the image for taking video flowing is converted to gray scale Image reduces interference of the color for Car license recognition, increases the speed of Car license recognition.
Detailed description of the invention
Fig. 1 is a kind of embodiment of the method for Car license recognition in the embodiment of the present application;
Fig. 2 is another embodiment of the method for Car license recognition in the embodiment of the present application;
Fig. 3 is another embodiment of the method for Car license recognition in the embodiment of the present application;
Fig. 4 is another embodiment of the method for Car license recognition in the embodiment of the present application;
Fig. 5 is a kind of one embodiment of the system embodiment of Car license recognition in the embodiment of the present application;
Fig. 6 is a kind of one embodiment of computer installation in the embodiment of the present application.
Specific embodiment
The embodiment of the present application provides a kind of method of Car license recognition, for reducing interference of the color for Car license recognition, Accelerate the speed of Car license recognition.
In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application Attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only The embodiment of the application a part, instead of all the embodiments.Based on the embodiment in the application, ordinary skill people Member's every other embodiment obtained without making creative work, all should belong to the model of the application protection It encloses.
The description and claims of this application and term " first ", " second ", " third ", " in above-mentioned attached drawing Four " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this way Data be interchangeable under appropriate circumstances, so that the embodiments described herein can be in addition to illustrating herein or describing Sequence other than appearance is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that covering is non-exclusive Include, for example, the process, method, system, product or equipment for containing a series of steps or units are not necessarily limited to clearly arrange Those of out step or unit, but may include be not clearly listed or it is solid for these process, methods, product or equipment The other step or units having.
The application is identified using video flowing, i.e., acquires video always, the positioning and knowledge of license plate are all carried out to each frame image Not.As, application scheme is the treatment process to each frame image, then directly terminates to work as when if it is determined that being no in treatment process Preceding treatment process and carry out to the image processing process of next frame, same time can only carry out the identification of a license board information, If can be understood as currently tracking list memory in license board information, the license board information in tracking list is tracked, if working as License board information is not present in preceding tracking list, then detects with the presence or absence of license board information among video flowing acquired image, if inspection Measure then to the license board information carry out confidence level detection, be added to if confidence level qualification tracking list carry out it is above-mentioned with Track operation.Synoptically, license plate locating method includes that car plate detection and license plate track two parts.This method uses first The inspection of Multi-Task Convolutional Neural Network (multitask convolutional neural networks, MTCNN) progress license plate It surveys.After successfully being detected license plate, just license plate area is tracked, tracking can reduce time-consuming.This method can effectively solve The difficulty of overseas car plate detection, and higher verification and measurement ratio can be obtained.
In order to make it easy to understand, the detailed process in the embodiment of the present application is described below, referring to Fig. 1, the application A kind of one embodiment of the method for Car license recognition includes: in embodiment
101, the video flowing of parking lot entrance is obtained in real time, license board information is contained in video flowing, and the license board information is used List is tracked in generating;
In the present embodiment, video flowing is shot, refers to that the capture apparatus of the entrance in parking lot shoots always video, cut The image of each frame in video flowing is taken, license board information is present in image, and license board information refers to working as generating tracking list License board information in image can be stored to later after testing in tracking list.
102, the image in video flowing is converted into gray level image;
In the present embodiment, to reduce interference of the color for Car license recognition, the image in video flowing can be converted to gray scale Image.Because when handling color image, it for RGB can be that three kinds of channels of RGB are handled, and for gray level image Processing only need to handle one channel of gray scale, this greatly reduced work of system during formatting Amount.It can be understood as after the color image of input is converted to gray image, just only exist two kinds of license plate background colors and font color ?.It i.e. or is that license plate background color is deep, font color is shallow, similar black matrix wrongly written or mispronounced character.It is that license plate background color is shallow, font color is deep, class Like white gravoply, with black engraved characters.The advantage of doing so is that firstly, overseas license plate there is a problem of more than type, such as the license plate kind of certain countries Class is more than 15 kinds, and license plate color is also very much.After being converted to gray level image, detection model can also be reduced to license plate background color and word The dependence for according with the information of color, causes the versatility of detection model more preferable.
103, tracking list is obtained;
In the present embodiment, since the detection of a license plate can only be carried out in the same time, that is, can only in the same time One license board information is handled, so needing first to obtain tracking list, if tracking list memory in license board information, preferentially License board information in processing tracking list.
104, whether detecting and tracking list is empty;
In the present embodiment, since the detection of a license plate can only be carried out in the same time, that is, can only in the same time One license board information is handled, if tracking list memory, in license board information, priority processing tracks the license plate letter in list Breath.It i.e. if list is empty, detects with the presence or absence of license board information in video flowing, if not empty, then to the vehicle in tracking list Board information is tracked, due to application scheme be present in each frame for the processing of image among, so tracking here It can be understood as whether the license board information in previous frame image is still present in the image of next frame.
105, whether the first license board information of detection tracks success;
In the present embodiment, whether the first license board information of detection, which tracks, successfully refers to that the license board information in previous frame image is No to be still present in the image of next frame, if still having, whether the confidence level for detecting the license board information is up to standard, if not depositing The license board information then is being deleted in tracking list, the image of next frame is being identified again.
106, whether the confidence level for detecting the first license board information is up to standard;
In the present embodiment, whether the confidence level of the first license board information of detection is up to standard to be referred to the license plate area traced into License Plate Segmentation and identification are carried out, the character each recognized has a confidence level, and the system meeting each character of Comprehensive Evaluation is set Reliability and all confidence levels of entire license plate and.If the score obtained is not less than the score defaulted in system, the license plate The confidence level of information is up to standard, if the confidence level of the license board information is not up to standard, if the license plate lower than the score defaulted in system The confidence level of information is up to standard, which is added in tracking list, if the confidence level of license board information is not up to standard, First license board information is then deleted out of tracking list, and exports the result without license plate.
107, the first license board information in tracking list is updated;
In the present embodiment, if the confidence level of the first license board information in tracking list is up to standard, it will be updated in tracking list License board information, include but are not limited to for the score of confidence level being added in license board information, the coordinate information of previous frame be changed to Coordinate information when current detection, coordinate information can be located at the location information in image for current license board information.
108, the first license board information in tracking list is deleted;
In the present embodiment, if the confidence level of the first license board information detected is not up to standard or tracks the first license board information not Success all can delete first license board information from tracking list, to re-start the detection of image in video flowing.
109, being detected using vehicle plate location model whether there is the second license board information in video flowing;
In the present embodiment, when list is empty for tracking, that is, there is currently no when the license board information tracked, it can examine It surveys and whether there is the second license board information in video flowing, it should be noted that the first license board information and the second license board information be not only to It can be actually the same license plate, such as stop when a vehicle drives into a kind of differentiation of the license board information of time identification Parking lot entrance if entering the stage of license plate tracking after recognizing after testing, but sails to license plate quilt after some position It blocks, will lead to tracking failure, then restart the part of car plate detection.
110, whether the confidence level for detecting the second license board information is up to standard;
In the present embodiment, when detecting that video stream memory, can be to the confidence of the second license board information in the second license board information Degree is detected, and detection confidence level can carry out license plate point for the license plate area to the second license board information in the image traced into It cuts and identifies, the character each recognized has a confidence level, and system can the confidence level of each character of Comprehensive Evaluation and entire The sum of all confidence levels of license plate.If the score obtained is not less than the score defaulted in system, the confidence of the license board information Spend it is up to standard, if lower than score in system is defaulted in, the confidence level of the license board information is not up to standard.If up to standard, by the second vehicle Board information is added to tracking list, and if it does not meet the standards, then detection terminates, and indicates that there is no license board informations among the image of identification.
111, the second license board information is added to tracking list.
In the present embodiment, if the confidence level of the second license board information is up to standard, the second license board information can be added to tracking list The stage tracked into license plate, then carry out the operation of the tracking and detection as described in above scheme.
In the present embodiment, shoot video flowing, contain license board information in the video flowing, the license board information for generate with Track list;Obtain tracking list;Detect whether the tracking list is empty;If the tracking list be not it is empty, described in tracking Track the first license board information in list;If tracking the confidence level for successfully detecting first license board information;If described first The confidence level of license board information is up to standard, then updates first license board information in the tracking list;If the tracking list is Sky then whether there is the second license board information using vehicle plate location model detection video flowing is interior;If detecting second license board information Success, then detect the confidence level of second license board information;If the confidence level of second license board information is up to standard, by described Two license board informations are added to the tracking list.Therefore, the application has used video flowing, and car plate detection is combined with license plate tracking Scheme all carries out detection confidence level to the image got in each frame, analyzes score value, the operation such as output regression value figure, phase The method for solving overseas License Plate using single technological means more in the prior art, improves the accurate of overseas License Plate The capture rate of rate and license plate.
In the present embodiment, based on car plate detection described in Fig. 1, a kind of mode of trained vehicle plate location model is proposed, have Body please refers to Fig. 2 and Fig. 3, and a kind of another embodiment of the method for Car license recognition includes:
201, training vehicle plate location model.
In the present embodiment, the process of Car license recognition mainly includes the process of car plate detection and license plate tracking, car plate detection Method predominantly using the vehicle plate location model after training in image whether there is license board information detect.For vehicle The training of board detection model, referring specifically to Fig. 3, a kind of another embodiment of the method for Car license recognition includes:
301, perhaps the image video or image include license plate area and background to the video of shooting parking lot entrance Region, generates a trained pictures, and the trained pictures include that the picture intercepted from video and field device obtain Picture;
In the present embodiment, training vehicle plate location model shoots a large amount of image and view firstly the need of in parking lot entrance Frequently, to improve the accuracy of subsequent detection license board information.Due to the randomness and the characteristic that shoots always of shooting, each frame In image background area can be also known as comprising license plate area and non-license plate area, non-license plate area.And in shooting process In convert the image into gray level image so that training pattern only needs to handle a kind of channel of gray level image, improve training pattern Versatility.
Specifically, after the color image of input being converted to gray image, two kinds of license plate background colors and font face are just only existed Color.It i.e. or is that license plate background color is deep, font color is shallow, similar black matrix wrongly written or mispronounced character.It is that license plate background color is shallow, font color is deep, Similar white gravoply, with black engraved characters.The advantage of doing so is that firstly, overseas license plate there is a problem of more than type, such as the license plate of certain countries Type is more than 15 kinds, and license plate color is also very much.After being converted to gray level image, it is possible to reduce model is to license plate background color and character face The dependence of the information of color.The versatility of model is more preferable.Second, if a kind of vehicle of new color is released in a country Board, if the license plate with license plate before in addition to variant in color, other differences are little, then use original model New license plate can be detected very well.
302, the license plate area for including in the trained pictures is marked;
In the present embodiment, to distinguish the required positive sample of subsequent training, part license plate sample and negative sample, due to this three The main distinction of kind sample is the size with license plate area lap, so can first mark out in the video or image The license plate area for including.
303, n1 positive sample, n2 part license plate sample, n3 negative samples are generated from the trained pictures using program This, the positive sample be with the overlapping region of tab area in the first preset ratio section, the part license plate sample be with In the second preset ratio section, the negative sample is the overlapping region with tab area in third for the overlapping region of tab area In preset ratio section;
In the present embodiment, for positive sample, part license plate and negative sample, illustratively, the present embodiment are proposed, order and license plate It is positive sample that region lap, which is more than 70%, in 30%-50% is part license plate with license plate area lap, with vehicle Board region lap is negative sample less than 20%, chooses three kinds of ratios and exists centainly because of such three kinds of ratios Otherness is differentiated more obvious.
304, using the positive sample, the part license plate sample and negative sample training MTCNN multitask convolution mind Pnet detection model through network;
In the present embodiment, illustratively, 5 positive samples, 5 part license plates and 10 can be generated at random for same license plate A negative sample, the color image that training uses, that is, the positive sample and negative sample of the training inputted all include tri- channels RGB. When Pnet training, using positive sample, part license plate is trained together with negative sample, obtains Pnet detection model.Such as using 5000 single layer Hong Kong license plate samples are trained, and can generate 25000 positive samples, 25000 part license plate samples and 50000 negative samples.When training, just above-mentioned 100000 samples are put into togerther in program.
305, training pictures are detected using the Pnet detection model, to obtain Pnet void inspection picture, the Pnet is empty It is license plate that inspection, which is Pnet detection model testing result, but is less than the detection of preset threshold with the lap of the tab area As a result;
In the present embodiment, after training Pnet, training sample is detected using Pnet detection model.Due to training Sample has contained mark, so that it may obtain the empty inspection of Pnet model, that is, the result detected is if it is license plate, but the result Region is overlapping with tab area be less than threshold value be exactly it is empty examine, illustratively, this programme threshold value is selected as 0.2, as trained As Pnet void inspection of the sample and license plate area lap out less than 20%.
306, using the positive sample, the part license plate sample and Pnet void inspection picture training multitask convolution mind Rnet detection model through network (MTCNN).
In the present embodiment, the positive sample that trained Pnet is used can be reused after training Pnet model, part license plate with Rnet model is trained in the inspection of Pnet void together.Positive and negative sample proportion is 1:1.Such as it is carried out using 5000 single layer Hong Kong license plate samples Training can generate 25000 positive samples, 25000 part license plate samples and 25000 Pnet void inspections.When training, Just above-mentioned 75000 samples are put into togerther in program, obtain Rnet model.
In the present embodiment, since MCTNN network is mainly used in the field of recognition of face, the Aspect Ratio of face is mostly 1:1, but apply in license plate field, for the overseas double-deck license plate, the Aspect Ratio of the ratio of 1:1 and license plate also phase Closely, it is suitble to directly use, but for the license plate of single layer, length is larger with wide gap, so the application is in training single layer It when the training pattern of license plate, is trained using the ratio of width to height of 2:1, when using such training pattern, in actual mechanical process Detection effect is good, is the improvement of a kind of pair of primitive network.
In the present embodiment, it whether there is the second license plate in video flowing based on being detected described in Fig. 1 with vehicle plate location model Information, referring specifically to Fig. 4, a kind of another embodiment of the method for Car license recognition includes:
40, car plate detection flow chart.
Illustratively, a kind of method that the present embodiment proposes car plate detection, in order to improve speed, the image of input is first passed through It reduces, obtains a series of images pyramid;Using the Pnet layers of progress full figure detection of trained CNN network, export shot chart and Regressand value figure;It chooses the rectangle frame for obtaining score value 0.6 or more and chooses 10 frames of highest scoring after traversing full figure;And to having The candidate frame of overlapping region carries out non-maxima suppression;If not finding candidate frame, detection terminates, and exports the knot without license plate Fruit;If finding candidate frame, rescaling is carried out to the candidate frame detected;Using Rnet layers of trained CNN network to inspection The candidate frame measured is detected, and shot chart and regressand value figure are exported;Choose the rectangle frame for obtaining score value 0.7 or more, traversal After, choose 5 frames of highest scoring;And non-maxima suppression is carried out to the candidate frame for having overlapping region.If do not looked for To candidate frame, detection terminates, and exports the result without license plate.If there is the candidate frame detected, then it is mapped in original image.
Above-mentioned steps can be understood as downscaled images to reduce space shared by image, and a system is obtained during diminution The image pyramid of column, image pyramid are one kind of multi-scale expression in image, the main segmentation for image, are a kind of Carry out the simple structure of effective but concept of interpretation of images with multiresolution.Image pyramid is used primarily for machine vision and image pressure Contracting, the pyramid of piece image are that a series of resolution ratio with Pyramid arrangement gradually reduce, and derive from same original The image collection of beginning figure.It is obtained by echelon to down-sampling, just stops sampling until reaching some termination condition.MCTNN net Pnet layers of network are trained Pnet model and Rnet model with Rnet layers of MCTNN network, and shot chart is that shot chart is square The score of shape frame, the adjusted value for the rectangle frame that regressand value figure network obtains.Non-maxima suppression can be understood as if there is more The overlapping region of a frame is greater than threshold value, and only keep score highest frame.
Since there are single layers and double-deck two kinds of forms for overseas license plate, so during actual car plate detection, single layer With the detection of double-deck model alternately license plate area.Such as first frame is detected using single-layer model, if detected License plate area, expanding 100% on license plate area, lower to expand 100%, left to expand 10%, then right expansion 10% is sent into the region subsequent License Plate Segmentation and identification.If the confidence level of the license plate recognized is up to standard, current license plate is added in tracking list, under One frame starts to track.If not detecting license plate area, current frame alignment terminates.Next frame replacement uses bilayer model, such as Fruit detects license plate area, and expanding 10% on license plate area, lower to expand 10%, left expansion 100% is right to expand 100%, then the region It is sent into subsequent License Plate Segmentation and identification.If the confidence level of the license plate recognized is up to standard, current license plate is added to tracking In list.If not detecting license plate area, current frame alignment terminates.
In the present embodiment, a kind of method for proposing car plate detection, wherein using the width of 2:1 when to single layer license plate model training It is high than so that training pattern for actual license plate model closer to so that model inspection to license plate area reduce calculating when Between and process, increase exploitativeness for scheme.And by converting the image into gray level image, reduce color for vehicle The interference of board identification, accelerates the speed of Car license recognition.
Referring to Fig. 5, a kind of one embodiment of the system of Car license recognition includes: in the embodiment of the present application
Shooting unit 501, for shooting video flowing;
Converting unit 502, for the image in video flowing to be converted to gray level image;
Acquiring unit 503, for obtaining tracking list;
Detection unit 504, for detecting whether the tracking list is empty;
Tracking cell 505, for not being empty, in the tracking tracking list the first license plate letter when the tracking list Breath;
First detection module 506, for detecting the confidence level of first license board information when tracking successfully;
Update module 507, it is up to standard for the confidence level when the first license board information, update described the in the tracking list One license board information;
Second detection module 508, for working as the tracking, list is empty, and being detected in video flowing using vehicle plate location model is It is no that there are the second license board informations;
Third detection module 509, for detecting second license board information when detecting the second license board information success Confidence level;
Adding unit 510, it is up to standard for the confidence level when second license board information, second license board information is added To the tracking list.
As a preferred embodiment, the system also includes:
Training unit 511, for training vehicle plate location model.
As a preferred embodiment, the training unit 511 includes:
Shooting module 5111 generates training pictures for shooting the video or image of parking lot entrance;
Labeling module 5112, for marking the license plate area for including in the trained pictures;
Sample generation module 5113, for generating described n1 positive sample of trained pictures, n2 part license plate, n3 negative Sample;
First training module 5114, using the positive sample, the part license plate sample and more of negative sample training The Pnet detection model of business convolutional neural networks (MTCNN).
4th detection module 5115 detects training pictures using the Pnet detection model, to obtain Pnet void inspection figure Piece;
Second training module 5116, using the positive sample, the part license plate sample and Pnet void inspection picture instruction Practice the Rnet detection model of multitask convolutional neural networks (MTCNN).
The computer installation in the embodiment of the present application is described from the angle of entity apparatus below, referring to Fig. 6, this One embodiment of computer installation includes: in application embodiment
The computer installation 600 can generate bigger difference because configuration or performance are different, may include one or one A above central processing unit (central processing units, CPU) 601 (for example, one or more processors) With memory 605, one or more application program or data are stored in the memory 605.
Wherein, memory 605 can be volatile storage or persistent storage.The program for being stored in memory 605 can wrap One or more modules are included, each module may include to the series of instructions operation in server.Further, in Central processor 601 can be set to communicate with memory 605, and a series of fingers in memory 605 are executed on intelligent terminal 600 Enable operation.
The computer installation 600 can also include one or more power supplys 602, one or more wired or nothings Wired network interface 603, one or more input/output interfaces 604, and/or, one or more operating systems, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM etc..
It is understood that the size of the serial number of above steps is not meant in the various embodiments of the application Execution sequence it is successive, the execution of each step sequence should be determined by its function and internal logic, without coping with the embodiment of the present application Implementation process constitute any restriction.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In several embodiments provided herein, it should be understood that disclosed system, device and method can be with It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit It divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components It can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown or The mutual coupling, direct-coupling or communication connection discussed can be through some interfaces, the indirect coupling of device or unit It closes or communicates to connect, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme 's.
It, can also be in addition, each functional unit in each embodiment of the application can integrate in one processing unit It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product When, it can recorde in a computer-readable recording medium.Based on this understanding, the technical solution of the application is substantially The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words It embodies, which is recorded in a recording medium, including some instructions are used so that a computer Equipment (can be personal computer, server or the network equipment etc.) executes the complete of each embodiment the method for the application Portion or part steps.And recording medium above-mentioned includes: USB flash disk, mobile hard disk, read-only logger (ROM, Read-Only Memory), arbitrary access logger (RAM, RandomAccess Memory), magnetic or disk etc. are various can recorde journey The medium of sequence code.
The above, above embodiments are only to illustrate the technical solution of the application, rather than its limitations;Although referring to before Embodiment is stated the application is described in detail, those skilled in the art should understand that: it still can be to preceding Technical solution documented by each embodiment is stated to modify or equivalent replacement of some of the technical features;And these It modifies or replaces, the spirit and scope of each embodiment technical solution of the application that it does not separate the essence of the corresponding technical solution.

Claims (10)

1. a kind of method of Car license recognition characterized by comprising
The video flowing of parking lot entrance is obtained in real time, license board information is contained in the video flowing, the license board information is for giving birth to At tracking list;
Image in video flowing is converted into gray level image;
Obtain the tracking list;
Detect whether the tracking list is empty;
If the tracking list is not sky, the first license board information in the tracking list is tracked;
If tracking the confidence level for successfully detecting first license board information;
If the confidence level of first license board information is up to standard, first license board information in the tracking list is updated;
If list is empty for the tracking, it whether there is the second license board information using vehicle plate location model detection video flowing is interior;
If detecting the second license board information success, the confidence level of second license board information is detected;
If the confidence level of second license board information is up to standard, second license board information is added to the tracking list.
2. the method according to claim 1, wherein believing in the detection video flowing with the presence or absence of the second license plate Before breath, the method also includes:
Training vehicle plate location model, the vehicle plate location model are used for car plate detection.
3. according to the method described in claim 2, it is characterized in that, the trained vehicle plate location model includes:
Perhaps the image video or image include license plate area and background area, life to the video of shooting parking lot entrance At a trained pictures, the trained pictures include the picture that the picture intercepted from video and field device obtain;
Mark the license plate area for including in the trained pictures;
Generate n1 positive sample from the trained pictures using program, n2 part license plate sample, n3 negative sample, it is described just Sample is the overlapping region with tab area in the first preset ratio section, and the part license plate sample is and tab area In the second preset ratio section, the negative sample is the overlapping region with tab area in third preset ratio area for overlapping region In;
Using the positive sample, the part license plate sample and the negative sample train MTCNN multitask convolutional neural networks Pnet detection model;
Training pictures are detected using the Pnet detection model, to obtain Pnet void inspection picture, the Pnet void inspection is Pnet Detection model testing result is license plate, but is less than the testing result of preset threshold with the lap of the tab area;
Using the positive sample, the part license plate sample and the Pnet void examine picture training MTCNN multitask convolutional Neural The Rnet detection model of network.
4. according to the method described in claim 3, it is characterized in that, whether described detected in video flowing with vehicle plate location model deposits Include: in the second license board information
Input picture pyramid is generated, the input picture is the video flowing obtained in real time, and the pyramid is by every in video flowing The scaled obtained a series of pictures of the image of one frame;
Full figure detection, output shot chart and regressand value figure are carried out using the Pnet;
The shot chart that X score is greater than preset fraction is chosen, to there are the shot charts of overlapping region to carry out non-maximum suppression System;
If detection terminates without candidate frame, the result without license plate is exported;
The candidate frame if it exists is then adjusted the candidate frame using the regressand value figure, and carries out rescaling;
The candidate frame is detected using the Rnet, exports the shot chart and the regressand value figure;
The shot chart that Y score is greater than preset fraction is chosen, to there are the shot charts of overlapping region to carry out non-maximum suppression System;
If detection terminates without the candidate frame, the result without license plate is exported;
The candidate frame if it exists, the then region for the candidate frame being adjusted, and being will test using the regressand value figure Map back original image.
5. the method according to claim 1, wherein updating first license board information in the tracking list Include:
First license board information of previous frame is revised as to first license board information of next frame.
6. a kind of system of Car license recognition characterized by comprising
Shooting unit, for shooting video flowing;
Converting unit, for the image in video flowing to be converted to gray level image;
Acquiring unit, for obtaining tracking list;
Detection unit, for detecting whether the tracking list is empty;
Tracking cell, for not being empty, in the tracking tracking list the first license board information when the tracking list;
First detection module, for detecting the confidence level of first license board information when tracking successfully;
Update module, it is up to standard for the confidence level when the first license board information, update first license plate in the tracking list Information;
Second detection module, for working as the tracking, list is empty, and being detected in video flowing using vehicle plate location model whether there is Second license board information;
Third detection module, for detecting the confidence level of second license board information when detecting the second license board information success;
Adding unit, it is up to standard for the confidence level when second license board information, second license board information is added to described Track list.
7. system according to claim 6, which is characterized in that described device further include:
Training unit, for training vehicle plate location model.
8. system according to claim 7, which is characterized in that the training unit includes:
Shooting module generates training pictures for shooting the video or image of parking lot entrance;
Labeling module, for marking the license plate area for including in the trained pictures;
Sample generation module, for generating described n1 positive sample of trained pictures, n2 part license plate, n3 negative sample;
First training module, using the positive sample, the part license plate sample and negative sample training multitask convolution mind Pnet detection model through network (MTCNN).
4th detection module detects training pictures using the Pnet detection model, to obtain Pnet void inspection picture;
Second training module, using the positive sample, the part license plate sample and the Pnet void examine picture training multitask The Rnet detection model of convolutional neural networks (MTCNN).
9. a kind of computer installation, which is characterized in that the computer installation includes: input/output interface, processor and storage Device is stored with program instruction in the memory;
The processor executes method a method as claimed in any one of claims 1 to 5 for executing the program instruction stored in memory.
10. a kind of computer readable storage medium, including instruction, which is characterized in that when described instruction is transported on a computing device When row, so that the computer equipment executes method according to any one of claims 1 to 5.
CN201811626428.1A 2018-12-28 2018-12-28 License plate recognition method and related device Active CN109711407B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811626428.1A CN109711407B (en) 2018-12-28 2018-12-28 License plate recognition method and related device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811626428.1A CN109711407B (en) 2018-12-28 2018-12-28 License plate recognition method and related device

Publications (2)

Publication Number Publication Date
CN109711407A true CN109711407A (en) 2019-05-03
CN109711407B CN109711407B (en) 2023-02-28

Family

ID=66259156

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811626428.1A Active CN109711407B (en) 2018-12-28 2018-12-28 License plate recognition method and related device

Country Status (1)

Country Link
CN (1) CN109711407B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110321969A (en) * 2019-07-11 2019-10-11 山东领能电子科技有限公司 A kind of vehicle face alignment schemes based on MTCNN
CN111666938A (en) * 2020-05-21 2020-09-15 珠海大横琴科技发展有限公司 Two-place double-license-plate detection and identification method and system based on deep learning
CN111862617A (en) * 2020-06-12 2020-10-30 浙江大华技术股份有限公司 License plate recognition method, device and system and computer equipment
CN112330715A (en) * 2020-10-09 2021-02-05 深圳英飞拓科技股份有限公司 Tracking method, tracking device, terminal equipment and readable storage medium
CN112362673A (en) * 2020-11-17 2021-02-12 清华大学天津高端装备研究院洛阳先进制造产业研发基地 Visual detection method and system for dumplings
CN112712708A (en) * 2020-12-28 2021-04-27 上海眼控科技股份有限公司 Information detection method, device, equipment and storage medium
CN113781790A (en) * 2021-08-31 2021-12-10 深圳市捷顺科技实业股份有限公司 Method for recognizing license plate and parking lot terminal
CN114677774A (en) * 2022-03-30 2022-06-28 深圳市捷顺科技实业股份有限公司 Barrier gate control method and related equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101183425A (en) * 2007-12-20 2008-05-21 四川川大智胜软件股份有限公司 License plate positioning method in Guangdong and Hong Kong
US20130028481A1 (en) * 2011-07-28 2013-01-31 Xerox Corporation Systems and methods for improving image recognition
CN104298976A (en) * 2014-10-16 2015-01-21 电子科技大学 License plate detection method based on convolutional neural network
CN105894004A (en) * 2014-12-16 2016-08-24 中防通用电信技术有限公司 Complement technology of locating incomplete license plate aiming at license plate identification system
CN106934396A (en) * 2017-03-09 2017-07-07 深圳市捷顺科技实业股份有限公司 A kind of license number search method and system
WO2018090771A1 (en) * 2016-11-16 2018-05-24 杭州海康威视数字技术股份有限公司 Vehicle license plate recognition method and apparatus

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101183425A (en) * 2007-12-20 2008-05-21 四川川大智胜软件股份有限公司 License plate positioning method in Guangdong and Hong Kong
US20130028481A1 (en) * 2011-07-28 2013-01-31 Xerox Corporation Systems and methods for improving image recognition
CN104298976A (en) * 2014-10-16 2015-01-21 电子科技大学 License plate detection method based on convolutional neural network
CN105894004A (en) * 2014-12-16 2016-08-24 中防通用电信技术有限公司 Complement technology of locating incomplete license plate aiming at license plate identification system
WO2018090771A1 (en) * 2016-11-16 2018-05-24 杭州海康威视数字技术股份有限公司 Vehicle license plate recognition method and apparatus
CN106934396A (en) * 2017-03-09 2017-07-07 深圳市捷顺科技实业股份有限公司 A kind of license number search method and system

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110321969A (en) * 2019-07-11 2019-10-11 山东领能电子科技有限公司 A kind of vehicle face alignment schemes based on MTCNN
CN110321969B (en) * 2019-07-11 2023-06-30 山东领能电子科技有限公司 A car face alignment method based on MTCNN
CN111666938A (en) * 2020-05-21 2020-09-15 珠海大横琴科技发展有限公司 Two-place double-license-plate detection and identification method and system based on deep learning
CN111862617A (en) * 2020-06-12 2020-10-30 浙江大华技术股份有限公司 License plate recognition method, device and system and computer equipment
CN112330715A (en) * 2020-10-09 2021-02-05 深圳英飞拓科技股份有限公司 Tracking method, tracking device, terminal equipment and readable storage medium
CN112330715B (en) * 2020-10-09 2024-09-24 深圳英飞拓仁用信息有限公司 Tracking method, tracking device, terminal equipment and readable storage medium
CN112362673A (en) * 2020-11-17 2021-02-12 清华大学天津高端装备研究院洛阳先进制造产业研发基地 Visual detection method and system for dumplings
CN112712708A (en) * 2020-12-28 2021-04-27 上海眼控科技股份有限公司 Information detection method, device, equipment and storage medium
CN113781790A (en) * 2021-08-31 2021-12-10 深圳市捷顺科技实业股份有限公司 Method for recognizing license plate and parking lot terminal
CN113781790B (en) * 2021-08-31 2022-08-12 深圳市捷顺科技实业股份有限公司 Method for recognizing license plate and parking lot terminal
CN114677774A (en) * 2022-03-30 2022-06-28 深圳市捷顺科技实业股份有限公司 Barrier gate control method and related equipment
CN114677774B (en) * 2022-03-30 2023-10-17 深圳市捷顺科技实业股份有限公司 Barrier gate control method and related equipment

Also Published As

Publication number Publication date
CN109711407B (en) 2023-02-28

Similar Documents

Publication Publication Date Title
CN109711407A (en) A kind of method and relevant apparatus of Car license recognition
CN110738101B (en) Behavior recognition method, device and computer-readable storage medium
CN113553977B (en) Improved YOLO V5-based safety helmet detection method and system
CN101667245B (en) Face Detection Method Based on Support Vector Novelty Detection Classifier Cascade
US20210004587A1 (en) Image detection method, apparatus, device and storage medium
CN110516560B (en) Target detection method of optical remote sensing image based on FPGA heterogeneous deep learning
CN109726678A (en) A kind of method and relevant apparatus of Car license recognition
CN105574550A (en) Vehicle identification method and device
WO2021115345A1 (en) Image processing method and apparatus, computer device, and storage medium
CN107563372A (en) A kind of license plate locating method based on deep learning SSD frameworks
US20220207266A1 (en) Methods, devices, electronic apparatuses and storage media of image processing
CN109670517A (en) Object detection method, device, electronic equipment and target detection model
CN112241667A (en) Image detection method, device, equipment and storage medium
CN111881803B (en) An animal face recognition method based on improved YOLOv3
CN112183356A (en) Driving behavior detection method, device and readable storage medium
CN111931661A (en) Real-time mask wearing detection method based on convolutional neural network
CN106874913A (en) A kind of vegetable detection method
CN109117746A (en) Hand detection method and machine readable storage medium
CN110309825A (en) Uighur language detection method, system and electronic equipment under a complex background
CN109977875A (en) Gesture identification method and equipment based on deep learning
CN114332915B (en) Human attribute detection method, device, computer equipment and storage medium
CN111814726A (en) A detection method for detecting robot visual objects
CN118470501A (en) Construction site safety helmet wearing detection method based on improved YOLOv n complex environment
CN115705682A (en) Article package damage detection method, article package damage detection device, computer device, and storage medium
CN116665015A (en) A YOLOv5-based detection method for weak and small targets in infrared sequence images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant