CN105117724B - A kind of license plate locating method and device - Google Patents

Abstract

Embodiments of the present invention provide a license plate location method and device. The method includes: inputting the Haar feature of the video frame image into the first classifier to obtain the first type of license plate image information sequence, and calculating and screening out the maximum similarity value of the first type of license plate. Value, judge whether it is greater than the preset first similarity threshold, if so, locate at least one valid license plate image, otherwise rotate the video frame image, input the rotated image into the first classifier, and obtain multiple second types License plate image information sequence, calculate and filter out the second type of license plate similarity value with the largest value, and judge whether it is greater than the preset second similarity threshold, if so, locate at least one valid license plate image, otherwise, the LBP of the video frame image The features are input into the second classifier to obtain the third type of license plate image information sequence, and at least one valid license plate image is located based on the maximum value of the similarity value of the third type of license plate. By applying the embodiment of the present invention, the accuracy of locating the license plate can be improved.

Description

A license plate location method and device

technical field

The invention relates to the technical field of computer vision, in particular to a method and device for locating a license plate.

Background technique

With the maturity of digital image processing, pattern recognition and artificial intelligence technology, Intelligent Transportation System (ITS) has gradually become the trend of road traffic development in the 21st century. The license plate recognition technology is the basis of realizing the intelligent transportation system, and the license plate location is a very important link in the license plate recognition. In complex scenes of non-traffic checkpoints such as communities, campuses, crossroads or multi-lane intersections, the license plate areas in the video frame images captured by surveillance cameras usually have obvious differences in position, angle, size and definition. In addition, The background area in the video frame images captured at non-traffic checkpoints is cluttered.

In the existing license plate location technology for non-traffic checkpoints, some features in the video frame images taken under non-traffic checkpoints are usually extracted, for example, a single feature such as gray value or color is used as the feature for locating the license plate .

Obviously, these unfavorable factors of the license plate area in the video frame images captured under non-traffic checkpoints have increased the difficulty of accurately locating the license plate in the later stage. Therefore, when a certain single feature is used to locate the license plate in the above-mentioned prior art, There are often positioning failures or missed detections.

Contents of the invention

The purpose of the embodiment of the present invention is to provide a license plate location method and device, so as to realize accurate positioning of the license plate and reduce the situation of location failure or missing detection.

In order to achieve the above object, the embodiment of the present invention discloses a license plate location method, the method comprising:

Extract the Haar Haar feature of the video frame image of the license plate to be located, and input the Haar feature into the pre-established first classifier to obtain the first type of license plate image information sequence corresponding to the video frame image, wherein, The first type of license plate image information sequence includes the position information of the first type of candidate license plate image searched in the video frame image, and the first classifier is: based on the extracted first sample image set for a predetermined The Haar features of each image, and the classifier trained by the adaptive promotion algorithm Adaboost algorithm;

Based on the position information of the first type of candidate license plate images, determine the first type of candidate license plate images in the video frame images;

Calculate the first type of license plate similarity value of the first type of candidate license plate image, and filter out the maximum value in the first type of license plate similarity value;

Judging whether the maximum value of the similarity values of the first type of license plate obtained by screening is greater than the preset first similarity threshold:

When the judgment result is greater than, at least one effective license plate image is located from the first type of candidate license plate images according to the preset first valid license plate location rule and the calculated similarity value of the first type of license plate. When is not greater than, according to the preset image rotation rules, the video frame image is rotated N times, and the video frame image obtained after each rotation is re-inputted into the first classifier to obtain N second classifiers. A similar license plate image information sequence, based on the position information of the second type of candidate license plate image in the second type of license plate image information sequence, determine the second type of candidate license plate image in the video frame image, and calculate each second type of license plate image The second type of license plate similarity value of the second type of candidate license plate image corresponding to the image information sequence, and filter out the second type of license plate similarity value of the second type of candidate license plate image corresponding to each second type of license plate image information sequence The maximum value, and then, from the maximum value among the second-type license plate similarity values obtained by screening, determine the target second-type license plate similarity value with the largest value, and judge whether the target second-type license plate similarity value is greater than the preset If the second similarity threshold is greater than the second type of license plate image information corresponding to the target second type of license plate similarity value, according to the preset second valid license plate location rule and the calculated second type of license plate similarity value At least one valid license plate image is located in the second type of candidate license plate images targeted by the sequence, if not greater than

Continue to extract the local binary pattern LBP feature of the video frame image, and input the LBP feature into the pre-established second classifier to obtain the third type of license plate image information sequence corresponding to the video frame image, wherein , the third type of license plate image information sequence includes the position information of the third type of candidate license plate image searched in the video frame image, and the second classifier is: based on the extracted second sample image set for a predetermined The local binary mode LBP feature of each image, and the classifier obtained by using the adaptive promotion algorithm Adaboost algorithm training;

Determining a third type of candidate license plate image in the video frame image based on the position information of the third type of candidate license plate image;

Calculate the third type of license plate similarity value of the third type of candidate license plate image, and filter out the maximum value in the third type of license plate similarity value;

Judging whether the maximum value of the similarity value of the third type of license plate obtained by screening is greater than the preset third similarity threshold; when the judgment result is greater than, according to the preset third valid license plate positioning rule and the calculated third type of license plate Similarity value, at least one valid license plate image is located from the third type of candidate license plate images, and when the judgment result is not greater than, it is determined that no valid license plate image has been located by the first classifier and the second classifier.

Preferably, the process of calculating the similarity value of the first type of license plate of any first type of candidate license plate images includes:

Perform binarization processing on the first type of candidate license plate images;

performing open operation morphological transformation on the first type of candidate license plate images obtained after binarization;

Through the Sobel algorithm, the first type of candidate license plate image after the opening operation morphological transformation is subjected to vertical filtering processing, and the first type vertical gradient image of the first type of candidate license plate image after the opening operation morphological transformation is obtained;

Perform edge detection on the first type of vertical gradient image by Canny algorithm to obtain the first type of edge image;

For the edge extracted in the first type of edge image, perform sextiles according to the vertical direction, and calculate the jump mean value of the edge part pixels in the horizontal direction in each equalization;

An average value of the transition average values is calculated, and the calculated average value is determined as the first type of license plate similarity value of the first type of candidate license plate images.

Preferably, the positioning of at least one valid license plate image from the first type of candidate license plate images according to the preset first valid license plate location rule and the calculated similarity value of the first type of license plate includes:

Determining the first type of candidate license plate image corresponding to the maximum value in the first type of license plate similarity value as the located valid license plate image;

or

Among the first type of license plate similarity values, the first type of candidate license plate images corresponding to the first type of license plate similarity values greater than the preset first positioning threshold are determined as valid license plate images located.

Preferably, after it is determined that no valid license plate image is located by the first classifier and the second classifier, the method further includes:

Input the Haar feature into the pre-established third classifier to obtain a fourth type of license plate image information sequence corresponding to the video frame image, wherein the fourth type of license plate image information sequence includes the video frame image information sequence The position information of the searched fourth type of candidate license plate image, the third classifier is: based on the Haar feature extracted for each image in the predetermined third sample image set, and adopts the adaptive promotion algorithm Adaboost algorithm The trained classifier;

Based on the position information of the fourth type of candidate license plate image, determine the fourth type of candidate license plate image in the video frame image;

Calculate the fourth type of license plate similarity value of the fourth type of candidate license plate image, and filter out the maximum value in the fourth type of license plate similarity value;

Judging whether the maximum value of the similarity values of the fourth type of license plate obtained by screening is greater than the preset fourth similarity threshold; license plate similarity value, locate at least one valid license plate image from the fourth type of candidate license plate images, and when the judgment result is not greater than, determine that the first classifier, the second classifier and the third classifier are not located to a valid license plate image.

Preferably, after it is determined that a valid license plate image is not located by the first classifier, the second classifier and the third classifier, the method further includes:

Input the Haar feature into the pre-established fourth classifier to obtain the fifth type of license plate image information sequence corresponding to the video frame image, wherein the fifth type of license plate image information sequence includes the video frame image information sequence The position information of the fifth type of candidate license plate images searched, the fourth classifier is: based on the extracted Haar features for each image in the fourth sample image set, and using the adaptive promotion algorithm Adaboost algorithm training to obtain classifier;

Based on the position information of the fifth type of candidate license plate image, determine the fifth type of candidate license plate image in the video frame image;

Calculate the fifth type of license plate similarity value of the fifth type of candidate license plate image, and filter out the maximum value in the fifth type of license plate similarity value;

Judging whether the maximum value of the similarity value of the fifth type of license plate obtained by screening is greater than the preset fifth similarity threshold; License plate similarity value, at least one valid license plate image is located from the fifth type of candidate license plate images, and when the judgment result is not greater than, it is determined that no valid license plate image has been located.

In order to achieve the above purpose, the embodiment of the present invention discloses a license plate positioning device, which includes:

The first license plate sequence acquisition module is used to extract the Haar Haar feature of the video frame image of the license plate to be located, and input the Haar feature into the pre-established first classifier to obtain the first corresponding to the video frame image. A type of license plate image information sequence, wherein the first type of license plate image information sequence includes the position information of the first type of candidate license plate image searched in the video frame image, and the first classifier is: based on the extracted Haar features of each image in the predetermined first sample image set, and a classifier obtained by using the adaptive promotion algorithm Adaboost algorithm training;

The first license plate image determination module is configured to determine the first type of candidate license plate images in the video frame images based on the position information of the first type of candidate license plate images;

The first license plate similarity calculation module is used to calculate the first type of license plate similarity value of the first type of candidate license plate image, and filter out the maximum value in the first type of license plate similarity value;

The first license plate similarity judging module is used to judge whether the maximum value in the first type of license plate similarity obtained by screening is greater than the preset first similarity threshold; when the judgment result is greater than, according to the preset first valid license plate Positioning rules and the calculated similarity value of the first type of license plate, locating at least one valid license plate image from the first type of candidate license plate images, and triggering the second license plate sequence acquisition module when the judgment result is not greater than;

The second license plate sequence obtaining module is configured to perform N rotations on the video frame image according to a preset image rotation rule, and re-input the video frame image obtained after each rotation into the first classifier , to obtain N second-type license plate image information sequences;

The second license plate image determining module is configured to determine the second type of candidate license plate image in the video frame image based on the position information of the second type of candidate license plate image in the second type of license plate image information sequence;

The second license plate similarity calculation module is used to calculate the second type of license plate similarity value of the second type of candidate license plate image corresponding to each second type of license plate image information sequence, and filter out the second type of license plate image information sequence. The maximum value of the second type of license plate similarity value of the corresponding second type of candidate license plate image, and then, from the maximum value of each second type of license plate similarity value obtained by screening, determine the target second type of license plate similarity with the largest value value;

The second license plate similarity judging module is used to judge whether the target second type of license plate similarity value is greater than the preset second similarity threshold, if greater, according to the preset second valid license plate location rule and the calculated first The second type of license plate similarity value, locate at least one valid license plate image from the second type of candidate license plate image targeted by the second type of license plate image information sequence corresponding to the target second type of license plate similarity value, if not greater than, trigger The module for obtaining the third license plate sequence;

The third license plate sequence obtaining module is used to continue to extract the local binary mode LBP feature of the video frame image, and input the LBP feature into the pre-established second classifier to obtain the image similar to the video frame image. The corresponding third type of license plate image information sequence, wherein the third type of license plate image information sequence includes the position information of the third type of candidate license plate image searched in the video frame image, and the second classifier is: based on extracting The obtained local binary pattern LBP feature for each image in the predetermined second sample image collection, and the classifier obtained by using the adaptive lifting algorithm Adaboost algorithm training;

A third license plate image determining module, configured to determine a third type of candidate license plate image in the video frame image based on the position information of the third type of candidate license plate image;

The third license plate similarity calculation module is used to calculate the third type of license plate similarity value of the third type of candidate license plate images, and select the maximum value among the third type of license plate similarity values;

The third license plate similarity judging module is used to judge whether the maximum value of the similarity value of the third type of license plate obtained by screening is greater than the preset third similarity threshold; The rules and the calculated similarity value of the third type of license plate are used to locate at least one valid license plate image from the third type of candidate license plate images, and when the judgment result is not greater than The classifier did not locate a valid license plate image.

Preferably, the first license plate similarity calculation module is used for:

Perform binarization processing on the first type of candidate license plate images;

performing open operation morphological transformation on the first type of candidate license plate images obtained after binarization;

Through the Sobel algorithm, the first type of candidate license plate image after the opening operation morphological transformation is subjected to vertical filtering processing, and the first type vertical gradient image of the first type of candidate license plate image after the opening operation morphological transformation is obtained;

Perform edge detection on the first type of vertical gradient image by Canny algorithm to obtain the first type of edge image;

For the edge extracted in the first type of edge image, perform sextiles according to the vertical direction, and calculate the jump mean value of the edge part pixels in the horizontal direction in each equalization;

An average value of the transition average values is calculated, and the calculated average value is determined as the first type of license plate similarity value of the first type of candidate license plate images.

Preferably, the first license plate similarity judging module includes a function for locating from the first type of candidate license plate images at least A first license plate location submodule of an effective license plate image;

The first license plate positioning submodule is specifically used for:

Determining the first type of candidate license plate image corresponding to the maximum value of the first type of license plate similarity value as the located valid license plate image; The first type of candidate license plate image corresponding to the first type of license plate similarity value of the threshold is determined to be a valid license plate image.

Preferably, the device also includes:

The fourth license plate sequence obtaining module is used to input the Haar feature into the pre-established third classifier to obtain the fourth type of license plate image information sequence corresponding to the video frame image, wherein the fourth type of license plate The image information sequence contains the position information of the fourth type of candidate license plate image searched in the video frame image, and the third classifier is: based on the extracted Haar feature for each image in the predetermined third sample image set , and use the adaptive boosting algorithm Adaboost algorithm to train the classifier;

A fourth license plate image determination module, configured to determine a fourth type of candidate license plate image in the video frame image based on the position information of the fourth type of candidate license plate image;

The fourth license plate similarity calculation module is used to calculate the fourth type of license plate similarity value of the fourth type of candidate license plate image, and filter out the maximum value in the fourth type of license plate similarity value;

The fourth license plate similarity judging module is used to judge whether the maximum value in the fourth type of license plate similarity obtained by screening is greater than the preset fourth similarity threshold; when the judgment result is greater than, according to the preset fourth valid license plate Positioning rules and the calculated similarity value of the fourth type of license plate, at least one valid license plate image is located from the fourth type of candidate license plate image, and when the judgment result is not greater than, it is determined that the first classifier, the second The second classifier and the third classifier did not locate valid license plate images.

Preferably, the device also includes:

The fifth license plate sequence obtaining module is used to input the Haar feature into the pre-established fourth classifier to obtain the fifth type of license plate image information sequence corresponding to the video frame image, wherein the fifth type of license plate The image information sequence includes the position information of the fifth type of candidate license plate image searched in the video frame image, and the fourth classifier is: based on the extracted Haar feature for each image in the fourth sample image set, and The classifier trained by the adaptive promotion algorithm Adaboost algorithm;

A fifth license plate image determination module, configured to determine the fifth type of candidate license plate image in the video frame image based on the position information of the fifth type of candidate license plate image;

The fifth license plate similarity calculation module is used to calculate the fifth type of license plate similarity value of the fifth type of candidate license plate image, and select the maximum value among the fifth type of license plate similarity values;

The fifth license plate similarity judging module is used to judge whether the maximum value in the fifth type of license plate similarity value obtained by screening is greater than the preset fifth similarity threshold; when the judgment result is greater than, according to the preset fifth valid license plate Based on the location rule and the calculated similarity value of the fifth type of license plate, at least one valid license plate image is located from the fifth type of candidate license plate images, and when the judgment result is not greater than that, it is determined that no valid license plate image has been located.

A method and device for locating a license plate provided by an embodiment of the present invention. The method and device design a multi-stage screening mechanism for a plurality of series-connected classifiers to extract a combination of multiple features of the image in the video frame image captured by the camera. feature, which greatly improves the failure or missed detection of the license plate that is prone to occur when using a single feature to locate the license plate. At the same time, when the Haar feature extracted from the video frame image input is input to the first classifier, no candidate that satisfies the condition is located. For the license plate image, the video frame image is further rotated multiple times, and the rotated image is input to the first classifier again, which can improve the positioning failure or missed detection caused by the image shooting angle. Obviously, Searching for candidate license plate images in the video frame image through layer-by-layer classifiers improves the possibility of accurately locating the license plate and reduces the probability of positioning failure or missed detection. Of course, implementing any product or method of the present invention does not necessarily need to achieve all the above-mentioned advantages at the same time.

Illustration

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic flow chart of a license plate location method provided by an embodiment of the present invention;

Fig. 2 provides two effective license plate images located for the embodiment of the present invention;

FIG. 3 is a schematic flow diagram of another license plate location method provided by an embodiment of the present invention;

4 is a schematic flow chart of another license plate location method provided by an embodiment of the present invention;

Figure 5(a) and Figure 5(b) are schematic diagrams of the results of the Hough transform provided by the embodiment of the present invention;

Fig. 6(a)-Fig. 6(d) are schematic diagrams of the results after desulphurization and binarization processing provided by the embodiment of the present invention;

Fig. 7 is a schematic structural diagram of a license plate positioning device provided by an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Fig. 1 is a schematic flow chart of a license plate location method provided by an embodiment of the present invention, the method may include the following steps:

Step S101: extract the Haar Haar feature of the video frame image of the license plate to be located, and input the Haar feature into the pre-established first classifier to obtain the first type of license plate image information sequence corresponding to the video frame image .

Wherein, the first type of license plate image information sequence includes the position information of the first type of candidate license plate images searched in the video frame images.

The "candidate license plate image" in the present invention is the image searched by the classifier from the video frame images obtained from the surveillance video. It should be noted that the candidate license plate images may or may not be license plate images. In addition, if the candidate license plate image located by the subsequent steps is the image of the license plate, it means that the classifier is positioned correctly; , which also indicates that the classifier is positioned correctly, otherwise, it indicates that the classifier has failed to locate.

It should be noted that the "license plate image information sequence" mentioned in the present invention includes the position information of the candidate license plate image searched in the video frame image, for example, the location of the license plate image in the video frame image is represented by pixel coordinates image area. In addition, the sequence of license plate image information includes position information of all candidate license plate images searched by the classifier.

Wherein, the first classifier is: based on the extracted Haar features for each image in the predetermined first sample image set, and using the adaptive boosting algorithm Adaboost algorithm to train the classifier.

In a specific implementation of the present invention, the first classifier can be established in the following manner: first, select positive and negative samples for training the classifier, wherein, there are 1000 license plate images (positive samples) and 1000 non-license plate images (negative samples). samples) 10,000 images; then, the Haar feature of each image in the positive and negative samples is proposed; finally, the extracted Haar feature is input into the Adaboost classifier training program that comes with OpenCV, and according to the false alarm of the established classifier Rate (that is: positioning the non-license plate image as the license plate image), hit rate and other parameters to optimize the established classifier.

It should be noted that the Adaboost classifier can be used for offline training for a feature of the license plate image to obtain a classifier that can locate the license plate. The classifier trained by the Adaboost algorithm has certain feasibility and reliability in the license plate location. In addition, in the process of training or establishing the classifier, it is best to make the training sample images as large as possible and the sample images as diverse as possible, so that the efficiency of the classifier established in practical applications is better.

It should also be noted that the above-mentioned classifiers trained based on the Adaboost algorithm are only examples, and the present invention does not need to limit the specific algorithm for establishing the classifier, and of course there are other ways of establishing the classifier, for example, Any possible implementation of the SVM-based classifier building method can be applied to the present invention, and those skilled in the art need to determine which method to use to build the classifier according to the actual situation.

Step S102: Based on the position information of the first type of candidate license plate images, determine the first type of candidate license plate images in the video frame images.

Step S103: Calculate the first type of license plate similarity values of the first type of candidate license plate images, and filter out the maximum value among the first type of license plate similarity values.

In a specific implementation of the present invention, the process of calculating the similarity value of the first type of license plate of any first type of candidate license plate images may include the following steps:

A: Perform binarization processing on the first type of candidate license plate images;

B: Perform open operation morphological transformation on the first type of candidate license plate images obtained after binarization processing;

C: Carry out vertical filtering processing on the first type candidate license plate image after the opening operation morphological transformation by Sobel algorithm, and obtain the first type vertical gradient image of the first type candidate license plate image after the opening operation morphological transformation;

D: performing edge detection on the first type of vertical gradient image by the Canny algorithm to obtain the first type of edge image;

E: For the edge extracted in the first type of edge image, the edge is divided into six equal parts according to the vertical direction, and the average jump value of the pixel points in the edge part in the horizontal direction is calculated in each equal part;

Wherein, in the edge image obtained through the detection in step D, the pixel value of the pixel point in the edge part is 255, and the pixel value of the pixel point in the non-edge part is 0. Therefore, when traversing the row of pixels in order from the beginning to the end of any height obtained after the six equal parts of step E, there will be an alternate transformation of black and white points. When the pixel values of two consecutive pixels are different It is recorded as a jump, and the number of jumps in this line is counted to obtain the average value of the jumps in the horizontal direction of the edge part pixels in each aliquot.

F: Calculate the average value of the transition average value, and determine the calculated average value as the first type of license plate similarity value of the first type of candidate license plate image.

It should be noted that the above-mentioned process of calculating the first-type license plate similarity value of any first-type candidate license plate image is just an example, and the embodiment of the present invention does not need to limit the calculation process of the license plate similarity value, and any possible implementation methods are can be applied to the present invention.

Step S104: Determine whether the maximum value among the similarity values of the first type of license plate obtained through screening is greater than the preset first similarity threshold, and if the judgment result is greater, perform step S105; otherwise, perform step S106.

Preferably, the first similarity threshold may include: for a single-row license plate, the threshold is set to 13; for a double-row license plate, the threshold is set to 12. It should be noted that the threshold set here is only a preferred mode of the present invention, and the present invention does not need to limit the specific value of the first similarity threshold. As for which threshold to choose, it needs to be determined by those skilled in the art. to determine the actual situation.

Step S105: According to the preset first effective license plate location rule and the calculated similarity value of the first type of license plate, locate at least one valid license plate image from the first type of candidate license plate images, see Figure 2, Figure 2 is The two located valid license plate images provided by the embodiment of the present invention.

In a specific implementation of the present invention, the first type of candidate license plate image corresponding to the maximum value among the first type of license plate similarity values may be determined as the located effective license plate image.

In another specific implementation of the present invention, among the first type of license plate similarity values, the first type of candidate license plate images corresponding to the first type of license plate similarity values greater than the first positioning threshold can be determined as valid positioning. license plate image.

Specifically, the preset first positioning threshold may be set as a multiple of the maximum value max among the similarity values of the first type of license plate, for example: max*70%. Of course, the present invention does not need to specifically limit this multiple, and those skilled in the art need to determine it according to actual application conditions.

For example, suppose: the similarity values of the first type of license plates are arranged in descending order: a=99%, b=95%, c=93%, d=89% and e=85%, preset The first localization threshold is 90%.

Obviously, according to the method mentioned in the first implementation above, the first type of candidate license plate image corresponding to a is determined as the effective license plate image located;

Obviously, according to the method mentioned in the above-mentioned second implementation manner, the first type of candidate license plate images corresponding to a, b and c with a similarity value greater than 90% are determined as the located effective license plate images.

It should be noted that the two specific implementations listed above are only examples. Of course, the present invention also has other possible specific implementations. The present invention does not need to limit the specific implementation of locating valid license plate images. Technicians need to determine according to the specific conditions in the actual application.

Step S106: According to the preset image rotation rules, the video frame image is rotated N times, and the video frame image obtained after each rotation is re-inputted into the first classifier to obtain N second-type license plates sequence of image information.

In a specific implementation of the present invention, the preset image rotation rule may be: rotate the video frame image clockwise within the range of [-20°, 20°], and each rotation is 2°, Get a new video frame image.

It should be noted that since the angles of the license plate images in the video frame images captured in non-traffic checkpoint scenes are very different, the license plate image may not be located or the license plate image may be misplaced during the positioning process of the license plate image. Judged as the situation of non-license plate image, therefore, the scheme in the present invention rotates the video frame image in the above-mentioned situation, finds the angle most suitable for positioning, so that the first classifier established in advance can be used for the license plate with too large inclination angle Images are positioned as accurately as possible. It should also be noted that in the present invention, there is no need to limit the allowable range of the video frame image rotation and the specific value of the angle rotated each time. Those skilled in the art need to make reasonable adjustments according to the specific conditions in the actual application. set up.

Step S107: Based on the position information of the second type of candidate license plate images in the second type of license plate image information sequence, determine the second type of candidate license plate images in the video frame images.

Step S108: Calculate the second-type license plate similarity value of the second-type candidate license plate image corresponding to each second-type license plate image information sequence, and filter out the second-type candidate license plate corresponding to each second-type license plate image information sequence The maximum value among the second-type license plate similarity values of the image, and then, from the maximum values among the second-type license plate similarity values obtained through screening, determine the target second-type license plate similarity value with the largest value.

Step S109: Determine whether the similarity value of the target second-type license plate is greater than a preset second similarity threshold, if so, perform step S110, otherwise perform step S11.

Step S110: According to the preset second effective license plate location rule and the calculated similarity value of the second type of license plate, from the second type of license plate image information sequence corresponding to the target second type of license plate similarity value At least one valid license plate image is located in the candidate license plate image.

It should be noted that steps S107-S110 are similar to steps S102-S105, and will not be repeated here.

Step S111: continue to extract the local binary pattern LBP feature of the video frame image, and input the LBP feature into the pre-established second classifier to obtain the third type of license plate image information corresponding to the video frame image sequence.

Wherein, the third type of license plate image information sequence includes the position information of the third type of candidate license plate images searched in the video frame images.

Wherein, the second classifier is: based on the extracted local binary pattern LBP feature for each image in the predetermined second sample image set, and adopting the adaptive boosting algorithm Adaboost algorithm to train the classifier. Compared with the first classifier, the set of sample images selected when training or establishing the second classifier is different, and the features of the set of sample images extracted are different.

It should be noted that the classifiers trained based on the Adaboost algorithm mentioned above are only examples, and the present invention does not need to limit the specific algorithm for establishing classifiers. Of course, there can also be other ways of establishing classifiers, for example, based on support The method for establishing a classifier of a vector machine SVM, any possible implementation manner can be applied to the present invention, and those skilled in the art need to determine which method to use to establish a classifier according to the actual situation.

Step S112: Based on the position information of the third type of candidate license plate images, determine the third type of candidate license plate images in the video frame images.

Step S113: Calculate the third type of license plate similarity values of the third type of candidate license plate images, and filter out the maximum value among the third type of license plate similarity values.

Step S114: Judging whether the maximum value of the similarity value of the third type of license plate obtained through screening is greater than the preset third similarity threshold; Three types of license plate similarity values, at least one valid license plate image is located from the third type of candidate license plate image, and when the judgment result is not greater than, it is determined that no valid license plate has been located by the first classifier and the second classifier image.

It should be noted that steps S112-S114 are similar to steps S102-S104 and will not be repeated here.

In the embodiment of the present invention, by designing a plurality of series-connected classifiers of a multi-level screening mechanism, a comprehensive feature composed of multiple features of the image in the video frame image captured by the camera is extracted, which greatly improves the possibility of using a single feature to locate the license plate. At the same time, when the Haar feature extracted from the video frame image input is input to the first classifier and the candidate license plate image that meets the conditions is not located, the video frame image is further multiplexed. The rotated image is input to the first classifier again, which can improve the positioning failure or missed detection caused by the image shooting angle. Obviously, through the layer-by-layer classifier, the video frame image The search of candidate license plate images improves the possibility of accurately locating the license plate and reduces the probability of positioning failure or missed detection.

Referring to Figure 3, in order to improve the accuracy of license plate image location, on the basis of the license plate location method shown in Figure 1, the following steps may also be included:

Step S115: Input the Haar feature into the pre-established third classifier to obtain the fourth type of license plate image information sequence corresponding to the video frame image.

Wherein, the fourth type of license plate image information sequence includes the position information of the fourth type of candidate license plate images searched in the video frame images.

Wherein, the third classifier is: based on the extracted Haar features for each image in the predetermined third sample image set, and using the adaptive boosting algorithm Adaboost algorithm to train the classifier. Compared with the first classifier, the sample image sets selected when training or establishing the third classifier are different, and the features of the extracted sample image sets are the same.

In addition, in the process of training or establishing the classifier, it is best to make the training sample images as large as possible and the sample images as diverse as possible, so that the efficiency of the classifier established in practical applications is better.

In a specific implementation of the present invention, the third classifier can be established in the following manner: first, select positive and negative samples for training the classifier, wherein, there are 3000 license plate images (positive samples) and 3000 non-license plate images (negative samples). samples) 10,000 images; then, the Haar feature of each image in the positive and negative samples is proposed; finally, the extracted Haar feature is input into the Adaboost classifier training program that comes with OpenCV, and according to the false alarm of the established classifier Rate (that is: positioning the non-license plate image as the license plate image), hit rate and other parameters to optimize the established classifier.

It should be noted that the classifiers trained based on the Adaboost algorithm mentioned above are only examples, and the present invention does not need to limit the specific algorithm for establishing classifiers. Of course, there can also be other ways of establishing classifiers, for example, based on support The method for establishing a classifier of a vector machine SVM, any possible implementation manner can be applied to the present invention, and those skilled in the art need to determine which method to use to establish a classifier according to the actual situation.

Step S116: Based on the position information of the fourth type of candidate license plate images, determine the fourth type of candidate license plate images in the video frame images.

Step S117: Calculate the fourth-type license plate similarity values of the fourth-type candidate license plate images, and filter out the maximum value among the fourth-type license plate similarity values.

Step S118: Judging whether the maximum value among the similarity values of the fourth type of license plate obtained by screening is greater than the preset fourth similarity threshold; The fourth type of license plate similarity value, at least one valid license plate image is located from the fourth type of candidate license plate images, and when the judgment result is not greater than, it is determined that the first classifier, the second classifier and the third classifier The device does not locate a valid license plate image.

It should be noted that steps S116-S118 are similar to steps S102-S104, and will not be repeated here.

In the embodiment of the present invention, by designing a plurality of series-connected classifiers of a multi-level screening mechanism, a comprehensive feature composed of multiple features of the image in the video frame image captured by the camera is extracted, which greatly improves the possibility of using a single feature to locate the license plate. At the same time, when the Haar feature extracted from the video frame image input is input to the first classifier and the candidate license plate image that meets the conditions is not located, the video frame image is further multiplexed. The rotated image is input to the first classifier again, which can improve the positioning failure or missed detection caused by the image shooting angle. Obviously, through the layer-by-layer classifier, the video frame image The search of candidate license plate images improves the possibility of accurately locating the license plate and reduces the probability of positioning failure or missed detection.

Further, referring to Fig. 4, on the basis of the license plate location method shown in Fig. 3, the following steps may also be included:

Step S119: Input the Haar feature into the pre-established fourth classifier to obtain the fifth type of license plate image information sequence corresponding to the video frame image.

Wherein, the fifth type of license plate image information sequence includes the position information of the fifth type of candidate license plate images searched in the video frame images.

Wherein, the fourth classifier is: based on the extracted Haar features for each image in the fourth sample image set, and using the adaptive boosting algorithm Adaboost algorithm to train the classifier. Compared with the first classifier, the sample image sets selected when training or establishing the fourth classifier are different, and the features of the sample image sets extracted are the same.

In addition, in the process of training or establishing the classifier, it is best to make the number of training sample images as large as possible, and the sample images are as diverse as possible, so that the efficiency of the classifier established in practical applications is better. .

In a specific implementation of the present invention, the fourth classifier can be established in the following manner: first, select positive and negative samples for training the classifier, wherein, there are 2000 license plate images (positive samples), and the license plate images All positive samples are double-row license plate images, and 10,000 non-license plate images (negative samples); then, the Haar features of each image in the positive and negative samples are proposed; finally, the extracted Haar features are input to the Adaboost classification that comes with OpenCV Operate in the machine training program, optimize the established classifier according to the false alarm rate of the established classifier (that is: the non-license plate image is positioned as the license plate image), hit rate and other parameters.

It should be noted that the classifiers trained based on the Adaboost algorithm mentioned above are only examples, and the present invention does not need to limit the specific algorithm for establishing classifiers. Of course, there can also be other ways of establishing classifiers, for example, based on support The method for establishing a classifier of a vector machine SVM, any possible implementation manner can be applied to the present invention, and those skilled in the art need to determine which method to use to establish a classifier according to the actual situation.

Step S120: Based on the position information of the fifth type of candidate license plate images, determine the fifth type of candidate license plate images in the video frame images.

Step S121: Calculate the fifth type of license plate similarity values of the fifth type of candidate license plate images, and filter out the maximum value among the fifth type of license plate similarity values.

Step S122: Judging whether the maximum value among the similarity values of the fifth type of license plate obtained by screening is greater than the preset fifth similarity threshold; For the fifth type of license plate similarity value, at least one valid license plate image is located from the fifth type of candidate license plate images, and when the judgment result is not greater than that, it is determined that no valid license plate image has been located.

It should be noted that steps S120-S122 are similar to steps S102-S104 and will not be repeated here.

In the embodiment of the present invention, by designing a plurality of series-connected classifiers of a multi-level screening mechanism, a comprehensive feature composed of multiple features of the image in the video frame image captured by the camera is extracted, which greatly improves the possibility of using a single feature to locate the license plate. At the same time, when the Haar feature extracted from the video frame image input is input to the first classifier and the candidate license plate image that meets the conditions is not located, the video frame image is further multiplexed. The rotated image is input to the first classifier again, which can improve the positioning failure or missed detection caused by the image shooting angle. Obviously, through the layer-by-layer classifier, the video frame image The search of candidate license plate images improves the possibility of accurately locating the license plate and reduces the probability of positioning failure or missed detection.

In a specific embodiment of the present invention, on the basis of the method for locating the license plate shown in Figure 1, the effective license plate image that is positioned can also be further finely positioned and processed, specifically the following steps can be included:

(1) Carry out decontamination binarization processing to the valid license plate image that locates, obtain the valid license plate image that only includes the license plate part;

(2) Utilize the Hough transform Hough transform to carry out the edge detection of the license plate area on the effective license plate image after decluttering, and extract the edge of the license plate area, see Fig. 5, Fig. 5 (a) and Fig. 5 (b) are this The schematic diagram of the results after the Hough transform provided by the embodiment of the invention;

(3) According to the edge of the license plate area extracted, calculate the license plate area in the horizontal and/or vertical direction inclination angle;

(4) Based on the calculated inclination angle, perform horizontal correction and miscut correction processing on the image of the license plate area to obtain the corrected license plate image;

(5) Perform vertical gradient processing on the obtained corrected license plate image in the vertical direction to determine the upper and lower boundaries of the corrected license plate image, and at the same time, preliminarily determine the corrected license plate according to the comparison with the preset threshold The left and right borders of the image;

(6) Using the flooding method to remove the edge fragments in the four determined upper, lower, left, and right boundaries, and optimized the determined upper, lower, left, and right boundaries of the license plate image.

Wherein, the "removal of impurities and binarization processing" mentioned in the present invention may mainly include the following steps:

1) Determine whether the vehicle body color corresponding to the license plate image is a light color according to the sum of the pixel values of the pixels in the top area of the grayscale image of the license plate image after widening.

In a specific implementation of the present invention, the license plate image that has determined the four boundaries of up, down, left, and right can be expanded by 0.125 times the license plate height up, down, and up, and 0.125 times the license plate width in the left and right direction; after setting the widened If the sum of the gray value of the image at the top 0.125 times the height of the license plate image is greater than 50% of the sum of the gray value when the size of the image in this area is all pure white pixels 255, it is judged as a light-colored body, otherwise it is judged as Dark body.

2) According to the obtained vehicle body color, binarize the valid license plate image corresponding to the license plate image to obtain a binarized image A, see FIG. 6( a ).

In a specific implementation of the present invention, it may include:

When being judged as light-colored vehicle body in step 1), select the S channel of the color image of the valid license plate image corresponding to this license plate image in the HSV color space, carry out binarization processing to the valid license plate image corresponding to this S channel;

When it is judged as a dark car body in step 1), it needs to be processed in combination with the color of the license plate as follows: if it is a blue license plate, select the valid license plate image corresponding to the B channel in the RGB color space to perform binarization processing; if it is a yellow license plate Or a white license plate, select the G and R channels in the RGB color space to merge, and perform binarization on the combined effective license plate image.

3) Select the b channel in the Lab color space of the color image of the effective license plate image corresponding to the license plate image, and perform binarization processing to obtain a binarized image B, see FIG. 6(b).

4) Compare the binary image A obtained in step 2) with the pixel at the corresponding position in the binary image B obtained in step 3), if the pixel values of the corresponding pixel are consistent, then the two The pixel value of the corresponding pixel in the valued image A is set to 255.

5) Calculating step 4) the ratio of the blank pixels in the processed binary image A, when the ratio is greater than the preset blank threshold, it is determined to replace the binary image A with the binary image B For subsequent processing, see Figure 6(c), otherwise the binary image A is still used for subsequent processing.

Preferably, the preset blank threshold can be set to 90%. Of course, the present invention does not need to limit the specific value of the threshold, and those skilled in the art need to set it reasonably according to the actual situation in the actual application.

6) Use the flooding method to remove the non-connected miscellaneous areas in the binarized image determined in step 5), see FIG. 6(d), and complete the fine positioning processing of the located effective license plate image.

Fig. 7 is a schematic structural diagram of a license plate positioning device provided by an embodiment of the present invention, the device may include the following modules:

The first license plate sequence obtaining module 201 is used to extract the Haar Haar feature of the video frame image of the license plate to be located, and input the Haar feature into the pre-established first classifier to obtain the corresponding image of the video frame The first type of license plate image information sequence.

Wherein, the first type of license plate image information sequence includes the position information of the first type of candidate license plate images searched in the video frame images.

Wherein, the first classifier is: based on the extracted Haar features for each image in the predetermined first sample image set, and using the adaptive boosting algorithm Adaboost algorithm to train the classifier.

The first license plate image determining module 202 is configured to determine the first type of candidate license plate images in the video frame images based on the position information of the first type of candidate license plate images.

The first license plate similarity calculation module 203 is configured to calculate the first type of license plate similarity values of the first type of candidate license plate images, and filter out the maximum value among the first type of license plate similarity values.

In a specific implementation of the present invention, the first license plate similarity calculation module 203 can be used for:

A: Perform binarization processing on the first type of candidate license plate images;

B: Perform open operation morphological transformation on the first type of candidate license plate images obtained after binarization processing;

C: Carry out vertical filtering processing on the first type candidate license plate image after the opening operation morphological transformation by Sobel algorithm, and obtain the first type vertical gradient image of the first type candidate license plate image after the opening operation morphological transformation;

D: performing edge detection on the first type of vertical gradient image by the Canny algorithm to obtain the first type of edge image;

E: For the edge extracted in the first type of edge image, the edge is divided into six equal parts according to the vertical direction, and the average jump value of the pixel points in the edge part in the horizontal direction is calculated in each equal part;

Wherein, in the edge image obtained through the detection in step D, the pixel value of the pixel point in the edge part is 255, and the pixel value of the pixel point in the non-edge part is 0. Therefore, when traversing the row of pixels in order from the beginning to the end of any height obtained after the six equal parts of step E, there will be an alternate transformation of black and white points. When the pixel values of two consecutive pixels are different It is recorded as a jump, and the number of jumps in this line is counted to obtain the average value of the jumps in the horizontal direction of the edge part pixels in each aliquot.

F: Calculate the average value of the transition average value, and determine the calculated average value as the first type of license plate similarity value of the first type of candidate license plate image.

It should be noted that the above-mentioned process of calculating the first-type license plate similarity value of any first-type candidate license plate image is just an example, and the embodiment of the present invention does not need to limit the calculation process of the license plate similarity value, and any possible implementation methods are can be applied to the present invention.

The first license plate similarity judging module 204 is used to judge whether the maximum value in the first type of license plate similarity obtained by screening is greater than the preset first similarity threshold; when the judgment result is greater than, trigger the first license plate location submodule 205, otherwise trigger the second license plate sequence obtaining module 206.

The first license plate location sub-module 205 is configured to locate at least one valid license plate image from the first type of candidate license plate images according to the preset first valid license plate location rule and the calculated similarity value of the first type of license plate.

Wherein, the first license plate positioning submodule 205 is specifically used for:

Determining the first type of candidate license plate image corresponding to the maximum value of the first type of license plate similarity value as the located valid license plate image; The first type of candidate license plate image corresponding to the first type of license plate similarity value of the threshold is determined to be a valid license plate image.

The second license plate sequence obtaining module 206 is configured to perform N rotations on the video frame image according to a preset image rotation rule, and re-input the video frame image obtained after each rotation into the first classifier, Obtain N second-type license plate image information sequences.

The second license plate image determining module 207 is configured to determine the second type of candidate license plate images in the video frame images based on the position information of the second type of candidate license plate images in the second type of license plate image information sequence.

The second license plate similarity calculation module 208 is used to calculate the second type of license plate similarity value of the second type of candidate license plate image corresponding to each second type of license plate image information sequence, and filter out each second type of license plate image information sequence The maximum value of the second-type license plate similarity value of the corresponding second-type candidate license plate image, and then, from the maximum value among the second-type license plate similarity values obtained by screening, determine the target second-type license plate with the largest value similar value.

The second license plate similarity judging module 209 is used to judge whether the target second type of license plate similarity value is greater than the preset second similarity threshold, if greater, trigger the second license plate positioning submodule 210, otherwise trigger the third license plate sequence Obtain module 211 .

The second license plate location sub-module 210 is used to obtain the second type of license plate image corresponding to the target second type of license plate similarity value according to the preset second valid license plate location rule and the calculated similarity value of the second type of license plate. At least one valid license plate image is located in the second type of candidate license plate images targeted by the information sequence.

The third license plate sequence obtaining module 211 is used to continue to extract the local binary mode LBP feature of the video frame image, and input the LBP feature into the pre-established second classifier to obtain the corresponding image corresponding to the video frame image. The third type of license plate image information sequence.

Wherein, the third type of license plate image information sequence includes the position information of the third type of candidate license plate images searched in the video frame images.

Wherein, the second classifier is: based on the extracted local binary pattern LBP feature for each image in the predetermined second sample image set, and adopting the adaptive boosting algorithm Adaboost algorithm to train the classifier.

A third license plate image determining module 212, configured to determine a third type of candidate license plate image in the video frame image based on the position information of the third type of candidate license plate image;

The third license plate similarity calculation module 213 is used to calculate the third type of license plate similarity value of the third type of candidate license plate image, and filter out the maximum value in the third type of license plate similarity value;

The third license plate similarity judging module 214, is used for judging whether the maximum value of the third type of license plate similarity obtained by screening is greater than the preset third similarity threshold; Positioning rules and the calculated similarity value of the third type of license plate, at least one valid license plate image is located from the third type of candidate license plate images, and when the judgment result is not greater than The binary classifier did not locate a valid license plate image.

In the embodiment of the present invention, by designing a plurality of series-connected classifiers of a multi-level screening mechanism, a comprehensive feature composed of multiple features of the image in the video frame image captured by the camera is extracted, which greatly improves the possibility of using a single feature to locate the license plate. At the same time, when the Haar feature extracted from the video frame image input is input to the first classifier and the candidate license plate image that meets the conditions is not located, the video frame image is further multiplexed. The rotated image is input to the first classifier again, which can improve the positioning failure or missed detection caused by the image shooting angle. Obviously, through the layer-by-layer classifier, the video frame image The search of candidate license plate images improves the possibility of accurately locating the license plate and reduces the probability of positioning failure or missed detection.

In a specific embodiment of the present invention, in order to improve the accuracy of license plate image positioning, on the basis of the license plate positioning device shown in Figure 7, the following modules can also be included:

The fourth license plate sequence obtaining module is used to input the Haar feature into the pre-established third classifier to obtain the fourth type of license plate image information sequence corresponding to the video frame image.

Wherein, the fourth type of license plate image information sequence includes the position information of the fourth type of candidate license plate images searched in the video frame images.

Wherein, the third classifier is: based on the extracted Haar features for each image in the predetermined third sample image set, and using the adaptive boosting algorithm Adaboost algorithm to train the classifier.

The fourth license plate image determining module is configured to determine the fourth type of candidate license plate image in the video frame images based on the position information of the fourth type of candidate license plate image.

The fourth license plate similarity calculation module is used to calculate the fourth type of license plate similarity value of the fourth type of candidate license plate images, and select the maximum value among the fourth type of license plate similarity values.

The fourth license plate similarity judging module is used to judge whether the maximum value in the fourth type of license plate similarity obtained by screening is greater than the preset fourth similarity threshold; when the judgment result is greater than, according to the preset fourth valid license plate Positioning rules and the calculated similarity value of the fourth type of license plate, at least one valid license plate image is located from the fourth type of candidate license plate image, and when the judgment result is not greater than, it is determined that the first classifier, the second The second classifier and the third classifier did not locate valid license plate images.

In another specific embodiment of the present invention, the device may further include the following modules:

The fifth license plate sequence obtaining module is used to input the Haar feature into the pre-established fourth classifier to obtain the fifth type of license plate image information sequence corresponding to the video frame image.

Wherein, the fifth type of license plate image information sequence includes the position information of the fifth type of candidate license plate images searched in the video frame images.

Wherein, the fourth classifier is: based on the extracted Haar features for each image in the fourth sample image set, and using the adaptive boosting algorithm Adaboost algorithm to train the classifier.

The fifth license plate image determination module is configured to determine the fifth type of candidate license plate images in the video frame images based on the position information of the fifth type of candidate license plate images.

The fifth license plate similarity calculation module is used to calculate the fifth type of license plate similarity value of the fifth type of candidate license plate images, and select the maximum value among the fifth type of license plate similarity values.

The fifth license plate similarity judging module is used to judge whether the maximum value in the fifth type of license plate similarity value obtained by screening is greater than the preset fifth similarity threshold; when the judgment result is greater than, according to the preset fifth valid license plate Based on the location rule and the calculated similarity value of the fifth type of license plate, at least one valid license plate image is located from the fifth type of candidate license plate images, and when the judgment result is not greater than that, it is determined that no valid license plate image has been located.

In the embodiment of the present invention, by designing a plurality of series-connected classifiers of a multi-level screening mechanism, a comprehensive feature composed of multiple features of the image in the video frame image captured by the camera is extracted, which greatly improves the possibility of using a single feature to locate the license plate. At the same time, when the Haar feature extracted from the video frame image input is input to the first classifier and the candidate license plate image that meets the conditions is not located, the video frame image is further multiplexed. The rotated image is input to the first classifier again, which can improve the positioning failure or missed detection caused by the image shooting angle. Obviously, through the layer-by-layer classifier, the video frame image The search of candidate license plate images improves the possibility of accurately locating the license plate and reduces the probability of positioning failure or missed detection.

As for the system or device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to part of the description of the method embodiments.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. any such actual relationship or order exists between them. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Those of ordinary skill in the art can understand that all or part of the steps in the implementation of the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, referred to herein as Storage media, such as: ROM/RAM, disk, CD, etc.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims (10)

1. A license plate location method, characterized in that said method comprises: Extract the Haar Haar feature of the video frame image of the license plate to be located, and input the Haar feature into the pre-established first classifier to obtain the first type of license plate image information sequence corresponding to the video frame image, wherein, The first type of license plate image information sequence includes the position information of the first type of candidate license plate image searched in the video frame image, and the first classifier is: based on the extracted first sample image set for a predetermined The Haar features of each image, and the classifier trained by the adaptive promotion algorithm Adaboost algorithm; Based on the position information of the first type of candidate license plate images, determine the first type of candidate license plate images in the video frame images; Calculate the first type of license plate similarity value of the first type of candidate license plate image, and filter out the maximum value in the first type of license plate similarity value; Judging whether the maximum value of the similarity values of the first type of license plate obtained by screening is greater than the preset first similarity threshold: When the judgment result is greater than, at least one effective license plate image is located from the first type of candidate license plate images according to the preset first valid license plate location rule and the calculated similarity value of the first type of license plate. When is not greater than, according to the preset image rotation rules, the video frame image is rotated N times, and the video frame image obtained after each rotation is re-inputted into the first classifier to obtain N second classifiers. A similar license plate image information sequence, based on the position information of the second type of candidate license plate image in the second type of license plate image information sequence, determine the second type of candidate license plate image in the video frame image, and calculate each second type of license plate image The second type of license plate similarity value of the second type of candidate license plate image corresponding to the image information sequence, and filter out the second type of license plate similarity value of the second type of candidate license plate image corresponding to each second type of license plate image information sequence The maximum value, and then, from the maximum value among the second-type license plate similarity values obtained by screening, determine the target second-type license plate similarity value with the largest value, and judge whether the target second-type license plate similarity value is greater than the preset If the second similarity threshold is greater than the second type of license plate image information corresponding to the target second type of license plate similarity value, according to the preset second valid license plate location rule and the calculated second type of license plate similarity value At least one valid license plate image is located in the second type of candidate license plate images targeted by the sequence, if not greater than Continue to extract the local binary pattern LBP feature of the video frame image, and input the LBP feature into the pre-established second classifier to obtain the third type of license plate image information sequence corresponding to the video frame image, wherein , the third type of license plate image information sequence includes the position information of the third type of candidate license plate image searched in the video frame image, and the second classifier is: based on the extracted second sample image set for a predetermined The local binary mode LBP feature of each image, and the classifier obtained by using the adaptive promotion algorithm Adaboost algorithm training; determining a third type of candidate license plate image in the video frame image based on the position information of the third type of candidate license plate image; Calculate the third type of license plate similarity value of the third type of candidate license plate image, and filter out the maximum value in the third type of license plate similarity value; Judging whether the maximum value of the similarity value of the third type of license plate obtained by screening is greater than the preset third similarity threshold; when the judgment result is greater than, according to the preset third valid license plate positioning rule and the calculated third type of license plate Similarity value, at least one valid license plate image is located from the third type of candidate license plate images, and when the judgment result is not greater than, it is determined that no valid license plate image has been located by the first classifier and the second classifier. 2. method according to claim 1, is characterized in that, calculates the process of the first type license plate similarity value of any first type candidate license plate image, comprises: Perform binarization processing on the first type of candidate license plate images; performing open operation morphological transformation on the first type of candidate license plate images obtained after binarization; Through the Sobel algorithm, the first type of candidate license plate image after the opening operation morphological transformation is subjected to vertical filtering processing, and the first type vertical gradient image of the first type of candidate license plate image after the opening operation morphological transformation is obtained; Perform edge detection on the first type of vertical gradient image by Canny algorithm to obtain the first type of edge image; For the edge extracted in the first type of edge image, perform sextiles according to the vertical direction, and calculate the jump mean value of the edge part pixels in the horizontal direction in each equalization; An average value of the transition average values is calculated, and the calculated average value is determined as the first type of license plate similarity value of the first type of candidate license plate images. 3. The method according to claim 1, wherein, according to the preset first valid license plate location rule and the calculated similarity value of the first type of license plate, the first type of candidate license plate image is located to at least one valid license plate image, including: Determining the first type of candidate license plate image corresponding to the maximum value in the first type of license plate similarity value as the located effective license plate image; or Among the first type of license plate similarity values, the first type of candidate license plate images corresponding to the first type of license plate similarity values greater than the preset first positioning threshold are determined as valid license plate images located. 4. The method according to claim 1, characterized in that, after said determining that no valid license plate image is located by said first classifier and second classifier, said method further comprises: Input the Haar feature into the pre-established third classifier to obtain a fourth type of license plate image information sequence corresponding to the video frame image, wherein the fourth type of license plate image information sequence includes the video frame image information sequence The position information of the searched fourth type of candidate license plate image, the third classifier is: based on the Haar feature extracted for each image in the predetermined third sample image set, and adopts the adaptive promotion algorithm Adaboost algorithm The trained classifier; Based on the position information of the fourth type of candidate license plate image, determine the fourth type of candidate license plate image in the video frame image; Calculate the fourth type of license plate similarity value of the fourth type of candidate license plate image, and filter out the maximum value in the fourth type of license plate similarity value; Judging whether the maximum value of the similarity values of the fourth type of license plate obtained by screening is greater than the preset fourth similarity threshold; license plate similarity value, locate at least one valid license plate image from the fourth type of candidate license plate images, and when the judgment result is not greater than, determine that the first classifier, the second classifier and the third classifier are not located to a valid license plate image. 5. The method according to claim 4, characterized in that, after the determination is not located to a valid license plate image by the first classifier, the second classifier and the third classifier, the method further comprises: Input the Haar feature into the pre-established fourth classifier to obtain the fifth type of license plate image information sequence corresponding to the video frame image, wherein the fifth type of license plate image information sequence includes the video frame image information sequence The position information of the fifth type of candidate license plate images searched, the fourth classifier is: based on the extracted Haar features for each image in the fourth sample image set, and using the adaptive promotion algorithm Adaboost algorithm training to obtain classifier; Based on the position information of the fifth type of candidate license plate image, determine the fifth type of candidate license plate image in the video frame image; Calculate the fifth type of license plate similarity value of the fifth type of candidate license plate image, and filter out the maximum value in the fifth type of license plate similarity value; Judging whether the maximum value of the similarity value of the fifth type of license plate obtained by screening is greater than the preset fifth similarity threshold; License plate similarity value, at least one valid license plate image is located from the fifth type of candidate license plate images, and when the judgment result is not greater than, it is determined that no valid license plate image has been located. 6. A license plate location device, characterized in that said device comprises: The first license plate sequence obtaining module is used to extract the Haar Haar feature of the video frame image of the license plate to be located, and input the Haar feature into the pre-established first classifier to obtain the first corresponding to the video frame image. A type of license plate image information sequence, wherein the first type of license plate image information sequence includes the position information of the first type of candidate license plate image searched in the video frame image, and the first classifier is: based on the extracted Haar features of each image in the predetermined first sample image set, and a classifier obtained by using the adaptive promotion algorithm Adaboost algorithm training; The first license plate image determination module is configured to determine the first type of candidate license plate images in the video frame images based on the position information of the first type of candidate license plate images; The first license plate similarity calculation module is used to calculate the first type of license plate similarity value of the first type of candidate license plate image, and filter out the maximum value in the first type of license plate similarity value; The first license plate similarity judging module is used to judge whether the maximum value in the first type of license plate similarity obtained by screening is greater than the preset first similarity threshold; when the judgment result is greater than, according to the preset first valid license plate Positioning rules and the calculated similarity value of the first type of license plate, locating at least one valid license plate image from the first type of candidate license plate images, and triggering the second license plate sequence acquisition module when the judgment result is not greater than; The second license plate sequence obtaining module is configured to perform N rotations on the video frame image according to a preset image rotation rule, and re-input the video frame image obtained after each rotation into the first classifier , to obtain N second-type license plate image information sequences; The second license plate image determining module is configured to determine the second type of candidate license plate image in the video frame image based on the position information of the second type of candidate license plate image in the second type of license plate image information sequence; The second license plate similarity calculation module is used to calculate the second type of license plate similarity value of the second type of candidate license plate image corresponding to each second type of license plate image information sequence, and screen out the second type of license plate image information sequence. The maximum value of the second-type license plate similarity value of the corresponding second-type candidate license plate image, and then, from the maximum value of each second-type license plate similarity value obtained by screening, determine the target second-type license plate similarity with the largest value value; The second license plate similarity judging module is used to judge whether the target second type of license plate similarity value is greater than the preset second similarity threshold, if greater, according to the preset second valid license plate location rule and the calculated first The second type of license plate similarity value, locate at least one valid license plate image from the second type of candidate license plate image targeted by the second type of license plate image information sequence corresponding to the target second type of license plate similarity value, if not greater than, trigger The module for obtaining the third license plate sequence; The third license plate sequence obtaining module is used to continue to extract the local binary mode LBP feature of the video frame image, and input the LBP feature into the pre-established second classifier to obtain the image similar to the video frame image. The corresponding third type of license plate image information sequence, wherein the third type of license plate image information sequence includes the position information of the third type of candidate license plate image searched in the video frame image, and the second classifier is: based on extracting The obtained local binary pattern LBP feature for each image in the predetermined second sample image collection, and the classifier obtained by using the adaptive lifting algorithm Adaboost algorithm training; A third license plate image determining module, configured to determine a third type of candidate license plate image in the video frame image based on the position information of the third type of candidate license plate image; The third license plate similarity calculation module is used to calculate the third type of license plate similarity value of the third type of candidate license plate images, and select the maximum value among the third type of license plate similarity values; The third license plate similarity judging module is used to judge whether the maximum value of the third type of license plate similarity obtained by screening is greater than the preset third similarity threshold; The rules and the calculated similarity value of the third type of license plate are used to locate at least one valid license plate image from the third type of candidate license plate images, and when the judgment result is not greater than The classifier did not locate a valid license plate image. 7. The device according to claim 6, wherein the first license plate similarity calculation module is used for: Perform binarization processing on the first type of candidate license plate images; performing open operation morphological transformation on the first type of candidate license plate images obtained after binarization; Through the Sobel algorithm, the first type of candidate license plate image after the opening operation morphological transformation is subjected to vertical filtering processing, and the first type vertical gradient image of the first type of candidate license plate image after the opening operation morphological transformation is obtained; Perform edge detection on the first type of vertical gradient image by Canny algorithm to obtain the first type of edge image; For the edge extracted in the first type of edge image, perform sextiles according to the vertical direction, and calculate the jump mean value of the edge part pixels in the horizontal direction in each equalization; An average value of the transition average values is calculated, and the calculated average value is determined as the first type of license plate similarity value of the first type of candidate license plate images. 8. The device according to claim 6, wherein the first license plate similarity judging module comprises a method for selecting from the first type of license plate similarity value according to the preset first valid license plate location rule and the calculated similarity value of the first type of license plate. The first license plate positioning submodule that locates at least one valid license plate image in the first type of candidate license plate images; The first license plate positioning submodule is specifically used for: Determining the first type of candidate license plate image corresponding to the maximum value of the first type of license plate similarity value as the located valid license plate image; The first type of candidate license plate image corresponding to the first type of license plate similarity value of the threshold is determined to be a valid license plate image. 9. The device according to claim 6, further comprising: The fourth license plate sequence obtaining module is used to input the Haar feature into the pre-established third classifier to obtain the fourth type of license plate image information sequence corresponding to the video frame image, wherein the fourth type of license plate The image information sequence contains the position information of the fourth type of candidate license plate image searched in the video frame image, and the third classifier is: based on the extracted Haar feature for each image in the predetermined third sample image set , and use the adaptive boosting algorithm Adaboost algorithm to train the classifier; A fourth license plate image determination module, configured to determine a fourth type of candidate license plate image in the video frame image based on the position information of the fourth type of candidate license plate image; The fourth license plate similarity calculation module is used to calculate the fourth type of license plate similarity value of the fourth type of candidate license plate image, and filter out the maximum value in the fourth type of license plate similarity value; The fourth license plate similarity judging module is used to judge whether the maximum value in the fourth type of license plate similarity obtained by screening is greater than the preset fourth similarity threshold; when the judgment result is greater than, according to the preset fourth valid license plate Positioning rules and the calculated similarity value of the fourth type of license plate, at least one valid license plate image is located from the fourth type of candidate license plate image, and when the judgment result is not greater than, it is determined that the first classifier, the second The second classifier and the third classifier did not locate valid license plate images. 10. The device according to claim 9, further comprising: The fifth license plate sequence obtaining module is used to input the Haar feature into the pre-established fourth classifier to obtain the fifth type of license plate image information sequence corresponding to the video frame image, wherein the fifth type of license plate The image information sequence includes the position information of the fifth type of candidate license plate image searched in the video frame image, and the fourth classifier is: based on the extracted Haar feature for each image in the fourth sample image set, and The classifier trained by the adaptive promotion algorithm Adaboost algorithm; A fifth license plate image determination module, configured to determine the fifth type of candidate license plate image in the video frame image based on the position information of the fifth type of candidate license plate image; The fifth license plate similarity calculation module is used to calculate the fifth type of license plate similarity value of the fifth type of candidate license plate image, and select the maximum value among the fifth type of license plate similarity values; The fifth license plate similarity judging module is used to judge whether the maximum value in the fifth type of license plate similarity value obtained by screening is greater than the preset fifth similarity threshold; when the judgment result is greater than, according to the preset fifth valid license plate Based on the location rule and the calculated similarity value of the fifth type of license plate, at least one valid license plate image is located from the fifth type of candidate license plate images, and when the judgment result is not greater than that, it is determined that no valid license plate image has been located.