CN109829401A - Traffic sign recognition method and device based on double capture apparatus - Google Patents
Traffic sign recognition method and device based on double capture apparatus Download PDFInfo
- Publication number
- CN109829401A CN109829401A CN201910051444.0A CN201910051444A CN109829401A CN 109829401 A CN109829401 A CN 109829401A CN 201910051444 A CN201910051444 A CN 201910051444A CN 109829401 A CN109829401 A CN 109829401A
- Authority
- CN
- China
- Prior art keywords
- traffic sign
- capture apparatus
- image
- traffic
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
This application discloses a kind of traffic sign recognition method and device based on double capture apparatus.Traffic sign recognition method based on double capture apparatus includes: the first image for obtaining the shooting of the first capture apparatus and the second image of the second capture apparatus shooting, and the time that the first capture apparatus shoots the first image is identical as the second capture apparatus shooting time of the second image；The traffic sign in the first image and the second image is identified by Traffic Sign Recognition System；When the first traffic sign and identical the second traffic sign, determine that the first coordinate of the first traffic sign and the second coordinate of the second traffic sign, the first traffic sign actual traffic mark corresponding with the second traffic sign are target traffic sign；Parallax of the target traffic sign in the first capture apparatus and the second capture apparatus is calculated according to the first coordinate and the second coordinate；According to the distance between disparity computation traffic sign and traveling apparatus.The application is able to solve the problem of can not determining the distance between traffic sign and vehicle.
Description
Technical field
This application involves field of computer technology more particularly to a kind of traffic sign recognition methods based on double capture apparatus
And device.
Background technique
The fast development of China's traffic route system keeps people's life more convenient, and driver needs when driving vehicle
Traffic sign is accurately identified, to drive safely according to the prompt of traffic sign, so driver needs when driving vehicle
It wants that traffic sign can be accurately identified.In order to make the identification traffic sign that driver is quick, accurate and effective, as far as possible reduction traffic
The hidden danger of safety would generally be realized by computer technology to Traffic Sign Recognition in the car at present, and the friendship that will identify that
Logical mark feeds back to driver, so as to assist driver fast and accurately to identify traffic sign in vehicle travel process,
Enable a driver to the driving vehicle safer according to traffic sign.
The common method of Traffic Sign Recognition includes that the color characteristic based on image is detected, to obtain traffic sign area
Then the image in domain obtains the shape feature of image target area using the algorithm of central Projection Transformation, what recycling was tested
Probabilistic neural network classifier goes out traffic sign to carry out Classification and Identification.
But only can recognize that traffic sign in current traffic sign recognition method, and can not determine traffic sign
The distance between vehicle, thus make driver can not accurately determine according to traffic sign adjust drive vehicle mode when
Between, and then bring inconvenience to driver.
Summary of the invention
This application provides a kind of traffic sign recognition method and device based on double capture apparatus, being able to solve can not be true
Determine the distance between traffic sign and vehicle, determine driver can not accurately according to traffic sign and adjust driving vehicle side
The time of formula, the problem of bringing inconvenience to driver.
In a first aspect, being set this application provides a kind of traffic sign recognition method based on double capture apparatus for travelling
Standby, the traveling apparatus includes the first capture apparatus and the second capture apparatus, and first capture apparatus and described second shoot
Equipment is in same horizontal line, the device parameter phase of the device parameter of first capture apparatus and second capture apparatus
Together, the traffic sign recognition method includes:
Obtain the first image of the first capture apparatus shooting and the second image of second capture apparatus shooting, institute
The time and second capture apparatus of stating the first capture apparatus shooting the first image shoot the time of second image
It is identical；
The traffic sign in the first image and second image, the friendship are identified by Traffic Sign Recognition System
Logical sign recognition system is training in advance, and the traffic sign includes the first traffic sign and the second traffic sign, described first
Traffic sign is the traffic sign identified in the first image, and second traffic sign is to identify in second image
Traffic sign out；
When first traffic sign is identical with second traffic sign, the first of first traffic sign is determined
Second coordinate of coordinate and the second traffic sign, first traffic sign actual traffic corresponding with second traffic sign
Mark is target traffic sign；
The target traffic sign is calculated in first capture apparatus according to first coordinate and second coordinate
With the parallax in second capture apparatus；
According to the distance between target traffic sign and the traveling apparatus described in the disparity computation.
In the embodiment of the present application, it is provided with the first capture apparatus and the second capture apparatus, obtains the shooting of the first capture apparatus
The first image and the shooting of the second equipment the second image after, identify described the by Traffic Sign Recognition System trained in advance
The traffic sign of one image and second image；When in the first traffic sign and second image in the first image
The second traffic sign it is identical when, determine the first coordinate of first traffic sign and the second coordinate of the second traffic sign,
And target traffic sign is calculated in first capture apparatus and described the according to first coordinate and second coordinate
Parallax in two capture apparatus, and then can be between target traffic sign and the traveling apparatus according to the disparity computation
Distance.Can be determined while identifying traffic sign in such the embodiment of the present application traffic sign and traveling apparatus it
Between distance, so as to so that driver accurately adjusts driving vehicle according to the distance between traffic sign and traffic sign
Mode, significantly more efficient guarantee safe driving.
With reference to first aspect, in the first possible embodiment of first aspect, known described by traffic sign
Before traffic sign in other system identification the first image and second image, further includes:
Traffic Sign Images set is obtained, in each Traffic Sign Images of the Traffic Sign Images set described in mark
The coordinate and type of traffic sign included by each Traffic Sign Images；
According to the default Traffic Sign Recognition System of Traffic Sign Images set training, the Traffic Sign Recognition is obtained
System.
With reference to first aspect, described according to the disparity computation in second of possible embodiment of first aspect
The distance between the traffic sign and the traveling apparatus include:
Determine the focal length of first capture apparatus and second capture apparatus；
According to the distance between traffic sign and the traveling apparatus described in the focal length and the disparity computation.
With reference to first aspect, in the third possible embodiment of first aspect, further includes:
Pass through distance described in voice reminder and the target traffic sign.
With reference to first aspect, in the 4th kind of possible embodiment of first aspect, first capture apparatus is double
The left side camera of mesh camera, second capture apparatus are the right side camera of binocular camera.
Second aspect, this application provides a kind of Traffic Sign Recognition devices based on double capture apparatus, including for real
The functional unit of method in each implementation of existing the application first aspect and first aspect.
The third aspect, this application provides a kind of Traffic Sign Recognition equipment based on double capture apparatus, comprising: processor
And memory；
The memory includes instruction for storing computer program code, the computer program code；
The processor is for executing described instruction, so that the Traffic Sign Recognition equipment based on double capture apparatus is real
The now method of the unsafe driving behavioral value as described in any embodiment of first aspect or first aspect.
Fourth aspect is deposited in the computer readable storage medium this application provides a kind of computer readable storage medium
Instruction is contained, when run on a computer, so that computer executes any implementation such as first aspect or first aspect
Traffic sign recognition method based on double capture apparatus described in mode.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application will make below to required in the embodiment of the present application
Attached drawing is briefly described.
Fig. 1 is according to a kind of signal of the traffic sign recognition method based on double capture apparatus provided by the embodiments of the present application
Property flow chart；
Fig. 2 is according to a kind of first capture apparatus provided by the embodiments of the present application, the second capture apparatus and target traffic mark
The schematic diagram of positional relationship between will；
Fig. 3 is according to a kind of signal of the Traffic Sign Recognition device based on double capture apparatus provided by the embodiments of the present application
Block diagram；
Fig. 4 is showing according to another Traffic Sign Recognition device based on double capture apparatus provided by the embodiments of the present application
Meaning block diagram；
Fig. 5 is according to a kind of signal of the Traffic Sign Recognition equipment based on double capture apparatus provided by the embodiments of the present application
Block diagram.
Specific embodiment
In the absence of conflict, the different characteristic in each embodiment and each embodiment in the application can be mutual group
It closes.
The embodiment of the present application is for traveling apparatus in the scene of Traffic Sign Recognition.Traveling apparatus can be specifically as follows
Driving vehicle, such as automobile etc. are illustrated in the embodiment of the present application by taking driving vehicle as an example.It exercises to be arranged in equipment and be used for
The device of Traffic Sign Recognition, the device of Traffic Sign Recognition include the first capture apparatus and the second capture apparatus.Usual first
The shooting direction of capture apparatus and the second capture apparatus is towards the front of traveling apparatus, in order to shoot the friendship exercised in front of equipment
Logical mark.First capture apparatus and the second capture apparatus need to be parallel to horizontal plane setting, and the first capture apparatus and second
Capture apparatus is in same horizontal line, and usual first capture apparatus and the second capture apparatus are in perpendicular to traveling apparatus traveling side
To horizontal line.The device parameter of the device parameter of first capture apparatus and the second capture apparatus is identical, i.e. the first capture apparatus
It is identical capture apparatus with the second capture apparatus, and each parameter setting is identical when shooting.Specific first capture apparatus
It can be camera etc. with the second capture apparatus.First capture apparatus and the second capture apparatus can be independent two and set
Standby, can be set in the left and right sides of traveling apparatus or the first capture apparatus and the second capture apparatus to be an equipment,
Such as binocular capture apparatus (binocular camera) etc..
One embodiment of the application provides a kind of traffic sign recognition method based on double capture apparatus, as shown in Figure 1, should
Method includes the following steps.
101, obtain the first image of the first capture apparatus shooting and the second image of the second equipment shooting.
Wherein, the time that the first capture apparatus shoots the first image shoots the time phase of the second image with the second capture apparatus
Together.
In traveling vehicle travel process, the first capture apparatus and the second capture apparatus can be shot in front of driving vehicle
The image of environment, when traffic sign occurs in front, traffic sign can be photographed simultaneously in two capture apparatus.As hereafter
The continuous traffic sign that can determine to occur in front of driving vehicle by the identification to the first image and the second image.In order to subsequent
The accuracy of calculating process usually obtains the image that the first capture apparatus and the second capture apparatus take simultaneously and is handled.
102, the traffic sign in the first image and the second image is identified by Traffic Sign Recognition System.
Wherein, Traffic Sign Recognition System is training in advance.Traffic sign includes the first traffic sign and the second traffic mark
Will, the first traffic sign are the traffic sign identified in the first image, and the second traffic sign is to identify in the second image
Traffic sign.Traffic Sign Recognition System is trained in the embodiment of the present application in advance, Traffic Sign Recognition System can input
Image in identify traffic sign, the traffic sign identified in the first image be the first traffic sign, in the second image
In the traffic sign that identifies be the second traffic sign.
103, when the first traffic sign and identical the second traffic sign, determine the first coordinate and of the first traffic sign
Second coordinate of two traffic signs.
Wherein, the first capture apparatus and the second capture apparatus are while shooting, so usual first image and the second image
The traffic sign of middle shooting is identical, and then the traffic sign identified is also identical.When the first traffic sign and the second traffic sign
When identical, actual traffic mark corresponding to the first traffic sign and the second traffic sign is target traffic sign.Identify
The coordinate that one traffic sign and the second traffic sign are imaged in both images will be different, it is possible to according to what is identified
The positional relationship between different coordinates and two capture apparatus that traffic sign is imaged in both images determines the traffic
The distance between mark and driving vehicle.
104, target traffic sign is calculated according to the first coordinate and the second coordinate and is set in the first capture apparatus and the second shooting
Parallax in standby.
Wherein, the same traffic sign can have parallax in the first capture apparatus and the second capture apparatus.It is determining
After first coordinate of the first traffic sign and the second coordinate of the second traffic sign, target traffic sign can be calculated first
Parallax in capture apparatus and the second capture apparatus.
For example, as shown in Fig. 2, the first capture apparatus and the second capture apparatus are in same corresponding perpendicular to traveling apparatus
The shooting direction of the horizontal line of driving direction, the first capture apparatus and the second capture apparatus is the driving direction of traveling apparatus, mesh
Mark traffic sign is imaged in the first capture apparatus and the second capture apparatus respectively, i.e. the first image and the second image, first count
Take the photograph equipment and the second capture apparatus.It, will be captured by the first image captured by the first capture apparatus and the second capture apparatus in Fig. 2
The second image be placed in a coordinate system, using the first capture apparatus straight line locating in shooting direction as y-axis, the first shooting
Equipment is identical with the parameter of the second capture apparatus, so the imaging of the first capture apparatus and the second capture apparatus is also at same water
Horizontal line, using straight line where the first image and the second image as x-axis.The distance between first capture apparatus and the second capture apparatus are
B, the distance between the first capture apparatus and the first image are the focal length f of the first capture apparatus, the second capture apparatus and the second figure
The distance between picture is the focal length f of the second capture apparatus.In a coordinate system, target traffic sign, the first traffic sign, second are handed over
Positional relationship between logical mark, the first capture apparatus and the second capture apparatus is as shown in Figure 2.First traffic mark in first image
The coordinate of will can be expressed as (x1, y0), and the coordinate of the second traffic sign can be expressed as (x2, y0) in the second image, then may be used
To show that the size of parallax d can be expressed as d=| x2-x1 |.
105, according to the distance between disparity computation traffic sign and the traveling apparatus.
Wherein, after step 104 calculates parallax, can be calculated based on parallaxometer traffic sign and the first capture apparatus and
The distance between second capture apparatus.Since the first capture apparatus and the second capture apparatus are set on traveling apparatus, so
The distance between traffic sign and traveling apparatus are just calculated.
In the embodiment of the present application, it is provided with the first capture apparatus and the second capture apparatus, obtains the shooting of the first capture apparatus
The first image and the shooting of the second equipment the second image after, pass through Traffic Sign Recognition System trained in advance and identify the first figure
The traffic sign of picture and the second image；When the second traffic sign phase in the first traffic sign and the second image in the first image
Meanwhile determining the first coordinate of the first traffic sign and the second coordinate of the second traffic sign, and according to the first coordinate and
Two coordinates calculate parallax of the target traffic sign in the first capture apparatus and the second capture apparatus, and then can be according to parallaxometer
Calculate the distance between target traffic sign and traveling apparatus.The same of traffic sign can be being identified in such the embodiment of the present application
When determine the distance between traffic sign and traveling apparatus, so as to so that driver according to traffic sign and traffic sign it
Between distance come accurately adjust drive vehicle mode, significantly more efficient guarantee safe driving.
In a kind of embodiment of the embodiment of the present application, step 105 can be executed specifically are as follows: determine the first capture apparatus and
The focal length of second capture apparatus；According to focal length and the distance between disparity computation traffic sign and traveling apparatus.
The distance between traffic sign and traveling apparatus can be based on the parallaxes and first count that step 104 is calculated
The focal length of the focal length and the second capture apparatus of taking the photograph equipment is calculated.The device parameter of first capture apparatus and the second capture apparatus
Device parameter it is identical, so the focal length of the focal length of the first capture apparatus and the second capture apparatus is also identical.
For example, as shown in Fig. 2, the distance between the first capture apparatus and the second capture apparatus can be expressed as b, the first figure
The coordinate of the first traffic sign can be expressed as (x1, y0) as in, and the coordinate of the second traffic sign can indicate in the second image
Coordinate for (x2, y0), target traffic sign can be expressed as (x, y), the focal length of the first capture apparatus and the second capture apparatus
Focal length be represented as f, according to triangle similarity law it can be concluded that distance z such as formula (1) shown in.
According to formula (1) it can be concluded that the calculation formula such as formula (1) of distance z.
It should be noted that d indicates parallax, in the embodiment of the present application, if the first capture apparatus and the second shooting are set
Standby is binocular capture apparatus, such as binocular camera, then b is also represented by the baseline of binocular camera.
It can also include Traffic Sign Recognition system before step 101 in another embodiment of the embodiment of the present application
The training process of system.It specifically include: to obtain Traffic Sign Images set, each traffic indication map of Traffic Sign Images set
The coordinate and type of traffic sign included by each Traffic Sign Images are marked as in；It is instructed according to Traffic Sign Images set
Practice default Traffic Sign Recognition System, obtains Traffic Sign Recognition System.
Default Traffic Sign Recognition System is the network pre-established, specifically, you only can need to see once according to
Third version (The third version of you only look once, yolov3) algorithm is established.Establish default traffic
It after sign recognition system, needs to be trained it, in order to be able to the Traffic Sign Recognition system of traffic sign be recognized accurately
System.
Training needs first to obtain training data, i.e. Traffic Sign Images set when presetting Traffic Sign Recognition System, specifically
It may include the image collection of German traffic signals standard (German Traffic Sign Benchmarks, GTSRB).Traffic
The coordinate and class of traffic sign included by each Traffic Sign Images are marked in each Traffic Sign Images of sign image set
Type, that is to say, that for each Traffic Sign Images in Traffic Sign Images set, need to mark out its included traffic mark
The coordinate and type of will, in order to be trained to default Traffic Sign Recognition System.Mark traffic sign coordinate refer to by
The area marking of included traffic sign comes out in Traffic Sign Images, the region of included traffic sign in Traffic Sign Images
It is generally rectangular, so notation methods can be the coordinate to angular vertex in mark traffic sign region, or mark traffic mark
One vertex in will region and the length and width for marking traffic sign region.The type of mark traffic sign refers to traffic sign
After traffic sign classification in image, all types of identifiers is determined, then the included traffic mark in Traffic Sign Images
The identifier of will mark corresponding types.
It should be noted that before Traffic Sign Images are labeled in Traffic Sign Images set, it can also be pre-
First Traffic Sign Images are handled, such as carry out Image Super Resolution Processing, in order to which preferably traffic mark is preset in training
Will identifying system.It, can also be to traffic mark meanwhile in Traffic Sign Images set before Traffic Sign Images are labeled
The size of will image is handled, and is adjusted to be suitble to the size of the image of the default Traffic Sign Recognition System of training.For example,
When Traffic Sign Images collection is combined into GTSRB, the size of Traffic Sign Images first can be amplified to original 4 times, then by pixel
It is adjusted to 416 × 416.
Specifically, in the default Traffic Sign Recognition System of training, scheme in the Traffic Sign Images set that can will acquire
It is specifically as follows 8:1:1 as being divided into training data, verification data and test data, ratio in proportion.Training data is used for pre-
If Traffic Sign Recognition System training, verification data are tested for verifying to trained default Traffic Sign Recognition System
Data are for testing trained default Traffic Sign Recognition System.Under normal conditions, to default Traffic Sign Recognition system
System is tested in training, in order to adjust the parameter in default Traffic Sign Recognition System in real time.Test data is used for training
Default Traffic Sign Recognition System accuracy rate after a period of time is tested, if the accuracy rate of test can satisfy requirement,
Then indicate that Traffic Sign Recognition System is completed.
For example, for establishing preparatory Traffic Sign Recognition System network according to yolov3 algorithm, including feature extraction net
Network (such as Darknet-53 network) and feature interaction and multi-scale prediction layer.It is that Darknet-53 network is with feature extraction network
Example, Darknet-53 network includes being characterized extraction network from the 0th layer to 74 layers, contains 53 convolutional layers, remaining is residual error
(residual) structure, by available 53 characteristic patterns of Darknet-53 network, including pixel size be 13 × 13,26 ×
The characteristic pattern of 26 and 52 × 52 3 kinds of sizes, the network structure make that deep layer network difficulty is trained to greatly reduce.Feature interaction and
The feature that multi-scale prediction network is 13 × 13,26 × 26 and 52 × 52 3 kinds for pixel size obtained in Darknet-53
Each characteristic pattern in figure carries out local feature interaction by convolution kernel (3 × 3 and 1 × 1) respectively, obtains bounding box prediction
With object Score on Prediction；It then can be using clustering algorithms such as K-means, to obtained institute's bounding box prediction and object score
Prediction is clustered, and 9 cluster centres are obtained, using as priori frame to get to 9 priori frames；Again by obtained priori frame by
Descending arrangement is carried out according to size, 3 priori frames preceding in sequence are distributed into the characteristic pattern that pixel size is 13 × 13, rear 3 elder generations
It tests frame and distributes to the characteristic pattern that pixel size is 52 × 52, remaining 3 priori frames distribute to the spy that pixel size is 26 × 26
(since characteristic pattern is bigger, receptive field is smaller, then more sensitive to Small object, so it is big to select small priori frame to distribute to for sign figure
Characteristic pattern；And since characteristic pattern is smaller, receptive field is bigger, then it is more sensitive to big target, so selecting big priori frame distribution
To small characteristic pattern).After priori frame to be distributed to the characteristic pattern that pixel size is 13 × 13,26 × 26,52 × 52 3 scales,
Bounding box prediction can be carried out, and predicts the object score of each bounding box using logistic regression.
After image inputs preparatory Traffic Sign Recognition System network in Traffic Sign Images set, preparatory traffic sign is inputted
Identifying system network can be first the image grid division of input, and the division mode of grid is carried out according to the size of characteristic pattern, for example,
The pixel size of characteristic pattern is 13 × 13,26 × 26 and 52 × 52, then needs in image grid division big according to pixel respectively
Small is 13 × 13,26 × 26 and 52 × 52 to be divided.The image of input, can be with each grid real border after grid division
The parameter of frame represents this grid, such as passes through the upper left point coordinate of net boundary frame, the width and Grid Edge of net boundary frame
The height of boundary's frame indicates.Subsequently through the processing of preparatory Traffic Sign Recognition System network, it is based on three kinds of different size features
Corresponding priori frame is schemed it can be concluded that corresponding to the predicted boundary frame of each characteristic pattern, be may then pass through non-Maximum method and is handled
The prediction probability of traffic sign corresponding types into the predicted boundary frame and image of final traffic sign.Then according in image
The parameter marked in advance, and by the prediction result of preparatory Traffic Sign Recognition System network, calculate loss function, and root
It determines whether preparatory Traffic Sign Recognition System network trains completion according to whether the value of loss function meets the condition of convergence, restrains
Condition can be arranged according to different scenes.
It should be noted that the identification object in Traffic Sign Recognition System network is commonly referred to as that object (is expressed as
Obj), in the embodiment of the present application, object obj, the reality of object obj can be referred to as training or the traffic mark identified
Box indicates the bounding box of the marked traffic mark of image in Traffic Sign Images set.
Equal side and the errors of preparatory Traffic Sign Recognition System Web vector graphic is as loss function loss, including predicted boundary
Frame loss function coordErr, it hands over and than (Intersection over Union, IOU) loss function iouErr and classification damage
Function clsErr is lost, shown in loss function loss such as formula (3), predicted boundary frame loss function coordErr such as formula (4) institute
Show, shown in IOU loss function iouErr such as formula (5), shown in Classification Loss function clsErr such as formula (6).
Wherein, S2Indicate image divided grid sum, i indicate in the divided grid of image i-th of grid (i's
Value is to be less than or equal to S greater than 02Integer).B is the number of priori frame corresponding to each grid, for example, the embodiment of the present application
In by taking each grid of every kind of characteristic pattern is corresponding with 3 priori frames as an example, then the value of B be 3, j indicate i-th of grid it is corresponding
J-th of priori frame (value of j is the integer for being less than or equal to B greater than 0).xiIndicate the real border frame upper left point of i-th of grid
Abscissa, yiIndicate the real border frame upper left point ordinate of i-th of grid, ωiIndicate the real border frame width of i-th of grid
Degree, hiIndicate the real border frame height degree of i-th of grid, andIndicate the predicted boundary frame upper left point abscissa of i-th of grid,Indicate the predicted boundary frame upper left point ordinate of i-th of grid,Indicate the predicted boundary width of frame of i-th of grid,Table
Show the predicted boundary frame height degree of i-th of grid.CiIndicate the real border frame of i-th of grid priori frame corresponding with i-th of grid
IOU,Indicate the IOU of the predicted boundary frame of i-th of grid priori frame corresponding with i-th of grid,Indicate object obj
Practical box and i-th of grid have corresponding relationship, i.e. the central point of object obj is fallen in i-th of grid；Expression thing
J-th of priori frame of the practical box of body obj and i-th of grid has corresponding relationship, i.e. the center of obj is fallen in i-th of grid,
And it is maximum with the IOU of j-th of priori frame of the grid,Indicate the practical box of object obj and the jth of i-th of grid
A priori frame does not have corresponding relationship.pi(c) the predicted boundary frame for indicating i-th of grid is the prediction probability of classification c,It indicates
The real border frame of i-th of grid is the probability of classification c, it is generally the case that if the real border frame of i-th of grid is classification
C, thenIf the real border frame of i-th of grid is not classification c,λnoobjAnd λcoordIt is for adjusting
Unbalanced hyper parameter between priori frame comprising object and the priori frame not comprising object, usual λnoobjAnd λcoordParameter
Value can be set according to demand, for example, λ can be set in the embodiment of the present applicationcoord=5, λnoobj=0.5 (because practical do not wrap
Priori frame containing object is more, and it is low to will lead to the priori frame contribution comprising object in this way, causes to train unstable, or even can not receive
It holds back).
It should be noted that the Traffic Sign Recognition that can be used after the completion of default Traffic Sign Recognition System training
System, then in a step 102 for when being identified to the traffic sign in the first image and the second image.Pass through traffic sign
The identification of identifying system, it can be deduced that the type of the first image and traffic mark and secondary traffic mark in the second image.
It can also include: to pass through voice reminder after step 105 in another embodiment of the embodiment of the present application
Distance and target traffic sign.
After determining traffic sign and the distance between traffic sign and traveling apparatus, the side of voice can be passed through
Formula reminds driver.
Specifically, after identifying traffic sign and the distance between traffic sign and traveling apparatus, it can be by the two
In conjunction with text semantic description is generated, voice reminder is then carried out.The generating mode of text semantic description can be according to fixed form
It generates, for example, template can be with are as follows: front m meter, there is n flag, it is noted that, m indicates the calculated distance of step 105, n expression mesh
Mark traffic sign.
It should be noted that in the embodiment of the present application, if judging that the first traffic sign and second is handed over before step 103
Logical mark is not identical, and traffic sign can not be usually calculated according to the first traffic sign and the second traffic sign and is set with the traveling
It is the distance between standby, then it can not execute step 103 to step 105, can directly identify at this time by voice reminder the
One traffic sign and the second traffic sign, in order to remind driver safety to drive.
Fig. 3 is according to a kind of Traffic Sign Recognition device 200 based on double capture apparatus provided by the embodiments of the present application
Schematic block diagram, Traffic Sign Recognition device 200 are used for traveling apparatus, and the Traffic Sign Recognition device includes that the first shooting is set
Standby and the second capture apparatus, first capture apparatus and second capture apparatus are in same horizontal line, the first count
It is identical with the device parameter of second capture apparatus to take the photograph equipment, as shown in figure 3, the Traffic Sign Recognition device 200 further include:
Acquiring unit 201, for obtaining the first image and second capture apparatus of the first capture apparatus shooting
Second image of shooting, the time of the first capture apparatus shooting the first image and second capture apparatus shoot institute
The time for stating the second image is identical；
Recognition unit 202, for being identified in the first image and second image by Traffic Sign Recognition System
Traffic sign, the Traffic Sign Recognition System be in advance training, the traffic sign include the first traffic sign and second
Traffic sign, first traffic sign are the traffic sign identified in the first image, and second traffic sign is
The traffic sign identified in second image；
Determination unit 203 determines described for when first traffic sign is identical with second traffic sign
First coordinate of one traffic sign and the second coordinate of the second traffic sign, first traffic sign and the second traffic mark
The corresponding actual traffic mark of will is target traffic sign；
Computing unit 204 exists for calculating the target traffic sign according to first coordinate and second coordinate
Parallax in first capture apparatus and second capture apparatus；
The computing unit 204 is also according between target traffic sign and the traveling apparatus described in the disparity computation
Distance.
In the embodiment of the present application, it is provided with the first capture apparatus and the second capture apparatus, obtains the shooting of the first capture apparatus
The first image and the shooting of the second equipment the second image after, identify described the by Traffic Sign Recognition System trained in advance
The traffic sign of one image and second image；When in the first traffic sign and second image in the first image
The second traffic sign it is identical when, determine the first coordinate of first traffic sign and the second coordinate of the second traffic sign,
And it is calculated between first traffic sign and second traffic sign according to first coordinate and second coordinate
Parallax, and then can the distance between the traffic sign according to the disparity computation and the traveling apparatus.This such Shen
At a distance from can please determining traffic sign between traveling apparatus while identifying traffic sign in embodiment, so as to
It is more efficient in a manner of making driver and accurately adjust driving vehicle according to the distance between traffic sign and traffic sign
Guarantee safe driving.
It is understood that the acquiring unit 201 is also used to obtain Traffic Sign Images set, the traffic indication map
The coordinate and type of traffic sign included by each Traffic Sign Images are marked in each Traffic Sign Images that image set closes；
As shown in figure 4, described device 200 further include:
Training unit 205, for obtaining according to the default Traffic Sign Recognition System of Traffic Sign Images set training
The Traffic Sign Recognition System.
It is understood that the computing unit 204 is specifically used for:
Determine the focal length of first capture apparatus and second capture apparatus；
According to the distance between traffic sign and the traveling apparatus described in the focal length and the disparity computation.
It is understood that as shown in figure 4, described device 200 can also include:
Reminding unit 206, for passing through distance and the target traffic sign described in voice reminder.
It is understood that first capture apparatus is the left side camera of binocular camera, second shooting is set
Standby is the right side camera of binocular camera.
According to the Traffic Sign Recognition device 200 based on double capture apparatus of the embodiment of the present application, can correspond to according to this
Apply for the executing subject in the traffic sign recognition method based on double capture apparatus of embodiment, and based on double capture apparatus
Modules in Traffic Sign Recognition device 200 respectively in order to realize the corresponding process in method shown in Fig. 1, for sake of simplicity,
Details are not described herein.
Fig. 5 is the signal according to a kind of Traffic Sign Recognition equipment 400 based on double capture apparatus of the embodiment of the present application
Property block diagram.As shown in figure 5, equipment 400 includes processor 401, memory 402 and communication interface 403, communication interface 403 is used for
With external device communication.
Processor 401 may include central processing unit (central processing unit, CPU), network processing unit
(network processor, NP) or combinations thereof.Processor 401 can further include hardware chip, such as dedicated integrated
Circuit (application-specific integrated circuit, ASIC), programmable logic device
(programmable logic device, PLD) or combinations thereof.Above-mentioned PLD can be Complex Programmable Logic Devices
(complex programmable logic device, CPLD), field programmable gate array (field-
Programmable gate array, FPGA), Universal Array Logic (generic array logic, GAL) or its any group
It closes.Each circuit in processor 401 can be independent, also can integrate on one or more chips.
Memory 401, which can be independent device also, can integrate in processor 401.Memory 401 may include easy
The property lost memory (volatile memory), such as random access memory (random-access memory, RAM).Storage
Device 401 also may include nonvolatile memory (non-volatile memory), such as flash memory (flash
Memory), hard disk (hard diskdrive, HDD) or solid state hard disk (solid-state drive, SSD).Memory 401 is also
It may include any combination of the memory of mentioned kind.
Optionally, memory 402 is also used to store computer program instructions, and processor 401 executes the memory 402 and deposits
The computer program instructions of storage realize method shown in FIG. 1 above.
Communication interface 403 can be that can be wireless interface or wireline interface.Wherein, it is mobile to can be honeycomb for wireless interface
Network interface, WLAN (WLAN) interface etc..Wireline interface can be Ethernet interface, such as or optical interface or electricity connect
Mouthful.
Equipment 400 can also include bus 404, and bus 404 is for connecting processor 401, memory 402 and communication interface
403, it is in communication with each other processor 401, memory 402 and communication interface 403 by bus 404.
In one embodiment, the memory 402 is for storing program code, and the processor 401 is for calling
Said program code is to realize the function and step in Fig. 4.
In the above-described embodiments, it can be realized wholly or partly by software, hardware or a combination thereof.When using soft
When part is realized, can entirely or partly it realize in the form of a computer program product.The computer program product includes one
A or multiple computer instructions.When loading on computers and executing the computer program instructions, entirely or partly generate
According to process or function described in the embodiment of the present application.The computer can be general purpose computer, special purpose computer, computer
Network or other programmable devices.The computer instruction may be stored in a computer readable storage medium, or from one
A computer readable storage medium is transmitted to another computer readable storage medium, for example, the computer instruction can be from
One web-site, computer, server or data center pass through wired (such as coaxial cable, twisted pair, optical fiber) or wireless
(such as infrared, wireless, microwave etc.) mode is transmitted to another web-site, computer, server or data center.Institute
Stating computer readable storage medium can be any usable medium or include that one or more can that computer can access
The data storage devices such as server, the data center integrated with medium.The usable medium can be magnetic medium, (for example, soft
Disk, hard disk, tape), optical medium (for example, CD) or semiconductor medium (such as solid state hard disk) etc..
Claims (10)
1. a kind of traffic sign recognition method based on double capture apparatus, which is characterized in that be used for traveling apparatus, the traveling is set
Standby includes the first capture apparatus and the second capture apparatus, and first capture apparatus and second capture apparatus are in same water
Horizontal line, the device parameter of first capture apparatus is identical with the device parameter of second capture apparatus, the traffic sign
Recognition methods includes:
Obtain the first image of first capture apparatus shooting and the second image of second capture apparatus shooting, described the
The time that one capture apparatus shoots the first image is identical as the time that second capture apparatus shoots second image；
The traffic sign in the first image and second image, the traffic mark are identified by Traffic Sign Recognition System
Will identifying system is training in advance, and the traffic sign includes the first traffic sign and the second traffic sign, first traffic
Mark is the traffic sign identified in the first image, is identified in the second traffic sign second image
Traffic sign；
When first traffic sign is identical with second traffic sign, the first coordinate of first traffic sign is determined
With the second coordinate of the second traffic sign, first traffic sign actual traffic mark corresponding with second traffic sign
For target traffic sign；
The target traffic sign is calculated in first capture apparatus and institute according to first coordinate and second coordinate
State the parallax in the second capture apparatus；
According to the distance between target traffic sign and the traveling apparatus described in the disparity computation.
2. the method according to claim 1, wherein identifying described by Traffic Sign Recognition System described
Before traffic sign in one image and second image, further includes:
Traffic Sign Images set is obtained, is marked in each Traffic Sign Images of the Traffic Sign Images set described each
The coordinate and type of traffic sign included by Traffic Sign Images；
According to the default Traffic Sign Recognition System of Traffic Sign Images set training, the Traffic Sign Recognition system is obtained
System.
3. the method according to claim 1, wherein the traffic sign according to the disparity computation and institute
Stating the distance between traveling apparatus includes:
Determine the focal length of first capture apparatus and second capture apparatus；
According to the distance between traffic sign and the traveling apparatus described in the focal length and the disparity computation.
4. the method according to claim 1, wherein further include:
Pass through distance described in voice reminder and the target traffic sign.
5. method according to claim 1-4, which is characterized in that first capture apparatus is binocular camera
Left side camera, second capture apparatus be binocular camera right side camera.
6. a kind of Traffic Sign Recognition device based on double capture apparatus, which is characterized in that be used for traveling apparatus, the traffic mark
Will identification device includes the first capture apparatus and the second capture apparatus, at first capture apparatus and second capture apparatus
In same horizontal line, first capture apparatus is identical with the device parameter of second capture apparatus, and the traffic sign is known
Other device further include:
Acquiring unit, for obtain first capture apparatus shooting the first image and second capture apparatus shooting the
Two images, the time of the first capture apparatus shooting the first image and second capture apparatus shoot second figure
The time of picture is identical；
Recognition unit, for identifying the traffic mark in the first image and second image by Traffic Sign Recognition System
Will, the Traffic Sign Recognition System are training in advance, and the traffic sign includes the first traffic sign and the second traffic sign,
First traffic sign is the traffic sign identified in the first image, and second traffic sign is second figure
The traffic sign identified as in；
Determination unit, for determining first traffic when first traffic sign is identical with second traffic sign
First coordinate of mark and the second coordinate of the second traffic sign, first traffic sign are corresponding with second traffic sign
Actual traffic mark be target traffic sign；
Computing unit, for calculating the target traffic sign described first according to first coordinate and second coordinate
Parallax in capture apparatus and second capture apparatus；
The computing unit is also according to the distance between target traffic sign and the traveling apparatus described in the disparity computation.
7. device according to claim 6, which is characterized in that the acquiring unit is also used to obtain traffic indication map image set
It closes, marks traffic included by each Traffic Sign Images in each Traffic Sign Images of the Traffic Sign Images set
The coordinate and type of mark；
Described device further include:
Training unit, for obtaining the friendship according to the default Traffic Sign Recognition System of Traffic Sign Images set training
Logical sign recognition system.
8. device according to claim 6, which is characterized in that the computing unit is specifically used for:
Determine the focal length of first capture apparatus and second capture apparatus；
According to the distance between traffic sign and the traveling apparatus described in the focal length and the disparity computation.
9. device according to claim 6, which is characterized in that further include:
Reminding unit, for passing through distance and the target traffic sign described in voice reminder.
10. according to the described in any item devices of claim 6-9, which is characterized in that first capture apparatus is binocular camera shooting
The left side camera of head, second capture apparatus are the right side camera of binocular camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910051444.0A CN109829401A (en) | 2019-01-21 | 2019-01-21 | Traffic sign recognition method and device based on double capture apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910051444.0A CN109829401A (en) | 2019-01-21 | 2019-01-21 | Traffic sign recognition method and device based on double capture apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109829401A true CN109829401A (en) | 2019-05-31 |
Family
ID=66861723
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910051444.0A Pending CN109829401A (en) | 2019-01-21 | 2019-01-21 | Traffic sign recognition method and device based on double capture apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109829401A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111553387A (en) * | 2020-04-03 | 2020-08-18 | 上海物联网有限公司 | Yolov 3-based personnel target detection method |
| CN112446920A (en) * | 2019-09-05 | 2021-03-05 | 华为技术有限公司 | Method and device for determining position of camera |
| CN112987707A (en) * | 2019-11-29 | 2021-06-18 | 北京京东乾石科技有限公司 | Automatic driving control method and device for vehicle |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104155470A (en) * | 2014-07-15 | 2014-11-19 | 华南理工大学 | Detecting method and system based on binocular camera for real-time vehicle speed |
| CN204596166U (en) * | 2013-01-05 | 2015-08-26 | 广西科技大学 | Vehicle intelligent Traffic Sign Recognition System |
| CN105718860A (en) * | 2016-01-15 | 2016-06-29 | 武汉光庭科技有限公司 | Positioning method and system based on safe driving map and binocular recognition of traffic signs |
| CN106225764A (en) * | 2016-07-01 | 2016-12-14 | 北京小米移动软件有限公司 | Based on the distance-finding method of binocular camera in terminal and terminal |
| CN106326858A (en) * | 2016-08-23 | 2017-01-11 | 北京航空航天大学 | Road traffic sign automatic identification and management system based on deep learning |
| CN106813649A (en) * | 2016-12-16 | 2017-06-09 | 北京远特科技股份有限公司 | A kind of method of image ranging localization, device and ADAS |
| CN106980855A (en) * | 2017-04-01 | 2017-07-25 | 公安部交通管理科学研究所 | Traffic sign quickly recognizes alignment system and method |
| CN109002764A (en) * | 2018-06-19 | 2018-12-14 | 长安大学 | A kind of building of Traffic Sign Images identification model and recognition methods |
-
2019
- 2019-01-21 CN CN201910051444.0A patent/CN109829401A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN204596166U (en) * | 2013-01-05 | 2015-08-26 | 广西科技大学 | Vehicle intelligent Traffic Sign Recognition System |
| CN104155470A (en) * | 2014-07-15 | 2014-11-19 | 华南理工大学 | Detecting method and system based on binocular camera for real-time vehicle speed |
| CN105718860A (en) * | 2016-01-15 | 2016-06-29 | 武汉光庭科技有限公司 | Positioning method and system based on safe driving map and binocular recognition of traffic signs |
| CN106225764A (en) * | 2016-07-01 | 2016-12-14 | 北京小米移动软件有限公司 | Based on the distance-finding method of binocular camera in terminal and terminal |
| CN106326858A (en) * | 2016-08-23 | 2017-01-11 | 北京航空航天大学 | Road traffic sign automatic identification and management system based on deep learning |
| CN106813649A (en) * | 2016-12-16 | 2017-06-09 | 北京远特科技股份有限公司 | A kind of method of image ranging localization, device and ADAS |
| CN106980855A (en) * | 2017-04-01 | 2017-07-25 | 公安部交通管理科学研究所 | Traffic sign quickly recognizes alignment system and method |
| CN109002764A (en) * | 2018-06-19 | 2018-12-14 | 长安大学 | A kind of building of Traffic Sign Images identification model and recognition methods |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112446920A (en) * | 2019-09-05 | 2021-03-05 | 华为技术有限公司 | Method and device for determining position of camera |
| CN112987707A (en) * | 2019-11-29 | 2021-06-18 | 北京京东乾石科技有限公司 | Automatic driving control method and device for vehicle |
| CN112987707B (en) * | 2019-11-29 | 2025-02-25 | 北京京东乾石科技有限公司 | A method and device for controlling automatic driving of a vehicle |
| CN111553387A (en) * | 2020-04-03 | 2020-08-18 | 上海物联网有限公司 | Yolov 3-based personnel target detection method |
| CN111553387B (en) * | 2020-04-03 | 2022-09-23 | 上海物联网有限公司 | Personnel target detection method based on Yolov3 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107230218B (en) | Method and apparatus for generating confidence measures for estimates derived from images captured by cameras mounted on vehicles | |
| US11043000B2 (en) | Measuring method and apparatus for damaged part of vehicle | |
| CN107025642B (en) | Vehicle's contour detection method and device based on point cloud data | |
| CN110059608A (en) | A kind of object detecting method, device, electronic equipment and storage medium | |
| CN113281780B (en) | Method, device and electronic device for labeling image data | |
| CN109242903A (en) | Generation method, device, equipment and the storage medium of three-dimensional data | |
| CN112927279A (en) | Image depth information generation method, device and storage medium | |
| CN111222395A (en) | Target detection method and device and electronic equipment | |
| CN111091023B (en) | A vehicle detection method, device and electronic equipment | |
| WO2021135766A1 (en) | Trajectory planning method and apparatus, controller and smart car | |
| JP2015041164A (en) | Image processor, image processing method and program | |
| CN114693963A (en) | Recognition model training and recognition method and device based on electric power data feature extraction | |
| CN116052026B (en) | Unmanned aerial vehicle aerial image target detection method, system and storage medium | |
| CN116071704B (en) | A target detection method for X-ray security inspection images based on dual-mode networks across multiple datasets | |
| CN105849770A (en) | Information processing device, information processing method, and program | |
| CN114972492A (en) | A bird's-eye view-based pose determination method, device and computer storage medium | |
| CN109829401A (en) | Traffic sign recognition method and device based on double capture apparatus | |
| CN117789160A (en) | Multi-mode fusion target detection method and system based on cluster optimization | |
| CN110060264B (en) | Neural network training method, video frame processing method, device and system | |
| CN111881984A (en) | Target detection method and device based on deep learning | |
| CN114611635B (en) | Object recognition method, device, storage medium and electronic device | |
| CN116009581A (en) | UAV inspection method for power transmission line, UAV control terminal and storage medium | |
| CN112507891B (en) | Method and device for automatically identifying high-speed intersection and constructing intersection vector | |
| CN119573658B (en) | Method and device for determining target distance from drone patrol infrared images | |
| CN116883981A (en) | License plate positioning and identifying method, system, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190531 |