CN104954741B - Profound self-teaching network implementations mine car sky expires the detection method and system of state - Google Patents
Profound self-teaching network implementations mine car sky expires the detection method and system of state Download PDFInfo
- Publication number
- CN104954741B CN104954741B CN201510290352.XA CN201510290352A CN104954741B CN 104954741 B CN104954741 B CN 104954741B CN 201510290352 A CN201510290352 A CN 201510290352A CN 104954741 B CN104954741 B CN 104954741B
- Authority
- CN
- China
- Prior art keywords
- mine car
- image
- module
- extreme value
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 38
- 238000012544 monitoring process Methods 0.000 claims abstract description 21
- 238000006073 displacement reaction Methods 0.000 claims description 22
- 230000008859 change Effects 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000000034 method Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 2
- 238000003709 image segmentation Methods 0.000 claims description 2
- 239000000284 extract Substances 0.000 claims 1
- 230000008901 benefit Effects 0.000 abstract description 3
- 238000004458 analytical method Methods 0.000 abstract 1
- 238000012549 training Methods 0.000 description 18
- 238000010801 machine learning Methods 0.000 description 8
- 238000013461 design Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 238000005065 mining Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000000428 dust Substances 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- FCHAMFUEENBIDH-UHFFFAOYSA-N Severin Natural products CC1CCC2C(C)C3CCC4(O)C(CC5C4CC(O)C6CC(CCC56C)OC(=O)C)C3CN2C1 FCHAMFUEENBIDH-UHFFFAOYSA-N 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses the detection methods that a kind of profound self-teaching network implementations mine car sky expires state, comprising the following steps: inputs the status image of multiple mine cars or non-mine car and is stored as sample database；Derivative expansion is carried out to sample database；Profound learning network is constructed to be acquired monitoring image, it analyses and compares according to the sample database after derivative expand to monitoring image, the mine car dummy status image, the mine car that store in monitoring image and sample database are expired into status image and compare difference with mine car status image respectively, mine car dummy status image pattern library, mine car in image deposit sample database are expired into status image sample database or non-mine car status image sample database according to difference is compared；In the respective sample library that the image of acquisition is carried out to derivative exptended sample library.The present invention has the advantage that can accurately pass through camera head monitor, detects that the sky of mine car load mine expires state, reduce manual intervention, realize the management of mine car operation automation.
Description
Technical field
The present invention relates to machine learning methods to be applied in mining site in mine car intelligent operation, realizes that mine car carries mine sky full two
The automatic detection of state of value.Particularly suitable under the conditions of complicated and changeable, including there are intensity of illumination variation, mine dust interference with
And mine car position arbitrarily changes, it would be desirable to be able to accurately by camera head monitor, detect that the sky of mine car load mine expires state, reduce people
The scene of mine car operation automation management is realized in work intervention.
Background technique
Currently, realizing the intelligence and automatic management of mine operation, mining can be greatlyd improve, the efficiency of mine is transported.
And wherein mine car sky expires the automatic detection of state, can dispatch for mine car and provide accurate information, and then can optimize mine car tune
Degree, improves the utilization efficiency of mine car.But in actual mining site operation, environmental condition locating for mine car is extremely complex changeable.
The randomness that open work bring illumination variation, the mine dust bring dimness of vision and mine car are parked, can all interfere mine car
Video monitoring.How accurately to detect that mine car carries the sky of mine and expires state, is one challenging and significantly
Problem.
Using machine learning, expires state to detect the sky of mine car, be a kind of very effective detection method.Machine learning energy
It is enough that sample learning is carried out to various complex conditions, by learning great amount of samples, generalization ability and the detection of learner can be promoted
Classification capacity, and then with high robust the sky to mine car can expire state and detect.It is being applied to the empty slow state-detection of mine car
During, it is related to the design of machine learning device.Robustness is designed, and the learner with very strong generalization ability is to mine
The monitor and detection of vehicle has very big meaning.In different mining sites and the different time sections of the same mine car, locating for mine car
Scene be different.A large amount of learning sample is made, needs very high artificial mark cost, this is turned into the intelligence of mine car
Industry is a no small problem.Therefore, the learner of design should have online ability of self-teaching, can update in real time
The sample database of oneself, so that it may manual intervention is reduced, to meet the detection scene that mine car carries mine state very usefully.
In the design and research of learner, Hinton et al. is in " ImageNet Classification with Deep
Convolutional Neural Networks ' (Neural Information Processing Systems 2012), specially
Door devises the machine learning device based on profound e-learning structure, by way of supervised learning, may be implemented very high
Object nicety of grading.Their work based on profound e-learning are the design of the machine learning device of High Precision Robust
Open a new visual angle.Then, profound learning network includes image recognition in many fields, speech recognition and from
Right word processing achieves huge success.But this kind of machine learning device, need a large amount of sample to carry out off-line training.Together
When the classifier that generates of off-line training, the situation of change of practical mine car monitoring scene can not be learnt online, and be only through big
Diversity and the otherness of training sample are measured to enhance the generalization ability of study.In addition, this kind of machine learning method, from a large amount of quiet
The base pixel of state samples pictures sets out, and goes angle point in study image, the information such as edge, and then construct high-level semantic, is learning
On have certain blindness.The monitor video sequence with time dimension is directly applied to, time dimension can not be excavated well
On, variation of the object to be detected or to be sorted in structure, the information such as color.Therefore, it is regarded for the monitoring with high correlation
Frequency sequence realizes that mine car sky expires the detection of state in the case where mine intelligent is turned into industry, needs to design special profound e-learning
Structure sufficiently excavates the information on time dimension, and then improves the precision of detection classification.
In addition, in order to improve the robustness of detection classification, profound learning network should have as we are set forth above
Have the ability of on-line study, mine car monitoring during, can the variation online to scene around make correct study,
Exclusive PCR.In terms of on-line study mechanism, Severin et al. " Beyond Semi-Supervised Tracking:
Tracking Should Be as Simple as Detection,but not Simpler than Recognition”
In, by on-line study candidate region and neighboring area, realize the self-teaching of shallow-layer network.This patent is directed to mine car
Operation, which carries mine sky about mine car, expire the monitoring of state, emphatically from profound e-learning structure design with self-teaching mechanism this two
Aspect is set about, and proposing the mine car sky based on profound self-teaching expires state robust detection method.
Summary of the invention
The present invention is directed at least solve one of above-mentioned technical problem.
For this purpose, the first purpose of this invention is to propose that a kind of profound self-teaching network implementations mine car sky expires state
Detection method.
Second object of the present invention is the inspection for proposing that a kind of profound self-teaching network implementations mine car sky expires state
Examining system.
To achieve the goals above, embodiment of the invention discloses a kind of profound self-teaching network implementations mine car is empty
The detection method of full state, comprising the following steps: A. inputs multiple mine car dummy status images, multiple mine cars expire status image and more
A non-mine car status image is simultaneously stored as initial sample database；B. derivative expansion is carried out to initial sample database；C. deep layer is constructed
Secondary learning network is acquired monitoring image, analyzes according to the sample database after derivative expand the monitoring image
It compares, the mine car dummy status image, the mine car that store in the monitoring image and the sample database is expired into status image
Difference is compared with the mine car status image respectively, described image is stored in by mine car in the sample database according to the comparison difference
Status image sample database or non-mine car status image sample database are expired in dummy status image pattern library, mine car；And D. is by the institute of acquisition
Image is stated to carry out in the derivative respective sample library for expanding the sample database.
A kind of profound self-teaching network implementations mine car sky according to an embodiment of the present invention expires the detection method of state, energy
Accurately by camera head monitor, detects that the sky of mine car load mine expires state, reduce manual intervention, realize mine car operation automation
Management.
In addition, a kind of profound self-teaching network implementations mine car sky according to the above embodiment of the present invention expires the inspection of state
Survey method can also have the following additional technical features:
Further, in stepb, the mode for telling derivative expansion includes the mine car dummy status figure to the sample database
Picture, mine car expire status image and non-mine car status image carries out affine transformation and/or noise addition and/or bright adjusting.
Further, in step C, further comprise: the video frame that C1. chooses continuous N frame is divided into three channel difference
It carries out color displacement, shape displacement and luminance information to extract, wherein color displacement refers to the video frame pixel described in N frame
RGB channel carries out difference frame by frame, and seeks the mean value of the RGB channel after difference, and shape displacement refers to that acquisition monitors the video
The change in location of frame motion parts, luminance information refer to the gray value for directly recording each frame of the video frame；C2. described logical by three
The channel information in road carries out convolution and seeks extreme value；C3. the mine car dummy status image pattern library is read, the mine car expires state
Image pattern library and non-mine car status image sample database calculate the difference with the extreme value, by video frame deposit and the mine
Vehicle expires status image sample database and the smallest mine car dummy status image pattern library of non-mine car status image sample database difference value, described
Mine car is expired in status image sample database or non-mine car status image sample database.
Further, in step C2, further comprise: C21. carries out the channel information in three channels for the first time
Convolution simultaneously seeks the first extreme value；C22. second of convolution is carried out according to first extreme value and seeks secondary extremal.
Further, in step C3, further comprise: C31. obtains first extreme value and the secondary extremal；
C32. secondary extremal described in first extreme value and the secondary extremal and the mine car dummy status image are calculated using SOFTMAX
Sample database, the mine car expire the difference value between status image sample database and non-mine car status image sample database, when the difference
When value is less than a preset value, the video frame is stored in the corresponding sample database.
Further, in step D, the described image of acquisition is subjected to the derivative respective sample library for expanding the sample database
In, the derivative method expanded includes being rotated, being tilted to described image, introduced noise, adjust bright and adjusting comparison
At least one of in degree.
To achieve the goals above, embodiment of the invention discloses a kind of profound self-teaching network implementations mine car is empty
The detection system of full state, including sample database, expire state for multiple mine car dummy status images, the multiple mine cars to initial input
Image and multiple non-mine car status images are stored；Derivative enlargement module, for the mine car dummy status image, the mine
Vehicle expires status image and the non-mine car status image carries out derivative expansion；Image capture module；And selection enlargement module, it uses
The mine car dummy status image, the mine car after reading that the derivative enlargement module is derivative and expanding expire status image and described
Non- mine car status image is compared to obtain difference value with the image that described image acquisition module respectively acquires, and according to the difference
Described image is not stored in the corresponding position of the sample database by value.
A kind of profound self-teaching network implementations mine car sky according to an embodiment of the present invention expires the detection system of state, energy
Accurately by camera head monitor, detects that the sky of mine car load mine expires state, reduce manual intervention, realize mine car operation automation
Management.
In addition, a kind of profound self-teaching network implementations mine car sky according to an embodiment of the present invention expires the detection system of state
System, can also have the following additional technical features:
Further, the selection enlargement module further comprises: read module, for reading the derivative enlargement module
The mine car dummy status image, the mine car after derivative expansion expire status image and the non-mine car status image；Segmentation mentions
Modulus block, image segmentation for acquiring video acquisition module are that three channels carry out color displacement respectively, shape displacement and
The extraction of luminance information, wherein color displacement, which refers to, carries out difference frame by frame to the RGB channel of N frame video frame pixel, and seeks difference
The mean value of the RGB channel afterwards, shape displacement refer to the change in location for obtaining and monitoring the video frame motion part, and luminance information refers to
Directly record the gray value of each frame of the video frame；Convolutional calculation module, for receiving from the logical of the segmentation extraction module
Road information simultaneously carries out convolutional calculation；Extreme value seeks module, seeks extreme value, the pole for the calculated result to convolutional calculation module
Value includes maximum and minimum；Categorization module, the derivative enlargement module for being read according to the read module are derivative
The mine car dummy status image, the mine car after expansion expire status image and the non-mine car status image respectively with the pole
Value seeks the ratio of extreme values that module is sought and classifies to difference, and according to the difference to described image；And enlargement module, it uses
In seeing that sorted described image extends in the corresponding position of the sample database.
Further, the convolutional calculation module includes the first convolution computing module, the second convolution computing module, third volume
Product computing module and Volume Four product computing module；It includes that the first extreme value seeks module, secondary extremal is asked that the extreme value, which seeks module,
Modulus block, third extreme value seek module and the 4th extreme value seeks module；Wherein, the first convolution computing module is for calculating institute
The first volume product value of channel information is stated, first extreme value seeks module for seeking the first extreme value to the first volume product value；
The second convolution computing module is used to calculate volume Two product value according to first extreme value, and the secondary extremal seeks module use
In seeking secondary extremal to the volume Two product value；The third convolutional calculation module is used to calculate the according to the secondary extremal
Three convolution values, the third extreme value seek module for seeking third extreme value to the third convolution value；The Volume Four product meter
It calculates module and is used to calculate Volume Four product value according to the third extreme value, the 4th extreme value seeks module for the Volume Four
Product value seeks the 4th extreme value.
Further, the selection enlargement module further includes full link block, and the full link block is described for receiving
First extreme value, secondary extremal, third extreme value and the 4th extreme value, and by first extreme value, secondary extremal, third extreme value and
4th extreme value is transmitted to the categorization module and classifies, and the categorization module is classified using SOFTMAX regression model.
Additional aspect and advantage of the invention will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect of the invention and advantage will become from the description of the embodiment in conjunction with the following figures
Obviously and it is readily appreciated that, in which:
Fig. 1 is the structural schematic diagram of profound learning network of the invention；
Fig. 2 is structural schematic diagram of the invention.
Specific embodiment
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, and for explaining only the invention, and is not considered as limiting the invention.
In the description of the present invention, it is to be understood that, term " center ", " longitudinal direction ", " transverse direction ", "upper", "lower",
The orientation or positional relationship of the instructions such as "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outside" is
It is based on the orientation or positional relationship shown in the drawings, is merely for convenience of description of the present invention and simplification of the description, rather than instruction or dark
Show that signified device or element must have a particular orientation, be constructed and operated in a specific orientation, therefore should not be understood as pair
Limitation of the invention.In addition, term " first ", " second " are used for description purposes only, it is not understood to indicate or imply opposite
Importance.
In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, term " installation ", " phase
Even ", " connection " shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected；It can
To be mechanical connection, it is also possible to be electrically connected；It can be directly connected, can also can be indirectly connected through an intermediary
Connection inside two elements.For the ordinary skill in the art, above-mentioned term can be understood at this with concrete condition
Concrete meaning in invention.
Referring to following description and drawings, it will be clear that these and other aspects of the embodiment of the present invention.In these descriptions
In attached drawing, some particular implementations in the embodiment of the present invention are specifically disclosed, to indicate to implement implementation of the invention
Some modes of the principle of example, but it is to be understood that the scope of embodiments of the invention is not limited.On the contrary, of the invention
Embodiment includes all changes, modification and the equivalent fallen within the scope of the spirit and intension of attached claims.
Embodiment according to the present invention is described below in conjunction with attached drawing.
Fig. 1 is the structural schematic diagram of profound learning network of the invention, and Fig. 2 is structural schematic diagram of the invention.It please join
Examine Fig. 1 and Fig. 2.
1, profound learning network pre-training
(1) mine car loading status image sample prepares
All sample standard deviations derive from mine supervision camera, to the video acquisition of mine car operation loading state.It is initial doing
When pre-training, by manually marking, sample is divided into three classes, first kind number tag representation is 0, represents the current shape of mine car
State is sky, i.e., no loading.He second-class number word tag representation be 1, indicate mine car current state be it is full, that is, be loaded with goods.The
Three classes digital label is expressed as 2, indicates other samples of non-mine car state.In order to ensure the diversity of sample, enhance extensive energy
Power, collecting sample carry out under the conditions of rainy day etc. in daytime, night, cloudy day, fine day, acquire under the conditions of various equal number of each
Class sample.In training sample, representing mine car state as empty number of samples is 100,000, represents mine current state as full sample
This number is also 100,000, and representing non-mine car state is 20,000.In verifying sample, mine car state is represented as empty number of samples
It is 1,000, representing mine current state as full sample number is also 1,000, represents the sample 2,000 of non-mine car state.Wherein number mark
The sample that label are 0 or 1, positive sample also referred to as above-mentioned.And the sample that digital label is 2, also refer to above-mentioned negative
Sample.
(2) the profound learning network constructed is trained the sample demarcated in advance
For mine car monitor video, profound learning network is designed, it is intended to which more accurately obtaining mine car sky expires state
Feature representation.Profound network has from pixel, arrives edge, then the accurate ability to express to high-rise shape semanteme.Of the invention
Profound learning network includes a dividing layer, and 4 convolutional layers, 4 extreme value layers, 1 is connected layer and 1 entirely and returned based on SOFTMAX
Return three classifiers of model.
Formula is trained to the mine car monitor video frame sequence of input, sequence of frames of video sample basic unit number we take
10.Video frame training sample is divided into three channels, carries out color displacement, the extraction of shape displacement and luminance information respectively.
Wherein color displacement refers to carrying out difference frame by frame respectively to the RGB channel of video frame pixel, and after seeking difference
The mean value of RGB channel.
And shape displacement refers to obtaining the change in location of monitoring video frame motion parts, changes and is assigned a value of 1, it is unchanged
It is 0.
And luminance information refers to the gray value for directly recording each frame of video frame.
For mine car monitor video sequence, obtain color displacement, shape displacement and luminance information, can sufficiently excavate frame with
Space structure and color between frame, brightness change relationship, to be sufficiently used the correlativity of interframe.It therefore can be more
Add the state change information for accurately describing mine car under monitor video, provides more fully information for profound level e-learning
Source.
For three channel informations of extraction, dividing layer is completed to three layers, the extraction of positive and negative sample areas.Wherein positive sample
This includes vehicle completely and the image-region under empty two states of vehicle, negative sample are the image-region of non-mine car.
By dividing layer, information enters convolutional layer 1, and the 3D convolution kernel based on 10 class 3x3x10 completes convolution.
Information enters extreme value layer 1 after convolution.This layer seeks extreme value using 3x3 template.Extreme value includes maximum and minimum.
Hereafter, the data of extreme value layer 1 flow into convolutional layer 2, then the 3D convolution kernel based on 10 class 3x3x10 completes convolution.
The data of convolutional layer 2 flow into extreme value layer 2, and similarly, completion seeks image-region extreme value based on 3x3 template.
The data of extreme value layer 2 flow into convolutional layer 3, and in the layer, the 4x4 convolution kernel of 10 class 2D of base is completed to extreme value layer data
Convolution.
The layer data flows into extreme value layer 3, similarly completes to seek image extreme value based on 3x3 template.
The data of extreme value layer 3 enter convolutional layer 4, complete in the 5x5 convolution kernel of the layer, 10 class 2D of base to extreme value layer data
Convolution.
Data after convolution flow into extreme value layer 4, and completion is sought based on extreme value of the 3x3 template to image-region.
Extreme value data finally flow into full articulamentum.Full articulamentum completes the full expansion to 4 data of extreme value layer, while receiving to come
From the data of extreme value layer 1,2,3.Receiving the data length of extreme value layer 1,2,3 is data rule that are fixed, being unfolded equal to extreme value layer 4
Mould.Simultaneously using random received mode.Using random received mode, full articulamentum can be enhanced to profound learning network
Different phase carry out feature representation ability.The data that full articulamentum receives extreme value layer 1,2,3,4 also make receptive field transmit, and anticipate
For the impression information of different layers is transmitted on full articulamentum.Quan Lian base followed by a upper SOFTMAX three classifiers,
It completes to mine car sky, the detection of full and non-mine car state.
SOFTMAX classifier judges mine car sky, and full and non-mine car state procedure is as follows:
Monitor video frame sequence passes through profound network, completes each convolutional layer and extreme value layer calculates, recently enter and connect entirely
Connect layer.Connect layer data entirely and be input in SOFTMAX classifier and participates in classified calculating.During SOFTMAX classified calculating, Quan Lian
The data for connecing layer output can carry out the calculating of logistic regression, eventually export about mine car with the parameter in SOFTMAX classifier
Sky, full and non-three states of mine car probability value, takes that state of maximum probability as the detection to monitor video frame sequence
Judgement.
During carrying out above-mentioned judgement to monitoring video frame training, the effect of convolution layer parameter and full connection layer parameter
To realize dimensionality reduction and feature abstraction to monitor video frame data, specially lead to for three that monitoring image pixel is split
Road figure, in convolutional layer 1, extreme value layer 1, convolutional layer 2, extreme value layer 2, the detection at realization channel image edge and angle point, in convolutional layer
3, maximum layer 3 realizes the detection of channel image shape by combination edge and angle point, in convolutional layer 4 and extreme value layer 4, completes
The combination of shape forms the complete description for realizing input channel image.And in full articulamentum, by the transmitting of receptive field, by each pole
The detection information of value layer is converged, and is classified for SOFTMAX.
Each extreme value layer of profound learning network is all connected to full articulamentum, and which enhance full articulamentums to each study rank
The information representation of the lower mine car state of section, can be improved the ability to express of full articulamentum, and then improve detection effect.
In the present invention, the convolution nuclear parameter of each convolutional layer, and layer parameter is connected entirely, it is equal when initial training
It is randomly generated.Learnt by training, obtains each convolution kernel and coefficient of connection.Meanwhile the coefficient of full articulamentum, deep layer
Secondary network self-teaching with it is new when, be randomly generated original state.Design can obtain higher in this way to avoid over-fitting
Generalization ability.
Profound learning network is trained using BP algorithm (error backpropagation algorithm), is iterated, is obtained pre-
Training pattern.This pre-training model, by the initial model as profound learning network, participate in mine car state judgement and
The self-teaching of profound learning network, therefore loss to pre-training and required precision are not very high.In the training process, I
Will training the damage control 10% hereinafter, precision controlling is 90% or more.
2, classified based on profound network to the vehicle-mounted state of mine car
(1) candidate region is positioned
In monitoring image, candidate region to be checked is obtained using full scan mode.It sweeps foundation the setting of window size
Previous frame is detected as the average-size of the window area of number 0 or 1, changes in the 0.9~1.1 of size proportional region.
Change window size out for the size as Current Scan window.In this way can to avoid a large amount of invalid, redundancy scanning window
It generates.
(2) profound learning network carries out detection classification
It for candidate region, is detected using the profound learning network that pre-training is practised, it is true for obtaining selective mechanisms
With false candidate region.Here the digital label for being detected as really referring to testing result is 0 and 1, that is, detects mine car loading state
It for sky or is full.The digital label of testing result is 2, and expression is detected as vacation, that is, is referred to currently as non-mine car cargo area.
3, Sample Refreshment
For profound network detected as a result, respectively to detection digital label result be 0,1,2 candidate region
It is converted.Specific transformation rule are as follows: given candidate region carries out rotation first and tilt variation derives 30 figures, then
Gauss change is done on the basis of 30 figures, the figure after deriving 30x20 Gauss change of scale.Continue on the basis of these figures
On, noise is added, adjusts bright and contrast, derives the sample graph of 30x20x10.In addition, it is analogous to pre-training process,
On the basis of these sample graphs, increase dimensional information.It is past to be pushed forward 9 frames or so specially since present frame.Based on this new 10
Frame picture training obtains candidate region and is displaced in color, shape is displaced upper and brightness according to the requirement of profound network inputs
On information, to obtain current correlation information of the vehicle-mounted state in time domain and airspace.
4, self-teaching and update
The sample information of update is input in initial profound learning network and is finely adjusted, profound learning network
The weight for connecting layer entirely is randomly generated, and profound network others parameter uses the parameter of upper one layer update.BP is used to whole network
Algorithm is trained.Algorithm passes through iterative learning, the new profound learning network model of generation.The model will be as next time
The introductory die model of update.Using above-mentioned method, the update of profound network learning model is completed.
5, it iterates
Monitoring image input, candidate region obtain, and current judgement result is obtained based on category of model.While sample is more
Newly, profound network self-teaching, for classification judgement next time.Whole process loop iteration.Sample database is because continuous
It updates and constantly increases, while network model has newest expression to vehicle-mounted current scene state, therefore, profound network
Learning ability can constantly enhance because of online updating and self-teaching iteration.
In addition, the profound self-teaching network implementations mine car sky of the embodiment of the present invention expires the detection method and system of state
Other compositions and effect be all for a person skilled in the art it is known, in order to reduce redundancy, do not repeat them here.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
Centainly refer to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be any
One or more embodiment or examples in can be combined in any suitable manner.
Although an embodiment of the present invention has been shown and described, it will be understood by those skilled in the art that: not
A variety of change, modification, replacement and modification can be carried out to these embodiments in the case where being detached from the principle of the present invention and objective, this
The range of invention is by claim and its equivalent limits.
Claims (8)
1. the detection method that a kind of profound level self-teaching network implementations mine car sky expires state, which is characterized in that including following step
It is rapid:
A. multiple mine car dummy status images are inputted, multiple mine cars expire status image and multiple non-mine car status images and store conduct
Initial sample database；
B. derivative expansion is carried out to initial sample database；
C. it constructs profound learning network to be acquired monitoring image, according to the sample database after derivative expand to the prison
Control image is analysed and compared, and the monitoring image is expired with the multiple mine car dummy status image, the multiple mine car respectively
Status image and the multiple non-mine car status image compare difference, and described image is stored in the sample according to the comparison difference
Mine car dummy status image pattern library, mine car expire status image sample database or non-mine car status image sample database in this library, wherein
In step C, specifically include:
C1. the video frame for choosing continuous N frame is divided into three channels and carries out color displacement, shape displacement and luminance information progress respectively
It extracts, wherein color displacement refers to that the RGB channel of the video frame pixel described in N frame carries out difference frame by frame, and seeks described after difference
The mean value of RGB channel, shape displacement refer to the change in location for obtaining and monitoring the video frame motion part, and luminance information refers to direct note
Record the gray value of each frame of the video frame；
C2., the channel information in three channels is carried out to convolution two-by-two and seeks extreme value；
C3. the mine car dummy status image pattern library is read, the mine car expires status image sample database and non-mine car status image
Sample database calculates the difference with the extreme value, by video frame deposit and the mine car dummy status image pattern library, the mine
Vehicle is expired in status image sample database and non-mine car status image sample database in the smallest sample database of difference value；And
D. the described image of acquisition is carried out in the derivative respective sample library for expanding the sample database.
2. profound level self-teaching network implementations mine car sky according to claim 1 expires the detection method of state, feature
It is, in stepb, the derivative mode expanded includes expiring state to mine car dummy status image, the mine car of the sample database
Image and non-mine car status image carry out affine transformation and/or noise addition and/or bright adjusting.
3. profound level self-teaching network implementations mine car sky according to claim 1 expires the detection method of state, feature
It is, in step C2, specifically includes:
C21., the channel information in three channels is carried out to first time convolution two-by-two and seeks the first extreme value；
C22. second of convolution is carried out according to first extreme value and seeks secondary extremal.
4. profound level self-teaching network implementations mine car sky according to claim 3 expires the detection method of state, feature
It is, in step C3, specifically includes:
C31. first extreme value and the secondary extremal are obtained；
C32. using SOFTMAX calculate first extreme value and the secondary extremal and the mine car dummy status image pattern library,
The mine car expires the difference value between status image sample database and non-mine car status image sample database, when the difference value is less than one
When a preset value, the video frame is stored in the corresponding sample database.
5. profound level self-teaching network implementations mine car sky according to claim 1 expires the detection method of state, feature
It is, in step D, the described image of acquisition is carried out in the derivative respective sample library for expanding the sample database, the derivative
The method of expansion includes being rotated, being tilted to described image, being introduced into noise, adjusting at least one for becoming clear and adjusting in contrast
?.
6. the detection system that a kind of profound level self-teaching network implementations mine car sky expires state, which is characterized in that including
Sample database expires status image and multiple non-mine cars for multiple mine car dummy status images, the multiple mine cars to initial input
Status image is stored；
Derivative enlargement module, for expiring status image and the non-mine car state to the mine car dummy status image, the mine car
Image carries out derivative expansion；
Image capture module；And
Enlargement module is selected, it is the mine car dummy status image after expanding for reading that the derivative enlargement module is derivative, described
Mine car is expired status image and the non-mine car status image and is compared respectively with the image of described image acquisition module acquisition
Described image is stored in the corresponding position of the sample database to difference value, and according to the difference value；Wherein, the selection is expanded
Mold filling block specifically includes:
Read module derives the mine car dummy status image, the mine car after expanding for reading the derivative enlargement module
Full status image and the non-mine car status image；
Divide extraction module, the image segmentation for acquiring video acquisition module is that three channels carry out color displacement respectively,
The extraction of shape displacement and luminance information, wherein color displacement, which refers to, carries out difference frame by frame to the RGB channel of N frame video frame pixel,
And the mean value of the RGB channel after difference is sought, shape displacement refers to the change in location for obtaining and monitoring the video frame motion part,
Luminance information refers to the gray value for directly recording each frame of the video frame；
Convolutional calculation module, for receiving the channel information from the segmentation extraction module and carrying out convolutional calculation；
Extreme value seeks module, seeks extreme value for the calculated result to convolutional calculation module, the extreme value includes maximum and pole
Small value；
Categorization module, the mine car after the derivative expansion of the derivative enlargement module for being read according to the read module are empty
Status image, the mine car expire status image and the non-mine car status image seeks the pole that module is sought with the extreme value respectively
Value compares difference, and is classified according to the difference to described image；And
Enlargement module, for extending to sorted described image in the corresponding position of the sample database.
7. profound level self-teaching network implementations mine car sky according to claim 6 expires the detection system of state, feature
Be, the convolutional calculation module include the first convolution computing module, the second convolution computing module, third convolutional calculation module and
Volume Four accumulates computing module；
The extreme value seek module include the first extreme value seeks module, secondary extremal seeks module, third extreme value seeks module and
4th extreme value seeks module；
Wherein, the first convolution computing module is used to calculate the first volume product value of the channel information, and first extreme value is asked
Modulus block is for seeking the first extreme value to the first volume product value；The second convolution computing module is used for according to first pole
Value calculates volume Two product value, and the secondary extremal seeks module for seeking secondary extremal to the volume Two product value；Described
Three convolutional calculation modules are used to calculate third convolution value according to the secondary extremal, and the third extreme value seeks module for institute
It states third convolution value and seeks third extreme value；The Volume Four product computing module is used to calculate Volume Four product according to the third extreme value
Value, the 4th extreme value seek module for seeking the 4th extreme value to the Volume Four product value.
8. profound level self-teaching network implementations mine car sky according to claim 7 expires the detection system of state, feature
It is, the selection enlargement module further includes full link block, and the full link block is for receiving first extreme value, second
Extreme value, third extreme value and the 4th extreme value, and by first extreme value, secondary extremal, third extreme value and the 4th extreme value
It is transmitted to the categorization module to classify, the categorization module is classified using SOFTMAX regression model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510290352.XA CN104954741B (en) | 2015-05-29 | 2015-05-29 | Profound self-teaching network implementations mine car sky expires the detection method and system of state |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510290352.XA CN104954741B (en) | 2015-05-29 | 2015-05-29 | Profound self-teaching network implementations mine car sky expires the detection method and system of state |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104954741A CN104954741A (en) | 2015-09-30 |
| CN104954741B true CN104954741B (en) | 2019-01-04 |
Family
ID=54169040
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510290352.XA Active CN104954741B (en) | 2015-05-29 | 2015-05-29 | Profound self-teaching network implementations mine car sky expires the detection method and system of state |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104954741B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106815596A (en) * | 2016-12-08 | 2017-06-09 | 中国银联股份有限公司 | A kind of Image Classifier method for building up and device |
| CN106874880A (en) * | 2017-02-21 | 2017-06-20 | 广东大仓机器人科技有限公司 | A Method of Enriching Face Recognition Samples |
| TWI643137B (en) * | 2017-04-21 | 2018-12-01 | 潘品睿 | Object recognition method and object recognition system |
| CN108881807A (en) * | 2017-05-09 | 2018-11-23 | 富士通株式会社 | Method and apparatus for being expanded the data in monitor video |
| CN107316289B (en) * | 2017-06-08 | 2020-05-08 | 华中农业大学 | Method for dividing rice ears in field based on deep learning and superpixel division |
| CN108510739A (en) * | 2018-04-28 | 2018-09-07 | 重庆交通大学 | A kind of road traffic state recognition methods, system and storage medium |
| CN109086737B (en) * | 2018-08-21 | 2021-11-02 | 武汉恒视途安科技有限公司 | Convolutional neural network-based shipping cargo monitoring video identification method and system |
| CN110060265A (en) * | 2019-05-15 | 2019-07-26 | 北京艺泉科技有限公司 | A method of divide from painting and calligraphy cultural relic images and extracts seal |
| CN112462788A (en) * | 2020-12-15 | 2021-03-09 | 济南浪潮高新科技投资发展有限公司 | Balance car automatic following implementation method and system based on mechanical vision and AI technology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003099778A (en) * | 2001-09-21 | 2003-04-04 | Koto Kado | Image recognition method and device |
| WO2009009697A1 (en) * | 2007-07-11 | 2009-01-15 | Behavioral Recognition Systems, Inc. | Cognitive model for a machine-learning engine in a video analysis system |
| CN104036323A (en) * | 2014-06-26 | 2014-09-10 | 叶茂 | Vehicle detection method based on convolutional neural network |
| CN104463241A (en) * | 2014-10-31 | 2015-03-25 | 北京理工大学 | Vehicle type recognition method in intelligent transportation monitoring system |
-
2015
- 2015-05-29 CN CN201510290352.XA patent/CN104954741B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003099778A (en) * | 2001-09-21 | 2003-04-04 | Koto Kado | Image recognition method and device |
| WO2009009697A1 (en) * | 2007-07-11 | 2009-01-15 | Behavioral Recognition Systems, Inc. | Cognitive model for a machine-learning engine in a video analysis system |
| CN104036323A (en) * | 2014-06-26 | 2014-09-10 | 叶茂 | Vehicle detection method based on convolutional neural network |
| CN104463241A (en) * | 2014-10-31 | 2015-03-25 | 北京理工大学 | Vehicle type recognition method in intelligent transportation monitoring system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104954741A (en) | 2015-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104954741B (en) | Profound self-teaching network implementations mine car sky expires the detection method and system of state | |
| CN105512640B (en) | A kind of people flow rate statistical method based on video sequence | |
| CN109902732B (en) | Automatic vehicle classification method and related device | |
| CN110321923B (en) | Target detection method, system and medium for fusion of feature layers of different scales of receptive fields | |
| CN111814621A (en) | A multi-scale vehicle pedestrian detection method and device based on attention mechanism | |
| CN110263697A (en) | Pedestrian based on unsupervised learning recognition methods, device and medium again | |
| CN108875754B (en) | A vehicle re-identification method based on multi-depth feature fusion network | |
| CN110097145A (en) | One kind being based on CNN and the pyramidal traffic contraband recognition methods of feature | |
| CN117152503A (en) | Remote sensing image cross-domain small sample classification method based on false tag uncertainty perception | |
| CN111199556A (en) | Indoor pedestrian detection and tracking method based on camera | |
| CN111368660A (en) | A single-stage semi-supervised image human object detection method | |
| CN112132200A (en) | Lithology identification method and system based on multi-dimensional rock image deep learning | |
| CN113610144A (en) | Vehicle classification method based on multi-branch local attention network | |
| CN114926733A (en) | Conveyor belt tearing detection method for improved regional convolutional neural network | |
| Jain et al. | Performance analysis of object detection and tracking algorithms for traffic surveillance applications using neural networks | |
| Xing et al. | Traffic sign recognition using guided image filtering | |
| CN112633149A (en) | Domain-adaptive foggy-day image target detection method and device | |
| Yin | Object Detection Based on Deep Learning: A Brief Review | |
| CN116778277A (en) | Cross-domain model training method based on progressive information decoupling | |
| CN119545164A (en) | Camera adaptive adjustment method, system and storage medium | |
| Gesnouin et al. | Assessing cross-dataset generalization of pedestrian crossing predictors | |
| CN108921872A (en) | A kind of robustness visual target tracking method suitable for long-range tracking | |
| Oni et al. | Optimized Fast R-CNN for Automated Parking Space Detection: Evaluating Efficiency with MiniFasterRCNN | |
| CN112949480A (en) | Rail elastic strip detection method based on YOLOV3 algorithm | |
| Wang et al. | Vehicle-Mounted Infrared Pedestrian Tracking Based on Scale Adaptive Kernel Correlation Filter. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |