CN106908038A - A kind of monitoring device and monitoring system based on fish eye lens video camera - Google Patents
A kind of monitoring device and monitoring system based on fish eye lens video camera Download PDFInfo
- Publication number
- CN106908038A CN106908038A CN201710004703.5A CN201710004703A CN106908038A CN 106908038 A CN106908038 A CN 106908038A CN 201710004703 A CN201710004703 A CN 201710004703A CN 106908038 A CN106908038 A CN 106908038A
- Authority
- CN
- China
- Prior art keywords
- fish eye
- monitoring device
- eye lens
- video camera
- strategy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012806 monitoring device Methods 0.000 title claims abstract description 74
- 241000251468 Actinopterygii Species 0.000 title claims abstract description 66
- 238000012544 monitoring process Methods 0.000 title claims abstract description 14
- 238000012545 processing Methods 0.000 claims description 13
- 230000000007 visual effect Effects 0.000 claims description 8
- 238000012795 verification Methods 0.000 claims description 7
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 239000000835 fiber Substances 0.000 claims description 4
- 230000006855 networking Effects 0.000 claims description 4
- 229920000728 polyester Polymers 0.000 claims description 4
- 238000009434 installation Methods 0.000 abstract description 10
- 238000004458 analytical method Methods 0.000 abstract description 6
- 230000003542 behavioural effect Effects 0.000 abstract description 3
- 230000004807 localization Effects 0.000 abstract 1
- 238000000034 method Methods 0.000 description 15
- 238000005516 engineering process Methods 0.000 description 7
- 238000005259 measurement Methods 0.000 description 7
- 241000208340 Araliaceae Species 0.000 description 6
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 6
- 235000003140 Panax quinquefolius Nutrition 0.000 description 6
- 235000008434 ginseng Nutrition 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 230000010365 information processing Effects 0.000 description 4
- 101100348047 Mus musculus Ncam2 gene Proteins 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000001681 protective effect Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 238000012935 Averaging Methods 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000005059 dormancy Effects 0.000 description 1
- 238000004134 energy conservation Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000010006 flight Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/36—Videogrammetry, i.e. electronic processing of video signals from a single source or from different sources to give parallax or range information
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Signal Processing (AREA)
- Closed-Circuit Television Systems (AREA)
- Studio Devices (AREA)
Abstract
The present invention relates to a kind of intelligent panoramic three-dimensional monitoring device and monitoring system based on fish eye lens video camera, system body is made up of multiple independent monitoring devices and special calibration car.Need the demarcation mode for carrying out complexity respectively different after installation is complete from traditional three-dimension monitor system based on image, the monitoring device designed in the present invention need to be only mounted and fixed on common street lamp or airfield runway, so as to substantially increase the convenience and practicality of system.System can also be realized surveying road traffic or airport landing region, monitored in addition to it can obtain the monitoring data on general statistical traffic data or airport, to purposes such as traffic unit identification, tracking, three-dimensional localization and behavioural analyses.
Description
Technical field
The present invention relates to image processing field, and in particular to a kind of intelligent panoramic three-dimensional position based on fish eye lens video camera
Put monitoring device and monitoring system.
Background technology
With the development of the technologies such as information technology, data communication transmission, Electronic transducer technology, computer technology, intelligence
Can traffic system be increasingly subject to various countries attention, the development of intelligent transportation system, be not limited to solve traffic congestion, traffic accident,
The problems such as traffic pollution.Intelligent transportation system can effectively utilize existing means of transportation and go to reduce traffic loading, it is ensured that traffic is pacified
Entirely, conevying efficiency is improved, for traffic participant provides various sex service, is made one, the interaction relationship between car, road is with brand-new
Mode present, intelligent transportation system is built upon on the basis of perfect information network, by sensor, detector and transmission
Equipment etc. is carried out to the Real-time Traffic Information on traffic information center transmission road, traffic information center to the data that transmission is come in
Treatment, measures to road traffic according to the information after treatment, controls, surveys.
In civil aviaton field, with the fast development of national economy, the sortie of taking off and landing of major airport airborne vehicles is on the increase, very
To reaching or beyond the maximum allowable landing number of times of design, causing the potential safety hazards such as runway intrusion event to happen occasionally, to boat
Empty safety belt carrys out significant threat.So need also exist for one kind can on the premise of existing equipment is not changed can on runway fly
Machine behavior carries out intelligent omnibearing monitoring and pre-warning system.
, can be by being arranged on video camera on street lamp or setting up in addition by the intelligent transportation system based on video detection technology
Video camera on road or airfield runway obtains road traffic video image.But traditional general camera lens, visual field is smaller,
It is difficult to shoot the video for obtaining large area using less camera lens, and uses fish eye lens video camera, although can obtains larger
Visual field, but demarcate difficulty it is higher, while more inconvenience.
The content of the invention
It is a first object of the present invention to provide a kind of monitoring device based on fish eye lens video camera, can during installation
To set up the position orientation relation of fish eye lens and earth axes, calibration cost is reduced, solve above technical problem；
It is a second object of the invention to provide a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera
System, can directly demarcate the relative pose relation of fish eye lens video camera in adjacent detection means after installation.
Technical problem solved by the invention can be realized using following technical scheme：
In order to realize the first object of the present invention, there is provided a kind of intelligent panoramic three-dimension monitor based on fish eye lens video camera
System, including housing and fish eye lens video camera,
The enclosure interior is provided with inertial sensor and core processing module, wireless transport module, and the inertia is passed
Sensor includes the three axis accelerometer module, the magnetometer module, the three axis accelerometer module, and the inertial sensor is consolidated
Closed due to there is the first relative pose between the enclosure interior, the inertial sensor module and the fish eye lens video camera
System,
The three axis accelerometer module is used to determine first absolute pose of the inertial sensor in earth coordinates；
The magnetometer module is used to determine the inertial sensor in the first direction relation of earth coordinates north orientation；
The three axis accelerometer module is used to determine the inertial sensor with the first of the horizontal plane of earth coordinates
Tilt relationship；
The processing module is configured with the first strategy, and first strategy is according to the described first absolute pose, described first
Direction relationses and first tilt relationship obtain the inertial sensor with the second relative position in the earth coordinates
Appearance relation；
The processing module is configured with the second strategy, and second strategy is according to the first relative pose relation and the
Two relative pose relations obtain threeth position orientation relation of the fish eye lens video camera in earth coordinates；
The enclosure interior of each monitoring device is respectively arranged with GPS module, and the GPS module is used to receive
GPS position information and GPS time information；
The GPS position information that the monitoring device is received according to it establishes corresponding absolute position；
The GPS time information that the monitoring device is received according to it sets up corresponding time reference.
In order to realize the second object of the present invention, there is provided a kind of intelligent panoramic three-dimension monitor based on fish eye lens video camera
System, it is characterised in that include at least two above-mentioned monitoring devices based on fish eye lens video camera being disposed adjacent, institute
Stating intelligent panoramic three-dimension monitor system configuration has the 3rd strategy, and the 3rd strategy is imaged according to each fish eye lens respectively
The 3rd position orientation relation in machine and earth axes, obtains the relative pose relation of the adjacent fish eye lens video camera.
Further, the monitoring device also includes wireless module, and the adjacent monitoring device passes through the wireless mould
Block completes networking.
Further, also including a calibration vehicle, the calibration vehicle is provided with pre- a demarcation and verifies figure；The intelligent panoramic
Three-dimension monitor system configuration has the 4th strategy, and the 4th strategy is included when the calibration vehicle is moved, the adjacent monitoring
Device synchronous acquisition demarcates verification figure to correct the corresponding relative pose relation in advance.
Further, also including a calibration vehicle, the calibration vehicle is provided with pre- a demarcation and verifies figure；The intelligent panoramic
Three-dimension monitor system configuration has the 5th strategy, and the 5th strategy includes that the monitoring device is adopted when the calibration vehicle is moved
Collection is pre- to be demarcated verification figure to correct the corresponding pre- demarcation intrinsic parameter.
Further, also including processing center, each monitoring device by wireless module and the processing center communicate with
Transmit the view data of the monitoring device collection；The processing center receives view data and according to the relative pose relation
The view data for splicing corresponding adjacent monitoring device collection forms the first view data.
Further, the processing center is configured with the 6th strategy, and the 6th strategy is according to the relative pose relation
Calculate the depth value for obtaining any feature point in described first image data.
Further, the monitoring device is additionally provided with structured light projector.
Further, when target is located in multiple adjacent monitoring device visual fields simultaneously, the processing center is configured with the
Seven strategies, the 7th strategy carries out adjustment according to the triangle polyester fibre result of multiple monitoring devices.
Further, when surface road is monitored, the processing center is configured with the 7th strategy, the 7th strategy
Using the plane restriction of road.
Beneficial effect：Due to using above technical scheme, system body is by multiple independent monitoring devices and special mark
School bus is constituted.Need the mode demarcated different after installation is complete from traditional image monitoring system, designed in the present invention
Monitoring device need to only be mounted and fixed on common street lamp or be directly anchored on airfield runway, and complexity need not be carried out
Professional staking-out work, so as to substantially increase the convenience and practicality of system.
Brief description of the drawings
Fig. 1 is the monitoring device structural representation of the embodiment of the present invention 1；
Fig. 2 is that the dolly in the embodiment of the present invention 2 demarcates scene graph；
Fig. 3 is that the dolly in the embodiment of the present invention 3 demarcates scene graph；
Fig. 4 is implementing procedure figure of the present invention.
Reference：1st, fish eye lens video camera；2nd, differential GPS module；3rd, three axis accelerometer module；4th, magnetometer module；
5th, accelerometer module；6th, wireless communication module；7th, power module；8th, structured light projector；9th, processing module；10th, housing.
Specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art obtained on the premise of creative work is not made it is all its
His embodiment, belongs to the scope of protection of the invention.It should be noted that in the case where not conflicting, the implementation in the present invention
Feature in example and embodiment can be mutually combined.
The invention will be further described with specific embodiment below in conjunction with the accompanying drawings, but not as limiting to the invention.
Embodiment 1 is supervising device, first, described in reference picture 1, a kind of intelligent panoramic based on fish eye lens video camera 1
Three-dimension monitor system, including housing 10 and fish eye lens video camera 1.
The housing 10 is internally provided with inertial sensor and core processing module 9, wireless transport module, the inertia
Sensor includes the three axis accelerometer module 3, the magnetometer module 4, the three axis accelerometer module 5, and the inertia is passed
Sensor is fixed on inside the housing 10, has the first phase between the inertial sensor module and the fish eye lens video camera 1
To position orientation relation,
The three axis accelerometer module 3 is configured with the first absolute pose for representing the inertial sensor position；
The magnetometer module 4 is used to determine the inertial sensor in the first direction on the horizontal plane of earth coordinates
Relation；
The three axis accelerometer module 5 is used to determine the inertial sensor with the first of the horizontal plane of earth coordinates
Tilt relationship；
The processing module 9 is configured with the first strategy, first strategy according to the three axis accelerometer module 3 described the
One absolute pose, the first direction relation and first tilt relationship obtain the inertial sensor in a ground and institute
State the second relative pose relation in earth coordinates；
The processing module 9 is configured with the second strategy, second strategy according to the first relative pose relation and
Second relative pose relation obtains threeth position orientation relation of the fish eye lens video camera 1 in earth axes.
The monitoring device is provided with temperature controller, heat abstractor and louvre, and the temperature controller drives according to environment temperature
Dynamic heat abstractor work.Differential GPS module 2, the difference are respectively arranged with inside the housing 10 of each monitoring device
Point GPS module 2 is used to receive GPS time information；When the GPS time information that the monitoring device is received according to it sets up corresponding
Between benchmark.The monitoring device is additionally provided with structured light projector.
Shown in reference picture 1, the composition structure of intelligent panoramic three-dimension monitor unit is as shown in Fig. 2 single monitoring device is main
Including fish eye lens video camera 1, differential GPS module 2, three axis accelerometer module 3, magnetometer module 4, accelerometer module 5, wireless
Communication module 6, power module 7, structured light projector 8, processing module 9, housing 10.Wherein, housing 10 is above-mentioned modules
, its external stability fish eye lens video camera 1 is fixed with differential GPS module 2, three axis accelerometer module 3, magnetic inside housing 10
Power meter module 4, accelerometer module 5, wireless communication module 6, power module 7, structured light projector 8, intelligent kernel processor
Module.As shown in figure 1, fish-eye camera is down-set and installs, for shooting road surface.Because the present apparatus needs to be chronically exposed to
Outdoor, it is therefore desirable to add special protective unit, protective unit includes a guard shield and temperature controller, guard shield peace
Mounted in three dimensional reconstructive CT system shell outside, as preferred, one temperature controller is installed in the side of guard shield, on right side
Open up heat abstractor and louvre.Because fish-eye camera has broader industry, the present invention preferably angle of visual field is 210 degree
Fish-eye camera, can be used to obtain the monitoring video information in big field range, substantially reduce intelligent transportation system into
This.
And next the principle of work and power to monitoring device is explained, fish eye lens video camera 1 is used for shooting road or machine
Field runway video image, differential GPS module 2 is used for the time synchronized between measurement baseline length and the adjacent camera of offer, three axles
Gyroscope modules 3, magnetometer module 4, accelerometer module 5 are used for demarcating the position orientation relation of adjacent flake camera, wireless telecommunications mould
Block 6 is used for transmitting video image to traffic information center, facilitates supervision of the traffic information center to video image, power module 7
For providing system and device power supply；Structured light projector 8 is preferred embodiment, and for the occasion high to required precision, structure light is thrown
Emitter 8 can provide the illumination around target object as special light source, not enough in no external light source or external light source
Under the conditions of, fish eye camera can also work, under conditions of having external light source, structure luminous energy enhancing steric information, so that fish
The three dimensional depth image that eye camera is obtained is more accurate, therefore, in the middle of practical application, can be come according to specific environment
Choose whether to use structured light projector 8.Processing module 9, for realizing the control of all modules and the treatment of data.
Advantage of the invention is by only apparatus of the present invention need to be fixedly mounted on street lamp and completes measurement task, fixed
The Rankine-Hugoniot relations and internal and external parameter of each monitoring device are fixed after installation, wherein, the interior participation reality of each monitoring device
Installation operation when border uses is unrelated, it is necessary to the first mark unified when dispatching from the factory, or can also be united when design condition is excellent
One uses theoretical value, that is to say, that the pre- demarcation intrinsic parameter of each monitoring device, it is known that and the outer ginseng of monitoring device due to reality
The installation on border is related, so cannot be determined in advance, it is necessary to carry out automatic Calibration using specific technological means.So, follow-up
When video image to shooting is processed, it is possible to which the target information that different flake cameras are obtained is projected to common coordinate by ground
Under system, and this common coordinate frame, elect earth axes as in the present invention, no matter how installation site changes, geographical coordinates
System will not change, so as to realize the purpose of three-dimensional measurement.
The pre- acquisition for demarcating intrinsic parameter, it is necessary to first respectively to the internal reference of each monitoring device before monitoring device installation
Demarcated, especially in monitoring device fish eye camera demarcation.The internal reference scaling method of fish eye camera has various, such as
Modelling, Ocam methods etc., will not be described here.Wherein modelling is several firstly the need of fish-eye projecting method is calibrated
Common projecting method have stereo projection, equidistant projection, etc. three-dimensional angular projection, rectangular projection etc., in addition it is also necessary to calibrate imaging
Radius and imaging center coordinate (cx,cy).And for Ocam standardizations, then need to calibrate the multinomial of fitting imaginary surface
Coefficient, imaging center coordinate (cx,cy) and the advance demarcation intrinsic parameter of affine transformation coefficient acquisition.
In order to the outer ginseng to monitoring device carries out automatic Calibration, the present invention is by by GPS module, gyroscope modules, magnetic force
Meter module 4 and accelerometer module 5 are demarcated come the outer ginseng indirectly to monitoring device, and this is accomplished by calibrating this in advance
A little relations between module local Coordinate System and fish eye camera coordinate system.Can be by using similar to Robotic Hand-Eye Calibration
Method, (and stand a period of time, measurement noise eliminated with by averaging by repeatedly rotating monitoring device device
Influence) method, calibrate the relation between fish eye camera coordinate system and gyro group coordinate system, that is, the first relative position
Appearance, then the data acquisition three axis accelerometer module 3 and the position orientation relation of magnetic north pole for passing through gyro group and magnetometer, by three axis accelerometer
Module 3 counts the position orientation relation for obtaining three axis accelerometer module 3 and horizontal plane with acceleration of gravity, you can to obtain gyroscope seat
Relation between mark system and earth coordinates, that is, the second relative pose, using earth coordinates as the frame of reference, so that it may
To obtain third phase of the fish eye lens video camera 1 relative to earth coordinates to pose, it is possible to calculate adjacent monitoring indirectly
Position orientation relation between the gyro group of device, it is hereby achieved that the relative pose relation of two neighboring fish eye camera.
Separately, GPS realizes the time synchronized between adjacent monitoring device, so as to reach the mesh of all detection means time synchronizeds
's.Synchronization control module is corrected by periodically being contrasted with the gps time for receiving, in each PPS porch, reception
To location data (including time) and the system time of traffic information center be mapped.So, traffic information center is
The system time just can be corresponding with the UTC time of GPS, and each three dimensional reconstructive CT system is corresponding with the fiducial time of GPS, then whole
The time of individual system can just be mapped, you can complete the synchronization between different fish eye cameras.
In example 2, a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera 1, it is characterised in that
At least two above-mentioned monitoring devices based on fish eye lens video camera 1 being disposed adjacent are included, the intelligent panoramic is three-dimensional
Monitoring system is configured with the 3rd strategy, and the 3rd strategy is respectively according to each fish eye lens video camera 1 in geographical coordinates
The 3rd position orientation relation in system, obtains the relative pose relation of the adjacent fish eye lens video camera 1.Differential GPS module 2 can
To determine the relative position of adjacent fish eye lens video camera 1.The monitoring device also includes wireless module, the adjacent dress
Put and networking is completed by the wireless module.Also include processing center, each monitoring device passes through wireless module and the treatment
Center communicates to transmit the view data that the monitoring device is gathered；The processing center receives view data and according to the phase
The view data of adjacent monitoring device corresponding to position orientation relation splicing collection forms the first view data.The system dress is installed
Put, it is ensured that point in monitored area can be photographed by two fish eye lens video cameras simultaneously, and obtain road or airfield runway is regarded
Frequency image.
Processing center placement algorithm is used for deep learning, and deep learning is a new research field of machine learning, it
Attempt the mechanism by allowing computer simulation human brain to perceive visual signal, and then design the network of deep layer to realize the work(of vision
Energy.Deep learning can with unsupervised from data learning to feature, therefore, when sample is sufficiently large, by depth network science
The feature practised often has certain semantic feature, is suitable for identification, the tracking of target, behavioural analysis etc..Depth
The advantage of habit is that the ability to express of model is strong, can preferably locate the more complicated problem such as target recognition, behavioural analysis, is learned
Practise more complicated functional relation.Target identification system of the present invention based on deep learning, by gathering and utilizing a large amount of roads
Sample (such as pedestrian, vehicle, aircraft etc.) on road or airfield runway, trains deep neural network, then to being wirelessly transferred mould
The video image that block is passed back carries out target identification, tracking.
The 6th strategy is explained below, processing center is configured with the 6th strategy, and the 6th strategy is according to institute
State relative pose relation and calculate the depth value for obtaining any feature point in described first image data.By deep learning network pair
Target on road or airfield runway is identified classification, and then extracting our targets interested carries out three-dimensional reconstruction.Obtain
Pedestrian, the three-dimensional information of vehicle or depth information are significant, such as detecting, survey traffic accident, violations of rules and regulations
Deng.
The present invention is inferred to the depth of each characteristic point of target according to triangulation.Fish eye camera demarcates it
Afterwards, we can obtain the projection matrix P of each video camera, if the coordinate of a pair of Feature Points Matchings pair is respectively (xi,yi)、
(xj,yj), had according to triangulation：
Similarly, X is abscissa value, and Y is ordinate value, and pij(i, j are 1-3) then represents the projection matrix of correspondence label,
Coordinate points (xj,yj) a prescription journey can also be obtained, W=1 is made, being solved with conventional least square method can calculate characteristic point
D coordinates value, wherein Z values are the depth value of this feature point, and final we can obtain the three-dimensional letter of interesting target
Breath.
After the completion of the prior demarcation of internal reference and the automatic Calibration of outer ginseng, in order to further improve certainty of measurement, also for
To the regular amendment of calibration result, can be examined by starting specific demarcation on road surface after three-dimensional monitoring device installation
School bus carries out calibration amendment to the result demarcated, and the top of calibration car is the plane or three-dimensional mark of known geometry physical relation
Will thing, as Figure 2-3.The system also includes a calibration vehicle, and the calibration vehicle is provided with pre- a demarcation and verifies figure；It is described
Intelligent panoramic three-dimension monitor system configuration has the 4th strategy, and the 4th strategy is included when the calibration vehicle is moved, adjacent
The monitoring device synchronous acquisition demarcates verification figure to correct the corresponding relative pose relation in advance.Also include a verification
Car, the calibration vehicle is provided with pre- a demarcation and verifies figure；The intelligent panoramic three-dimension monitor system configuration has the 5th strategy, institute
Stating the 5th strategy is included when the calibration vehicle is moved, and the monitoring device collection is pre- to demarcate verification figure to correct corresponding institute
State pre- demarcation intrinsic parameter.
Also include processing center, each monitoring device is communicated to transmit by wireless transport module with the processing center
State view data, position data, the attitude data of monitoring device collection；The processing center receives the picture number of each monitoring device
According to, position data, attitude data, monitoring device ID data and according between the adjacent monitoring device relative pose relation utilize
Triangle polyester fibre method solves the three-dimensional space position of target of the same name in public view field.When simultaneously target is located at multiple (being more than 2)
When in adjacent monitoring device visual field, the processing center is configured with the 6th strategy, and the 6th strategy is according to multiple monitoring devices
Triangle polyester fibre result carry out adjustment, so as to improve certainty of measurement.When surface road is monitored, the processing center configuration
There is the 7th strategy, the 7th strategy further improves certainty of measurement using the plane restriction of road.
The principle that dolly is demarcated is illustrated below, the pose of the adjacent fish eye camera obtained by said process is closed
System there may be certain error, or nominal data has slow change (such as due to the four seasons due to the influence of external environment
What the temperature change for converting and producing caused expands with heat and contract with cold), it is therefore desirable to monitoring device is further corrected.
One block of plane reference plate is carried on calibration vehicle, by synchronization control module to public in adjacent fish eye camera
Calibration vehicle in visual field is imaged, then two position orientation relations of adjacent fish-eye camera can be repaiied by the following method
Just：
Assuming that P=[X Y 0 1]TIt is the homogeneous world coordinates of planar target, p=[u v 1]TIt is corresponding target picture point
Homogeneous coordinates,It is fish eye camera Intrinsic Matrix, R is camera coordinate system relative to world coordinate system
Spin matrix, t be camera coordinate system relative to world coordinate system translation vector, from perspective projection model：
Sp=K [R t] P
(1)
Can further obtain：
Wherein, K is the prior fish eye camera internal reference matrix demarcated, and can obtain above-mentioned etc. for each characteristic point
Formula, therefore multiple characteristic points are obtained with multiple equatioies, finally, according to the characteristic point of the plane reference plate on calibration vehicle, can
To obtain [r1 r2T], you can the purpose being modified to R and t is reached, can also be by connection between multiple adjacent monitoring devices
The method for closing adjustment further improves the precision of amendment.
Embodiment 2 is applied to be arranged on highway both sides, for surveying the traffic of this area.By present system
It is fixed on the street lamp of highway both sides, and fish eye camera is demarcated.The video image of highway is shot, and is led to
Cross wireless transport module and pass transport information processing center back in real time.With this patent method to be transmitted back to come video image at
Reason and analysis.Output result, realization is surveyed to road traffic, monitored, to traffic unit identification, tracking and behavior
The purposes such as analysis.
Embodiment 3 is applied to be arranged on the indicator lamp of airfield runway both sides, for monitoring all kinds of flights of traffic pattern
Device landing, intrusion of the improper target of airfield runway etc..By present system be fixed on airfield runway both sides indicator lamp or its
His suitable place, and fish eye camera is demarcated.The video image of traffic pattern is shot, and by wireless transport module
Pass transport information processing center back in real time.Video image to being transmitted back to come is processed and analyzed.Output result, realizes
Airport landing region is surveyed, is monitored, identification, tracking and row to related individuals such as all kinds of aircraft, personnel, vehicles
It is the purposes such as analysis.
Shown in reference picture 4, monitoring system implementing procedure of the invention is introduced,
1st, corresponding functional module is selected and in transport information processing center and detection means according to actual conditions
By program Predistribution Algorithm；
2nd, internal reference staking-out work is carried out to monitoring device, it is preferred to use Ocam methods calibrate the internal reference of each fish eye camera
Number；
3rd, the present apparatus is arranged on the street lamp or the body of rod of selected road or airfield runway both sides；Wherein, handed over for road
Shooting is installed upside down during logical monitoring, and shooting is head-up installed when being monitored for airfield runway aircraft；
4th, adjacent monitoring device connection networking is carried out using the mode of radio communication；Using sparse signal treatment technology to each
Measuring unit carries out intelligent dormancy dispatching, the purpose for reach energy-conservation, increasing the service life；
5th, carry out outer ginseng to demarcate, calibrate the position orientation relation of fish eye camera and corresponding gyro group, then calibrate top
Relation between spiral shell group and earth coordinates, such that it is able to obtain the relation between adjacent gyro group coordinate system indirectly, finally obtains
Obtain the position orientation relation between adjacent fish eye camera；
6th, travelled one time on road or airfield runway using calibration vehicle (roof carries one block of specific plane reference plate)
Or many times, each adjacent monitoring device is carried out into by synchronization control module to the calibration vehicle in adjacent fish eye camera public view field
Picture, so as to be modified to above internal reference obtained by calibrating and outer ginseng；
7th, road or airfield runway video image are gathered, transport information processing center is passed back by wireless transport module, profit
Video image is analyzed with interpretation software；
8th, the target in video image is identified, classified, track algorithm；
9th, feature extraction is carried out to target interested and corresponding Feature Points Matching is obtained to gathering, triangulation,
The depth value of each characteristic point is obtained, the three-dimensional information of interesting target may finally be obtained.
Preferred embodiments of the present invention are the foregoing is only, embodiments of the present invention and protection model is not thereby limited
Enclose, to those skilled in the art, should can appreciate that done by all utilization description of the invention and diagramatic content
Scheme obtained by equivalent and obvious change, should be included in protection scope of the present invention.
Claims (10)
1. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera, including housing and fish eye lens are imaged
Machine, it is characterised in that：
The enclosure interior is provided with inertial sensor and core processing module, wireless transport module, the inertial sensor
Including the three axis accelerometer module, the magnetometer module, the three axis accelerometer module, the inertial sensor is fixed on
The enclosure interior, there is the first relative pose relation between the inertial sensor module and the fish eye lens video camera,
The three axis accelerometer module is used to determine first absolute pose of the inertial sensor in earth coordinates；
The magnetometer module is used to determine the inertial sensor in the first direction relation of earth coordinates north orientation；
The three axis accelerometer module is used to determine the first inclination of the inertial sensor and the horizontal plane of earth coordinates
Relation；
The processing module is configured with the first strategy, and first strategy is according to the described first absolute pose, the first direction
Relation and first tilt relationship obtain the inertial sensor and are closed with the second relative pose in the earth coordinates
System；
The processing module is configured with the second strategy, and second strategy is according to the first relative pose relation and the second phase
Threeth position orientation relation of the fish eye lens video camera in earth coordinates is obtained to position orientation relation；
The enclosure interior of each monitoring device is respectively arranged with GPS module, and the GPS module is used to receive GPS
Confidence ceases and GPS time information；
The GPS position information that the monitoring device is received according to it establishes corresponding absolute position；
The GPS time information that the monitoring device is received according to it sets up corresponding time reference.
2. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera, it is characterised in that include at least two
The monitoring device based on fish eye lens video camera as claimed in claim 1 being disposed adjacent, the intelligent panoramic three-dimension monitor
System configuration has the 3rd strategy, and the 3rd strategy is respectively according in each fish eye lens video camera and earth axes
3rd position orientation relation, obtains the relative pose relation of the adjacent fish eye lens video camera.
3. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera according to claim 2, its feature
It is that the monitoring device also includes wireless module, the adjacent monitoring device completes networking by the wireless module.
4. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera according to claim 2, its feature
It is that, also including a calibration vehicle, the calibration vehicle is provided with pre- a demarcation and verifies figure；The intelligent panoramic three-dimension monitor system
The 4th strategy is configured with, the 4th strategy is included when the calibration vehicle is moved, the adjacent monitoring device synchronous acquisition
It is pre- to demarcate verification figure to correct the corresponding relative pose relation.
5. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera according to claim 2, its feature
It is that, also including a calibration vehicle, the calibration vehicle is provided with pre- a demarcation and verifies figure；The intelligent panoramic three-dimension monitor system
The 5th strategy is configured with, the 5th strategy is included when the calibration vehicle is moved, the monitoring device collection is pre- to demarcate verification
Figure is correcting the corresponding pre- demarcation intrinsic parameter.
6. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera according to claim 2, its feature
It is that, also including processing center, each monitoring device communicates to transmit the monitoring by wireless module with the processing center
The view data of device collection；The processing center receives view data and splices corresponding phase according to the relative pose relation
The view data of adjacent monitoring device collection forms the first view data.
7. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera according to claim 6, its feature
It is that the processing center is configured with the 6th strategy, and the 6th strategy is according to the relative pose relation calculates acquisition
The depth value of any feature point in first view data.
8. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera according to claim 2, its feature
It is that the monitoring device is additionally provided with structured light projector.
9. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera according to claim 2, its feature
It is that, when target is located in multiple adjacent monitoring device visual fields simultaneously, the processing center is configured with the 7th strategy, described the
Seven strategies carry out adjustment according to the triangle polyester fibre result of multiple monitoring devices.
10. a kind of intelligent panoramic three-dimension monitor system based on fish eye lens video camera according to claim 2, its feature
It is that when surface road is monitored, the processing center is configured with the 7th strategy, and the 7th strategy is using the flat of road
Face constrains.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710004703.5A CN106908038B (en) | 2017-01-04 | 2017-01-04 | A kind of monitoring device and monitoring system based on fish eye lens video camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710004703.5A CN106908038B (en) | 2017-01-04 | 2017-01-04 | A kind of monitoring device and monitoring system based on fish eye lens video camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106908038A true CN106908038A (en) | 2017-06-30 |
| CN106908038B CN106908038B (en) | 2019-01-25 |
Family
ID=59207234
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710004703.5A Active CN106908038B (en) | 2017-01-04 | 2017-01-04 | A kind of monitoring device and monitoring system based on fish eye lens video camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106908038B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107655472A (en) * | 2017-08-30 | 2018-02-02 | 杨华军 | A kind of high-precision inertial navigation set error compensating method based on deep learning |
| CN108121968A (en) * | 2017-12-20 | 2018-06-05 | 佛山市洛克威特科技有限公司 | A kind of fish monitoring method |
| CN108717704A (en) * | 2018-05-15 | 2018-10-30 | 珠海全志科技股份有限公司 | Method for tracking target, computer installation based on fish eye images and computer readable storage medium |
| CN110706280A (en) * | 2018-09-28 | 2020-01-17 | 成都家有为力机器人技术有限公司 | Lightweight semantic driven sparse reconstruction method based on 2D-SLAM |
| CN112862818A (en) * | 2021-03-17 | 2021-05-28 | 合肥工业大学 | Underground parking lot vehicle positioning method combining inertial sensor and multi-fisheye camera |
| CN114445727A (en) * | 2020-10-30 | 2022-05-06 | 虫极科技（北京）有限公司 | Method for detecting three-dimensional object from panoramic video and restoring pose with six degrees of freedom |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102175221A (en) * | 2011-01-20 | 2011-09-07 | 上海杰图软件技术有限公司 | Vehicle-mounted mobile photographic surveying system based on fisheye lens |
| CN203405206U (en) * | 2013-09-06 | 2014-01-22 | 中科宇图天下科技有限公司 | Panoramic image unmanned aerial vehicle acquisition system |
| US20150351200A1 (en) * | 2012-12-20 | 2015-12-03 | Zumtobel Lighting Gmbh | Method for addressing light fixtures, light fixture for lighting and system for lighting a room |
-
2017
- 2017-01-04 CN CN201710004703.5A patent/CN106908038B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102175221A (en) * | 2011-01-20 | 2011-09-07 | 上海杰图软件技术有限公司 | Vehicle-mounted mobile photographic surveying system based on fisheye lens |
| US20150351200A1 (en) * | 2012-12-20 | 2015-12-03 | Zumtobel Lighting Gmbh | Method for addressing light fixtures, light fixture for lighting and system for lighting a room |
| CN203405206U (en) * | 2013-09-06 | 2014-01-22 | 中科宇图天下科技有限公司 | Panoramic image unmanned aerial vehicle acquisition system |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107655472A (en) * | 2017-08-30 | 2018-02-02 | 杨华军 | A kind of high-precision inertial navigation set error compensating method based on deep learning |
| CN107655472B (en) * | 2017-08-30 | 2019-11-01 | 杨华军 | A kind of high-precision inertial navigation set error compensating method based on deep learning |
| CN108121968A (en) * | 2017-12-20 | 2018-06-05 | 佛山市洛克威特科技有限公司 | A kind of fish monitoring method |
| CN108717704A (en) * | 2018-05-15 | 2018-10-30 | 珠海全志科技股份有限公司 | Method for tracking target, computer installation based on fish eye images and computer readable storage medium |
| CN110706280A (en) * | 2018-09-28 | 2020-01-17 | 成都家有为力机器人技术有限公司 | Lightweight semantic driven sparse reconstruction method based on 2D-SLAM |
| CN114445727A (en) * | 2020-10-30 | 2022-05-06 | 虫极科技（北京）有限公司 | Method for detecting three-dimensional object from panoramic video and restoring pose with six degrees of freedom |
| CN112862818A (en) * | 2021-03-17 | 2021-05-28 | 合肥工业大学 | Underground parking lot vehicle positioning method combining inertial sensor and multi-fisheye camera |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106908038B (en) | 2019-01-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106908038B (en) | A kind of monitoring device and monitoring system based on fish eye lens video camera | |
| CN102654940B (en) | Processing method of traffic information acquisition system based on unmanned aerial vehicle and | |
| CN107202982B (en) | A kind of beacon arrangement and image processing method based on UAV position and orientation calculating | |
| CN110322702A (en) | A kind of Vehicular intelligent speed-measuring method based on Binocular Stereo Vision System | |
| CN106441233B (en) | Power channel corridor method for inspecting based on oblique photograph three-dimensional reconstruction | |
| CN107194989A (en) | The scene of a traffic accident three-dimensional reconstruction system and method taken photo by plane based on unmanned plane aircraft | |
| CN108832997A (en) | A kind of unmanned aerial vehicle group searching rescue method and system | |
| CN106412502B (en) | Intelligent Skynet System for Video Surveillance Based on Face Recognition and Its Working Method | |
| CN109479119A (en) | System and method for UAV interactive video broadcasting | |
| CN105812733A (en) | Civil aviation air traffic control scene monitoring and guiding system | |
| CN109547769B (en) | A highway traffic dynamic three-dimensional digital scene acquisition and construction system and its working method | |
| CN103226838A (en) | Real-time spatial positioning method for mobile monitoring target in geographical scene | |
| CN106375713B (en) | A video surveillance intelligent skynet system and its working method | |
| CN110197097B (en) | Harbor district monitoring method and system and central control system | |
| CN109946729A (en) | A kind of air target tracking method and device | |
| CN114252868A (en) | Lidar calibration method, device, computer equipment and storage medium | |
| CN109819223A (en) | A kind of unmanned plane and the intelligent video monitoring system based on recognition of face | |
| CN114295139A (en) | Cooperative sensing positioning method and system | |
| CN114252883A (en) | Target detection method, apparatus, computer device and medium | |
| US10713527B2 (en) | Optics based multi-dimensional target and multiple object detection and tracking method | |
| CN116295290A (en) | A 3D modeling and mapping method for high-rise buildings based on laser scanning | |
| CN114252884A (en) | Method and device for positioning and monitoring roadside radar, computer equipment and storage medium | |
| CN106384097B (en) | Intelligent video surveillance system based on face recognition | |
| CN114255264B (en) | Multi-base-station registration method and device, computer equipment and storage medium | |
| CN207068060U (en) | The scene of a traffic accident three-dimensional reconstruction system taken photo by plane based on unmanned plane aircraft |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |