CN112954274B - Video snapshot method and system for ship - Google Patents
Video snapshot method and system for ship Download PDFInfo
- Publication number
- CN112954274B CN112954274B CN202110156215.2A CN202110156215A CN112954274B CN 112954274 B CN112954274 B CN 112954274B CN 202110156215 A CN202110156215 A CN 202110156215A CN 112954274 B CN112954274 B CN 112954274B
- Authority
- CN
- China
- Prior art keywords
- target
- dome camera
- image
- dome
- target detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000001514 detection method Methods 0.000 claims abstract description 88
- 238000013527 convolutional neural network Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 3
- 230000006872 improvement Effects 0.000 description 12
- 230000008569 process Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a video snapshot method for a ship, which comprises the following steps: carrying out target detection on the video stream of the gunlock to obtain a target detection result, wherein the target detection result comprises position information and confidence information of a target; carrying out target tracking on video stream of the gunlock, acquiring position information of a target in a shooting image of the gunlock, and determining the distance and the direction of the target relative to the ball machine according to the calibration relation between the gunlock and the ball machine so as to determine the moving direction and the moving speed of the target; adjusting the horizontal angle and the pitch angle of the dome camera according to the movement speed of the target, so that the target is positioned at the central position of the shot image of the dome camera to obtain the real-time snapshot position of the dome camera; the ball machine continuously shoots the target at a real-time shooting position at a certain time interval. The invention also discloses a video snapshot system for the ship. The invention can set the snapshot position in a self-adaptive way according to different speeds of the ship, thereby improving the snapshot effectiveness and accuracy.
Description
Technical Field
The invention relates to the technical field of ships, in particular to a video snapshot method and system for a ship.
Background
In the related art, for ship snapshot monitoring, a target is difficult to lock due to delay of a photoelectric device during shooting, when a camera moves to a specified position, the target leaves an image picture, and especially for a ship with a high moving speed, the moving speed of the camera is far lower than the moving speed of the ship, so that the target cannot be snapshot at an optimal time.
Disclosure of Invention
In order to solve the above problems, an object of the present invention is to provide a method and a system for video snapshot of a ship, which can adaptively set a snapshot position according to different speeds of the ship, and improve the snapshot effectiveness and accuracy.
The invention provides a video snapshot method for a ship, which comprises the following steps:
carrying out target detection on a video stream of a gunlock to obtain a target detection result, wherein the target detection result comprises position information and confidence information of a target;
carrying out target tracking on the video stream of the gunlock, acquiring position information of the target in the image shot by the gunlock, and determining the movement direction and the movement speed of the target according to the calibration relation between the gunlock and the ball machine;
adjusting the horizontal angle and the pitch angle of the dome camera according to the movement speed of the target, so that the target is positioned at the central position of the image shot by the dome camera to obtain the real-time snapshot position of the dome camera;
and the ball machine continuously shoots the target at the real-time shooting position at a certain time interval.
As a further improvement of the present invention, the method further comprises: and carrying out linkage calibration on the rifle bolt and the ball machine according to the video streams of the rifle bolt and the ball machine to obtain the calibration relation between the rifle bolt and the ball machine.
As a further improvement of the present invention, the performing linkage calibration on the rifle bolt and the ball machine according to the video streams of the rifle bolt and the ball machine to obtain a calibration relationship between the rifle bolt and the ball machine includes:
acquiring video streams of the rifle bolt and the ball machine;
determining dome camera parameters respectively corresponding to a plurality of gun camera pixel points in the gun camera image according to the video stream of the gun camera and the video stream of the dome camera, wherein the dome camera parameters comprise a horizontal angle and a pitch angle;
performing fitting calibration on the dome camera parameters respectively corresponding to the plurality of rifle bolt pixel points by using a least square method to obtain dome camera parameters corresponding to each rifle bolt pixel point in the rifle bolt image;
and determining the longitude and latitude of the central point of the dome camera image corresponding to each dome camera parameter according to the mounting information of the dome camera to finish the linkage calibration of the gun camera and the dome camera, wherein the mounting information comprises the longitude and latitude and height information.
As a further improvement of the invention, the target detection is realized by a convolutional neural network.
As a further improvement of the present invention, the tracking a target on a video stream of the rifle bolt, obtaining position information of the target in a shooting image of the rifle bolt, and determining a movement direction and a movement speed of the target according to a calibration relationship between the rifle bolt and the ball machine includes:
for each frame of image in the video stream of the gunlock, carrying out target tracking on a detection frame appearing in the current frame of image, and storing position information of the target under the current frame of image;
respectively acquiring corresponding position information of the target under multi-frame images in the video stream of the rifle bolt as historical position information;
and determining the distance and the orientation of the target relative to the dome camera and determining the movement direction and the movement speed of the target according to the historical position information and the calibration relation between the gunlock and the dome camera.
As a further improvement of the present invention, the method further comprises:
controlling the ball machine to rotate according to the calibration relation between the gun camera and the ball machine, so that the ball machine is positioned near the target before snapshot;
and adjusting the focal length of the dome camera to a focal length corresponding to the current target detection frame accounting for two thirds of the image according to the width of the current target detection frame in the image of the dome camera, so that the dome camera starts to capture at the adjusted focal length.
As a further improvement of the present invention, the adjusting, according to the movement speed of the target, the horizontal angle and the pitch angle of the dome camera to locate the target at the center of the image captured by the dome camera, so as to obtain the real-time capturing position of the dome camera includes:
when the movement speed of the target is smaller than a first threshold value, taking the center of a current target detection frame in the dome camera image as the center position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the center position of the dome camera image;
when the movement speed of the target is greater than or equal to a first threshold value and less than a second threshold value, taking the center of a left bottom frame or a right bottom frame of a current target detection frame in the dome camera image as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the central position of the dome camera image;
and when the movement speed of the target is greater than or equal to a second threshold value, predicting the position of the target in the dome camera image by using a least square method, taking the predicted position as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of a target detection frame in the dome camera image to be located at the central position of the dome camera image.
The invention also provides a video capture system for a vessel, the system comprising:
the target detection module is used for carrying out target detection on the video stream of the gunlock to obtain a target detection result, wherein the target detection result comprises position information and confidence information of a target;
the target tracking module is used for tracking a target of a video stream of the gunlock, acquiring position information of the target in a shooting image of the gunlock, and determining the movement direction and the movement speed of the target according to a calibration relation between the gunlock and the ball machine;
the adjusting module is used for adjusting the horizontal angle and the pitch angle of the dome camera according to the movement speed of the target, so that the target is positioned at the center of the image shot by the dome camera to obtain the real-time snapshot position of the dome camera;
and the snapshot module is used for continuously snapshotting the target at the real-time snapshot position by the dome camera at a certain time interval.
As a further improvement of the present invention, the system further comprises:
and the calibration module is used for carrying out linkage calibration on the rifle bolt and the ball machine according to the video streams of the rifle bolt and the ball machine to obtain the calibration relation between the rifle bolt and the ball machine.
As a further improvement of the invention, the calibration module comprises:
acquiring video streams of the rifle bolt and the ball machine;
determining dome camera parameters respectively corresponding to a plurality of gun camera pixel points in the gun camera image according to the video stream of the gun camera and the video stream of the dome camera, wherein the dome camera parameters comprise a horizontal angle and a pitch angle;
performing fitting calibration on the dome camera parameters respectively corresponding to the plurality of rifle bolt pixel points by using a least square method to obtain dome camera parameters corresponding to each rifle bolt pixel point in the rifle bolt image;
and determining the longitude and latitude of the image center point of the dome camera corresponding to each dome camera parameter according to the mounting information of the dome camera so as to finish the linkage calibration of the gun camera and the dome camera, wherein the mounting information comprises the longitude and latitude and the height information.
As a further improvement of the invention, the target detection is realized by a convolutional neural network.
As a further improvement of the present invention, the target tracking module comprises:
for each frame of image in the video stream of the gunlock, carrying out target tracking on a detection frame appearing in the current frame of image, and storing position information of the target under the current frame of image;
respectively acquiring corresponding position information of the target under multi-frame images in the video stream of the rifle bolt as historical position information;
and determining the distance and the direction of the target relative to the dome camera and determining the moving direction and the moving speed of the target according to the historical position information and the calibration relation between the rifle bolt and the dome camera.
As a further improvement of the present invention, the system further comprises:
the snapshot initialization module is used for controlling the dome camera to rotate according to the calibration relation between the gun camera and the dome camera, so that the dome camera is positioned near the target before snapshot; and adjusting the focal length of the dome camera to a focal length corresponding to the current target detection frame accounting for two thirds of the image according to the width of the current target detection frame in the image of the dome camera, so that the dome camera starts to capture at the adjusted focal length.
As a further improvement of the invention, the adjusting module comprises:
when the movement speed of the target is smaller than a first threshold value, taking the center of a current target detection frame in the dome camera image as the center position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the center position of the dome camera image;
when the movement speed of the target is greater than or equal to a first threshold value and smaller than a second threshold value, taking the center of the left bottom frame or the right bottom frame of the current target detection frame in the dome camera image as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the central position of the dome camera image;
and when the movement speed of the target is greater than or equal to a second threshold value, predicting the position of the target in the dome camera image by using a least square method, taking the predicted position as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of a target detection frame in the dome camera image to be located at the central position of the dome camera image.
The invention also provides an electronic device comprising a memory and a processor, the memory storing one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method.
The invention also provides a computer-readable storage medium having stored thereon a computer program for execution by a processor to perform the method.
The invention has the beneficial effects that: the gunlock and the dome camera are linked, and the target is detected and tracked in real time through the gunlock, so that the snapshot position of the dome camera is adjusted in a real-time self-adaptive mode according to different speeds of a ship, an effective snapshot image is obtained, and the snapshot accuracy is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a schematic flowchart of a video capture method for a ship according to an exemplary embodiment of the present invention;
fig. 2 is a schematic flowchart of a video capture method for a ship according to an exemplary embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that, if directional indications (such as up, down, left, right, front, and back … …) are involved in the embodiment of the present invention, the directional indications are only used to explain the relative positional relationship between the components, the movement situation, and the like in a specific posture (as shown in the drawing), and if the specific posture is changed, the directional indications are changed accordingly.
In addition, in the description of the present invention, the terms used are for illustrative purposes only and are not intended to limit the scope of the present invention. The terms "comprises" and/or "comprising" are used to specify the presence of stated elements, steps, operations, and/or components, but do not preclude the presence or addition of one or more other elements, steps, operations, and/or components. The terms "first," "second," and the like may be used to describe various elements, not necessarily order, and not necessarily limit the elements. In addition, in the description of the present invention, "a plurality" means two or more unless otherwise specified. These terms are only used to distinguish one element from another. These and/or other aspects will become apparent to those of ordinary skill in the art in view of the following drawings, and the description of the embodiments of the present invention will be more readily understood by those of ordinary skill in the art. The drawings are only for purposes of illustrating the described embodiments of the invention. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated in the present application may be employed without departing from the principles described in the present application.
As shown in fig. 1, the video capturing method for a ship according to the embodiment of the present invention includes:
carrying out target detection on a video stream of a gunlock to obtain a target detection result, wherein the target detection result comprises position information and confidence information of a target;
carrying out target tracking on the video stream of the gunlock, acquiring position information of the target in the image shot by the gunlock, and determining the movement direction and the movement speed of the target according to the calibration relation between the gunlock and the ball machine;
adjusting the horizontal angle and the pitch angle of the dome camera according to the movement speed of the target, so that the target is positioned at the central position of the image shot by the dome camera to obtain the real-time snapshot position of the dome camera;
and the ball machine continuously shoots the target at the real-time shooting position at a certain time interval.
The existing photoelectric equipment for detecting and snapshotting the intelligent ship is difficult to lock a target due to problems such as photoelectric delay and the like during shooting, so that the best time for snapshotting is missed, and the accuracy of monitoring and evidence obtaining is low. For example, a ship with a high speed has strong maneuverability, a delay is caused from sending a shooting instruction to turning the camera to a specified position, and after the camera reaches the specified position, the ship is not in an image picture, so that the snapshot is invalid. The method provided by the invention utilizes the linkage of the gun camera and the dome camera, detects and tracks the target in real time through the gun camera, adjusts the snapshot position of the dome camera in real time according to the movement speed of the target, obtains an effective snapshot image, improves the snapshot accuracy, and is suitable for different types of ships. It will be appreciated that the target may be a different type of vessel.
In an alternative embodiment, the method further comprises: and carrying out linkage calibration on the rifle bolt and the ball machine according to the video streams of the rifle bolt and the ball machine to obtain the calibration relation between the rifle bolt and the ball machine.
In an optional embodiment, the performing linkage calibration on the gun camera and the dome camera according to the video streams of the gun camera and the dome camera to obtain a calibration relationship between the gun camera and the dome camera includes:
acquiring video streams of the rifle bolt and the ball machine;
determining dome camera parameters respectively corresponding to a plurality of gun camera pixel points in the gun camera image according to the video stream of the gun camera and the video stream of the dome camera, wherein the dome camera parameters comprise a horizontal angle and a pitch angle;
performing fitting calibration on the dome camera parameters respectively corresponding to the plurality of rifle bolt pixel points by using a least square method to obtain dome camera parameters corresponding to each rifle bolt pixel point in the rifle bolt image;
and determining the longitude and latitude of the image center point of the dome camera corresponding to each dome camera parameter according to the mounting information of the dome camera so as to finish the linkage calibration of the gun camera and the dome camera, wherein the mounting information comprises the longitude and latitude and the height information.
The method provided by the invention has the advantages that the gunlock and the ball machine are linked to calibrate the two so as to be associated, and then the ball machine can rotate to the position of the current target tracked by the gunlock to capture the current target. The method can acquire coordinates of 15 gunlock pixel points in the gunlock image, and further acquire ball machine parameters (including a horizontal angle value and a pitch angle value) corresponding to the 15 gunlock pixel points. And fitting by a least square method, and further obtaining the dome camera parameters corresponding to each gunlock pixel point (namely any one gunlock pixel point) in the images of the gunlocks. And then according to the dome camera installation information (including longitude and latitude and height information), the longitude and latitude corresponding to each dome camera parameter can be obtained, and each dome camera parameter corresponds to one dome camera image central point, so that the longitude and latitude of the dome camera image central point corresponding to each dome camera parameter can be obtained, namely the calibration relation between the gun camera and the dome camera is obtained, and the calibration relation is used for adjusting the horizontal angle and the pitch angle of the dome camera in the follow-up process.
In an alternative embodiment, the target detection is performed by a convolutional neural network.
It can be understood that the method of the present invention utilizes a convolutional neural network to perform target detection on the video stream of the bolt, so as to achieve the preliminary positioning of the target. For example, a YOLO model may be used to perform target detection on each frame of image of the bolt, and return position information and confidence information of the target in each frame of image. The YOLO model uses a regression method, and uses the whole image as the input of the convolutional neural network, and directly regresses the target frame at a plurality of positions of the image and the category to which the target belongs. The model of the convolutional neural network is not particularly limited.
In an optional embodiment, the tracking a target on a video stream of the gun camera, obtaining position information of the target in a shot image of the gun camera, and determining a moving direction and a moving speed of the target according to a calibration relationship between the gun camera and the ball machine includes:
for each frame of image in the video stream of the gunlock, carrying out target tracking on a detection frame appearing in the current frame of image, and storing position information of the target under the current frame of image;
respectively acquiring corresponding position information of the target under multi-frame images in a video stream of the bolt as historical position information;
and determining the distance and the direction of the target relative to the dome camera and determining the moving direction and the moving speed of the target according to the historical position information and the calibration relation between the rifle bolt and the dome camera.
It is understood that, when the target is continuously tracked by the bolt face, for example, 10 frames of position information can be accumulated as the historical position information. The historical position information can extract the movement information of the gunlock pixel points, and the speed of the target movement can be calculated according to the movement information of the gunlock pixel points. And according to the calibration relation between the gunlock and the dome camera (the longitude and latitude of the center point of the dome camera image corresponding to each dome camera parameter), the distance and the direction of the target relative to the dome camera can be calculated.
In an alternative embodiment, the method further comprises:
controlling the ball machine to rotate according to the calibration relation between the gun camera and the ball machine, so that the ball machine is positioned near the target before snapshot;
and adjusting the focal length of the dome camera to a focal length corresponding to the current target detection frame accounting for two thirds of the image according to the width of the current target detection frame in the image of the dome camera, so that the dome camera starts to capture at the adjusted focal length.
In the moving process of the target, if the moving speed is high, the target can not be captured in time easily. Therefore, the position of the ball machine can be adjusted before the snapshot is started, so that the position of the image currently captured by the ball machine is always near the position to be snapshot. The ball yielding machine follows the target in advance, so that the delay caused by large-amplitude rotation of the camera can be reduced. Correspondingly, in the position adjusting process, the focal length of the dome camera is adaptively adjusted, so that clear images can be captured.
In an optional embodiment, the adjusting the horizontal angle and the pitch angle of the dome camera according to the movement speed of the target to locate the target at the center of the shot image of the dome camera to obtain the real-time capturing position of the dome camera includes:
when the movement speed of the target is smaller than a first threshold value, taking the center of a current target detection frame in the dome camera image as the center position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the center position of the dome camera image;
when the movement speed of the target is greater than or equal to a first threshold value and smaller than a second threshold value, taking the center of the left bottom frame or the right bottom frame of the current target detection frame in the dome camera image as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the central position of the dome camera image;
and when the movement speed of the target is greater than or equal to a second threshold value, predicting the position of the target in the dome camera image by using a least square method, taking the predicted position as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of a target detection frame in the dome camera image to be located at the central position of the dome camera image.
The method adjusts the angle of the dome camera in real time according to the movement speed of the target, so that the target can be positioned at the center of the image shot by the dome camera during snapshot, clear and accurate snapshot of ships with different speeds is realized, the snapshot effectiveness and accuracy are improved, and high-quality snapshot pictures are provided for next ship type identification and ship number plate identification.
For example, when the moving speed is less than 10 knots, it can be understood that the moving speed of the target is very slow, and the target detection frame in the current picture of the dome camera can be directly utilized to perform snapshot, and 5 images can be continuously snapshot at intervals of 1 second, for example. For example, when the moving speed is greater than 10 knots and less than 20 knots, it can be understood that the target moving speed is slow, the left or right bottom frame center of the target detection frame in the current picture of the dome camera is taken as the center position of the image of the dome camera, and the horizontal angle and the pitch angle of the dome camera are adjusted so that the center of the target detection frame in the picture of the dome camera is located at the center position, at which the dome camera continuously captures 5 images at intervals of, for example, 500 ms. When the target enters from the left side of the picture of the dome camera and moves towards the right side, the position of the bottom frame on the right side of the target detection frame can be used as the central position of the picture of the dome camera, and when the target enters from the right side of the picture of the dome camera and moves towards the left side, the position of the bottom frame on the left side of the target detection frame can be used as the central position of the picture of the dome camera. For example, when the target movement speed is greater than 20 knots, it can be understood that the target movement speed is high, the position of the target in the dome camera image is predicted by using the least square method (the data used in the prediction may be, for example, the position of the target in the first several frames of dome camera images), the predicted position is taken as the center position of the dome camera image, the horizontal angle and the pitch angle of the dome camera are adjusted so that the center of the target detection frame in the dome camera image is located at the center position, and the dome camera continuously captures 5 images at intervals of, for example, 300 ms. The speed threshold and the time interval are illustrated schematically, and the values of the speed threshold and the time interval are not particularly limited, and can be adaptively designed according to the use requirement.
For example, as shown in fig. 2, the method of the present invention may implement a snapshot of a ship by using the following process.
S1, acquiring real-time video streams including the video streams of the gunlock and the dome camera, acquiring 15 gunlock pixel points and dome camera parameters (values of a horizontal angle and a pitch angle) corresponding to the dome camera, performing gun and ball linkage calibration by using a least square method, and acquiring dome camera parameters corresponding to each gunlock pixel point;
s2, acquiring dome camera installation information including longitude, latitude and height, and calculating the longitude and latitude of dome image center pixels under each dome camera parameter (horizontal angle and pitch angle);
s3, performing target detection on the gunlock video stream frame by using a YOLO model, and returning the information of all ships in the current frame, including the information of pixel positions, confidence coefficient and the like in the picture;
s4, tracking the target of the detection frame appearing in the current frame, storing historical position information, calculating the distance and the orientation between the target and the dome camera according to the calibration relation between the gunlock and the dome camera after the historical position information of the target is accumulated to 10 frames, further calculating the moving direction and the moving speed of the target relative to the dome camera, and repeating S5 if the number of accumulated target tracking frames is less than 10 frames;
s5, before snapshot is started, rotating the position of the dome camera according to the position of the dome camera calibrated by linkage of the gun camera and the dome camera, keeping the current picture position of the dome camera near the position to be snapshot, and adjusting the focal length of the dome camera to calculate the corresponding focal length when the detection frame occupies two thirds of the picture of the image according to the width of the current detection frame;
s6, judging a target snapshot method according to the movement speed of the target, adjusting the horizontal angle and the pitch angle of the dome camera when the speed is less than 10 knots, enabling the center of the current target detection frame to be the center position of the image of the dome camera, starting to continuously snapshot 5 images at intervals of 1 second, and entering S7 when the speed is more than 10 knots and less than 20 knots;
s7, adjusting the horizontal angle and the pitch angle of the dome camera according to the moving direction of the target, enabling the center of the bottom frame on the left side or the right side of the current target detection frame to be the center position of the dome camera image, starting to continuously shoot 5 pieces at an interval of 500 milliseconds, and entering S8 when the speed is higher than 20 knots;
and S8, judging whether the speed is more than 30 knots or not again, predicting the position of the target in the image of the dome camera by using a least square method when the speed is more than 30 knots, adjusting the horizontal angle and the pitch angle of the dome camera to enable the predicted position to be the central position of the image of the dome camera, and starting to continuously take 5 shots at an interval of 300 milliseconds.
The embodiment of the invention provides a video snapshot system for a ship, which comprises:
the target detection module is used for carrying out target detection on the video stream of the gunlock to obtain a target detection result, wherein the target detection result comprises position information and confidence information of a target;
the target tracking module is used for tracking a target of a video stream of the gunlock, acquiring position information of the target in a shooting image of the gunlock, and determining the movement direction and the movement speed of the target according to a calibration relation between the gunlock and the ball machine;
the adjusting module is used for adjusting the horizontal angle and the pitch angle of the dome camera according to the movement speed of the target, so that the target is positioned at the center of the image shot by the dome camera to obtain the real-time snapshot position of the dome camera;
and the snapshot module is used for continuously snapshotting the target at the real-time snapshot position by the dome camera at a certain time interval.
In an alternative embodiment, the system further comprises:
and the calibration module is used for carrying out linkage calibration on the rifle bolt and the ball machine according to the video streams of the rifle bolt and the ball machine to obtain the calibration relation between the rifle bolt and the ball machine.
In an alternative embodiment, the calibration module includes:
acquiring video streams of the rifle bolt and the ball machine;
determining dome camera parameters respectively corresponding to a plurality of gun camera pixel points in the gun camera image according to the video stream of the gun camera and the video stream of the dome camera, wherein the dome camera parameters comprise a horizontal angle and a pitch angle;
performing fitting calibration on the dome camera parameters respectively corresponding to the plurality of rifle bolt pixel points by using a least square method to obtain dome camera parameters corresponding to each rifle bolt pixel point in the rifle bolt image;
and determining the longitude and latitude of the image center point of the dome camera corresponding to each dome camera parameter according to the mounting information of the dome camera so as to finish the linkage calibration of the gun camera and the dome camera, wherein the mounting information comprises the longitude and latitude and the height information.
In an alternative embodiment, the target detection is performed by a convolutional neural network.
In an alternative embodiment, the target tracking module comprises:
for each frame of image in the video stream of the gunlock, performing target tracking on a detection frame appearing in a current frame of image, and storing position information of the target under the current frame of image;
respectively acquiring corresponding position information of the target under multi-frame images in the video stream of the rifle bolt as historical position information;
and determining the distance and the direction of the target relative to the dome camera and determining the moving direction and the moving speed of the target according to the historical position information and the calibration relation between the rifle bolt and the dome camera.
In an alternative embodiment, the system further comprises:
the snapshot initialization module is used for controlling the dome camera to rotate according to the calibration relation between the gun camera and the dome camera, so that the dome camera is positioned near the target before snapshot; and adjusting the focal length of the dome camera to a focal length corresponding to the current target detection frame accounting for two thirds of the image according to the width of the current target detection frame in the image of the dome camera, so that the dome camera starts to capture at the adjusted focal length.
In an alternative embodiment, the adjustment module comprises:
when the movement speed of the target is smaller than a first threshold value, taking the center of a current target detection frame in the dome camera image as the center position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the center position of the dome camera image;
when the movement speed of the target is greater than or equal to a first threshold value and smaller than a second threshold value, taking the center of the left bottom frame or the right bottom frame of the current target detection frame in the dome camera image as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the central position of the dome camera image;
and when the movement speed of the target is greater than or equal to a second threshold value, predicting the position of the target in the dome camera image by using a least square method, taking the predicted position as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of a target detection frame in the dome camera image to be located at the central position of the dome camera image.
The disclosure also relates to an electronic device comprising a server, a terminal and the like. The electronic device includes: at least one processor; a memory communicatively coupled to the at least one processor; and a communication component communicatively coupled to the storage medium, the communication component receiving and transmitting data under control of the processor; wherein the memory stores instructions executable by the at least one processor to implement the method of the above embodiments.
In an alternative embodiment, the memory is used as a non-volatile computer-readable storage medium for storing non-volatile software programs, non-volatile computer-executable programs, and modules. The processor executes various functional applications of the device and data processing, i.e., implements the method, by executing nonvolatile software programs, instructions, and modules stored in the memory.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store a list of options, etc. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and such remote memory may be connected to the external device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
One or more modules are stored in the memory and, when executed by the one or more processors, perform the methods of any of the method embodiments described above.
The product can execute the method provided by the embodiment of the application, has corresponding functional modules and beneficial effects of the execution method, and can refer to the method provided by the embodiment of the application without detailed technical details in the embodiment.
The present disclosure also relates to a computer-readable storage medium for storing a computer-readable program for causing a computer to perform some or all of the above-described method embodiments.
That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Furthermore, those of ordinary skill in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
It will be understood by those skilled in the art that while the present invention has been described with reference to exemplary embodiments, various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed, but that the invention will include all embodiments falling within the scope of the appended claims.
Claims (9)
1. A video capture method for a marine vessel, the method comprising:
carrying out target detection on a video stream of a gunlock to obtain a target detection result, wherein the target detection result comprises position information and confidence information of a target;
carrying out target tracking on the video stream of the rifle bolt to obtain position information of the target in the image shot by the rifle bolt so as to determine the movement information of pixel points of the rifle bolt, determining the movement speed of the target according to the movement information of the pixel points of the rifle bolt, and determining the distance and the direction of the target relative to the ball machine according to the calibration relation between the rifle bolt and the ball machine;
according to the movement speed of the target, adjusting the horizontal angle and the pitch angle of the dome camera to enable the target to be located at the central position of the image shot by the dome camera so as to obtain the real-time snapshot position of the dome camera, and the method comprises the following steps:
when the movement speed of the target is smaller than a first threshold value, taking the center of a current target detection frame in the dome camera image as the center position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the center position of the dome camera image;
when the movement speed of the target is greater than or equal to a first threshold value and smaller than a second threshold value, taking the center of the left bottom frame or the right bottom frame of the current target detection frame in the dome camera image as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the central position of the dome camera image;
when the movement speed of the target is greater than or equal to a second threshold value, predicting the position of the target in the dome camera image by using a least square method, taking the predicted position as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of a target detection frame in the dome camera image to be located at the central position of the dome camera image;
and the ball machine continuously shoots the target at the real-time shooting position at a certain time interval.
2. The method of claim 1, wherein the method further comprises: and carrying out linkage calibration on the rifle bolt and the ball machine according to the video streams of the rifle bolt and the ball machine to obtain the calibration relation between the rifle bolt and the ball machine.
3. The method of claim 2, wherein the calibrating the rifle bolt and the ball machine in a linkage manner according to the video streams of the rifle bolt and the ball machine to obtain the calibration relationship between the rifle bolt and the ball machine comprises:
acquiring video streams of the rifle bolt and the ball machine;
determining dome camera parameters respectively corresponding to a plurality of gun camera pixel points in the gun camera image according to the video stream of the gun camera and the video stream of the dome camera, wherein the dome camera parameters comprise a horizontal angle and a pitch angle;
performing fitting calibration on the dome camera parameters respectively corresponding to the plurality of rifle bolt pixel points by using a least square method to obtain dome camera parameters corresponding to each rifle bolt pixel point in the rifle bolt image;
and determining the longitude and latitude of the image center point of the dome camera corresponding to each dome camera parameter according to the mounting information of the dome camera so as to finish the linkage calibration of the gun camera and the dome camera, wherein the mounting information comprises the longitude and latitude and the height information.
4. The method of claim 1, wherein the target detection is achieved by a convolutional neural network.
5. The method of claim 1, wherein the tracking the target in the video stream of the gun camera, obtaining the position information of the target in the image taken by the gun camera, and determining the moving direction and the moving speed of the target according to the calibration relationship between the gun camera and the ball machine comprises:
for each frame of image in the video stream of the gunlock, carrying out target tracking on a detection frame appearing in the current frame of image, and storing position information of the target under the current frame of image;
respectively acquiring corresponding position information of the target under multi-frame images in the video stream of the rifle bolt as historical position information;
and determining the distance and the direction of the target relative to the dome camera and determining the moving direction and the moving speed of the target according to the historical position information and the calibration relation between the rifle bolt and the dome camera.
6. The method of claim 1, wherein the method further comprises:
controlling the ball machine to rotate according to the calibration relation between the gun camera and the ball machine, so that the ball machine is positioned near the target before snapshot;
and adjusting the focal length of the dome camera to a focal length corresponding to the current target detection frame accounting for two thirds of the image according to the width of the current target detection frame in the image of the dome camera, so that the dome camera starts to capture at the adjusted focal length.
7. A video capture system for a marine vessel, the system comprising:
the target detection module is used for carrying out target detection on the video stream of the gunlock to obtain a target detection result, wherein the target detection result comprises position information and confidence information of a target;
the target tracking module is used for tracking the target of the video stream of the gunlock, acquiring the position information of the target in the image shot by the gunlock so as to determine the movement information of pixel points of the gunlock, determining the movement speed of the target according to the movement information of the pixel points of the gunlock, and determining the distance and the direction of the target relative to the dome camera according to the calibration relation between the gunlock and the dome camera;
the adjusting module is used for adjusting the horizontal angle and the pitch angle of the dome camera according to the movement speed of the target, so that the target is located at the central position of the image shot by the dome camera to obtain the real-time snapshot position of the dome camera, and comprises the following steps:
when the movement speed of the target is smaller than a first threshold value, taking the center of a current target detection frame in the dome camera image as the center position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the center position of the dome camera image;
when the movement speed of the target is greater than or equal to a first threshold value and smaller than a second threshold value, taking the center of the left bottom frame or the right bottom frame of the current target detection frame in the dome camera image as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of the target detection frame in the dome camera image to be located at the central position of the dome camera image;
when the movement speed of the target is greater than or equal to a second threshold value, predicting the position of the target in the dome camera image by using a least square method, taking the predicted position as the central position of the dome camera image, and adjusting the horizontal angle and the pitch angle of the dome camera so as to enable the center of a target detection frame in the dome camera image to be located at the central position of the dome camera image;
and the snapshot module is used for continuously snapshotting the target at the real-time snapshot position by the dome camera at a certain time interval.
8. An electronic device comprising a memory and a processor, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any of claims 1-6.
9. A computer-readable storage medium, on which a computer program is stored, the computer program being executable by a processor for implementing the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110156215.2A CN112954274B (en) | 2021-02-04 | 2021-02-04 | Video snapshot method and system for ship |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110156215.2A CN112954274B (en) | 2021-02-04 | 2021-02-04 | Video snapshot method and system for ship |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112954274A CN112954274A (en) | 2021-06-11 |
| CN112954274B true CN112954274B (en) | 2022-09-20 |
Family
ID=76244034
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110156215.2A Active CN112954274B (en) | 2021-02-04 | 2021-02-04 | Video snapshot method and system for ship |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112954274B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115527176B (en) * | 2022-11-24 | 2023-04-07 | 三亚海兰寰宇海洋信息科技有限公司 | Target object identification method, device and equipment |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8026945B2 (en) * | 2005-07-22 | 2011-09-27 | Cernium Corporation | Directed attention digital video recordation |
| CN102883144B (en) * | 2012-09-26 | 2015-04-22 | 青岛海信网络科技股份有限公司 | Video monitoring system and video monitoring method |
| CN104184932B (en) * | 2013-05-20 | 2018-03-23 | 浙江大华技术股份有限公司 | Ball machine control method and device |
| CN104853104B (en) * | 2015-06-01 | 2018-08-28 | 深圳市微队信息技术有限公司 | A kind of method and system of auto-tracking shooting moving target |
| CN105430326A (en) * | 2015-11-03 | 2016-03-23 | 中国电子科技集团公司第二十八研究所 | A Smooth Tracking Method for CCTV Ship Video |
| CN109151375B (en) * | 2017-06-16 | 2020-07-24 | 杭州海康威视数字技术股份有限公司 | Target object snapshot method and device and video monitoring equipment |
| CN109922250B (en) * | 2017-12-12 | 2021-04-02 | 杭州海康威视数字技术股份有限公司 | Target object snapshot method and device and video monitoring equipment |
| CN110113560B (en) * | 2018-02-01 | 2021-06-04 | 中兴飞流信息科技有限公司 | Intelligent video linkage method and server |
| CN108495085A (en) * | 2018-03-14 | 2018-09-04 | 成都新舟锐视科技有限公司 | A kind of ball machine automatic tracking control method and system based on moving target detection |
| CN111126235B (en) * | 2019-12-18 | 2023-06-16 | 浙江大华技术股份有限公司 | Detection processing method and device for illegal berthing of ship |
| CN111246172A (en) * | 2020-01-20 | 2020-06-05 | 深圳英飞拓科技股份有限公司 | Linkage image acquisition device and image acquisition method |
| CN111738211B (en) * | 2020-07-17 | 2023-12-19 | 浙江大学 | PTZ camera moving object detection and recognition method based on dynamic background compensation and deep learning |
-
2021
- 2021-02-04 CN CN202110156215.2A patent/CN112954274B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112954274A (en) | 2021-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110163885B (en) | Target tracking method and device | |
| CN110969097B (en) | Method, equipment and storage device for controlling linkage tracking of monitoring target | |
| US9565348B2 (en) | Automatic tracking apparatus | |
| CN109151375B (en) | Target object snapshot method and device and video monitoring equipment | |
| CN109922250B (en) | Target object snapshot method and device and video monitoring equipment | |
| CN112954315B (en) | Image focusing measurement method and system for security camera | |
| US20200167959A1 (en) | Method and device for capturing target object and video monitoring device | |
| US11892812B1 (en) | Gimbal adjustment method, gimbal adjustment device, mobile platform and storage medium | |
| US20190320118A1 (en) | Anti-Shake Method for Camera and Camera | |
| EP3319042B1 (en) | Target tracking device and target tracking method | |
| CN110991550B (en) | Video monitoring method and device, electronic equipment and storage medium | |
| CN108090893B (en) | Optimization method and device for pesticide spraying of plant protection drones based on thermal infrared recognition | |
| CN112104842B (en) | Image acquisition equipment correction method, device, equipment and medium | |
| CN110570454A (en) | Method and device for detecting foreign matter invasion | |
| CN112312068B (en) | Target detection method, device, equipment and storage medium | |
| CN109905641B (en) | Target monitoring method, device, equipment and system | |
| CN105979143B (en) | Method and device for adjusting shooting parameters of dome camera | |
| CN112954274B (en) | Video snapshot method and system for ship | |
| CN103310442A (en) | Multi-frequency information fusion-based intelligent positioning system and method | |
| CN112911249A (en) | Target object tracking method and device, storage medium and electronic device | |
| CN115442521B (en) | Image pickup control method and related device, camera and storage medium | |
| CN114463385A (en) | Target tracking method, device, equipment and medium based on gun-ball linkage system | |
| CN112305534A (en) | Target detection method, device, equipment and storage medium | |
| CN118283419A (en) | Lens control method, device, object sensing system and storage medium | |
| CN115174861B (en) | Method and device for automatically tracking moving target by holder camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: No. 1156, F1, Building 2, Building A, Qingshuiwan International Information Industry Park, No. 1 Lehuo Avenue, Yingzhou Town, Lingshui Li Autonomous County, Hainan Province, 572427 Patentee after: Hainan Hailan Huanyu Ocean Information Technology Co.,Ltd. Country or region after: China Address before: 572025 rooms 425 and 426, 4th floor, building 4, Baitai Industrial Park, yazhouwan science and Technology City, Yazhou District, Sanya City, Hainan Province Patentee before: Sanya Hai Lan world marine Mdt InfoTech Ltd. Country or region before: China |