CN118646857A - A method for measuring the response delay of a pan-tilt camera in a video monitoring system - Google Patents

Abstract

The present invention discloses a method for measuring the response delay of a pan-tilt camera of a video monitoring system, comprising the following steps: obtaining test point information, determining a video area to be detected based on the test point information; issuing a control instruction to the pan-tilt based on a mouse click operation, capturing the moment of the mouse click operation as the start moment; detecting the video changes in the video area to be detected and performing contour calculation based on feature matching and background subtraction algorithms to obtain the video change contour area; presetting a threshold, and the first moment when the proportion of the video change contour area meets the threshold is the end moment; based on the time difference between the end moment and the start moment, obtaining the response delay of the pan-tilt camera. The measurement method proposed in the present invention not only simplifies the test process, but also improves the operability and practicality of the test, and provides a convenient and reliable delay measurement method for evaluating the control response performance of the pan-tilt camera of the video monitoring system.

Description

A method for measuring the response delay of a pan-tilt camera in a video monitoring system

Technical Field

The invention belongs to the technical field of electronic equipment, and in particular relates to a method for measuring the response delay of a pan/tilt camera of a video monitoring system.

Background Art

With the rapid development of technology, video surveillance systems are becoming more and more widely used in security, monitoring and other fields. These systems are usually composed of cameras, image processing units and central servers, which are used to capture, process and store images and video data of monitoring scenes. Among them, PTZ cameras, as one of the key components of video surveillance systems, have the functions of remote control and adjustment of viewing angles, providing great flexibility for monitoring personnel.

The application of PTZ cameras covers many fields, including security, traffic supervision, industrial monitoring, and smart city construction. Its remote controllability and flexibility make PTZ cameras an ideal choice for handling various monitoring scenarios, providing technical support for effective monitoring and management.

However, with the increasing complexity of monitoring scenes, the system response delay involved in remotely controlling PTZ cameras has become an increasingly prominent technical challenge. This delay is affected by many factors, including but not limited to network latency, system processing speed, and communication protocol. The existence of this response delay directly affects the monitoring personnel's ability to respond quickly to real-time scenes, which may affect the real-time and responsiveness of the monitoring system.

Although there are some methods for measuring system response delay, such as manually using a stopwatch to record the delay, the existing technology may have certain limitations in the special situation of PTZ cameras in video surveillance systems. Therefore, it is urgent to propose a new method to more accurately measure the response delay of PTZ cameras and solve the problems faced by current technologies in a targeted manner.

Summary of the invention

In order to solve the above technical problems, the present invention proposes a method for measuring the response delay of a pan-tilt camera in a video monitoring system, aiming to use computer vision to effectively measure and optimize the response delay of the pan-tilt camera, improve the efficiency of remote control of the pan-tilt camera, so as to better meet the growing monitoring needs and solve the problems existing in the above-mentioned prior art.

To achieve the above object, the present invention provides a method for measuring the response delay of a pan/tilt camera of a video monitoring system, comprising the following steps:

Acquire test point information, and determine a video area to be detected based on the test point information;

Issue control instructions to the PTZ based on the mouse click operation, and the moment of capturing the mouse click operation is the start moment;

Based on feature matching and background subtraction algorithms, detecting video changes in the video area to be detected and performing contour calculation to obtain a video change contour area;

A preset threshold value is used, and the first moment when the proportion of the contour area of the video change meets the threshold value is the end moment;

Based on the time difference between the end time and the start time, the response delay of the pan-tilt camera is obtained.

Optionally, the process of determining the start time includes: when the test starts, the video monitoring system monitors the mouse click operation, and when the pan-tilt camera control based on the mouse click operation is monitored, the current video monitoring system timestamp is recorded, and the time when the pan-tilt camera control instruction is issued is determined as the start time.

Optionally, the process of determining the end time includes: in the video area to be detected, based on the feature matching algorithm, performing feature extraction on the two frames of images before and after the movement of the pan-tilt camera, calculating the image space coordinate transformation parameters based on the extracted feature parameters, and eliminating the interference of the pan-tilt camera jitter based on the transformation parameters; then using the background subtraction algorithm based on the OpenCV library to separate the foreground and background of the video frame in the pan-tilt video surveillance screen, obtain the object contour of the foreground image, calculate the video change contour area based on the object contour of the foreground image, preset a threshold, and the first moment when the video change contour area meets the threshold is the end time.

Optionally, the process of eliminating the interference of the PTZ camera jitter includes:

The feature matching method based on ORB key points is used to describe the features of the image. The pixels that are significantly different from the surrounding pixels are found as key points in the image, and the descriptors of each key point are calculated.

Perform feature matching on the image by using the Hamming distance between each descriptor in the descriptor set and the query descriptor;

Sort all Hamming distances from small to large, and select the first 90% of the descriptors in the sorted sequence as the matching results;

Calculate the homography matrix to determine the horizontal and vertical movement distances of the image, and eliminate the interference of the gimbal camera shake through the parallax method.

Optionally, the process of obtaining the moving distance based on the homography matrix includes: obtaining the homography matrix between the two images based on the position representation of the same point in two different images; solving the homography matrix based on the least squares method to obtain the mapping relationship between the two images, and obtaining the moving distance of the horizontal and vertical images based on the mapping relationship.

Optionally, before obtaining the video change contour area, the method further includes: converting the foreground image into a grayscale image, and performing Gaussian blurring and binarization preprocessing.

Optionally, the process of obtaining the video change contour area includes: depicting the object contour of the preprocessed foreground image based on the finfContours() function and the drawContours() function of the OpenCV library, and then calculating the video change contour area based on the object contour through the contourArea() function.

Optionally, the test point information includes a test point name and a test data number.

Compared with the prior art, the present invention has the following advantages and technical effects:

Compared with the traditional manual method, the present invention accurately obtains the moment when the tester controls the pan-tilt camera on the video monitoring system and the moment when the pan-tilt camera successfully responds, that is, the start time and the end time, and obtains the control response delay of the pan-tilt camera through the time difference between the start time and the end time. It not only simplifies the test process, but also improves the operability and practicality of the test, and provides a convenient and reliable delay measurement method for the control response performance evaluation of the pan-tilt camera of the video monitoring system.

The present invention obtains the start time by capturing the mouse click operation of the tester, and obtains the end time by the change of the image video frame, thereby improving the accuracy of delay measurement and achieving more efficient and accurate measurement of delay.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

FIG1 is a schematic flow chart of a method for measuring a pan-tilt camera control response delay according to an embodiment of the present invention;

FIG2 is a schematic diagram of a process for eliminating interference of pan/tilt camera jitter on video picture change detection according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a process of detecting image changes in a PTZ video monitoring screen according to an embodiment of the present invention.

DETAILED DESCRIPTION

It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

Embodiment 1

In this embodiment, a method for measuring the response delay of the pan-tilt camera of a video monitoring system is provided, and the control response delay of the pan-tilt camera is obtained by accurately obtaining the difference between the moment when the tester controls the pan-tilt camera on the video monitoring system and the moment when the pan-tilt camera successfully responds. The operation process includes the time from when the tester clicks the pan-tilt camera operation button with the mouse to when the pan-tilt video monitoring area screen changes in the video monitoring system screen meet the threshold requirements. First, the control event of the pan-tilt camera by the tester through the mouse click in the video monitoring system is captured, and the start time _ts of the pan-tilt camera control operation is determined. Then, based on the feature matching technology, the video area S to be inspected is detected, and the end time te of the last frame of the pan-tilt monitoring screen reaching the stable screen is determined from the moving video frame screen, _and the time difference between the end time and the start time is determined as the delay of the video monitoring system calling the real-time video.

Specifically, FIG1 is a flow chart of a method for measuring a pan-tilt camera control response delay provided in this embodiment, comprising the following steps:

S101, input the test point information, and select the screen display area of the PTZ video monitoring in the video monitoring system, that is, determine the video area S to be tested.

During specific implementation, the test point information includes the test point name and the number of test data. The tester needs to determine and enter the basic information of the point for this response delay test according to the specific PTZ camera controlled and the test requirements.

S102, after clicking to start the test, monitor the tester's mouse click operation to issue a control command to the PTZ, and capture the moment of the mouse click operation as the start moment _ts .

In specific implementation, the control instruction issued by the tester to the pan-tilt camera is the tester's mouse clicking the button operation of the pan-tilt camera rotation in the video surveillance system. This embodiment uses pynput.mouse to monitor the tester's mouse click event. Once the tester starts the test, the system starts to monitor the mouse click event. After the event of the mouse click operation to control the pan-tilt camera is detected, the current system timestamp is recorded to determine the start time _ts when the tester issues the control instruction for the pan-tilt camera rotation.

S103, based on feature matching and background subtraction algorithms, detect the image changes of the video area to be tested S, determine the moment when the pan-tilt camera of the video area to be tested completes the instruction and successfully rotates the camera as the end time t _e . The pan-tilt camera starts to respond to the control instruction, and the moment when the pan-tilt camera successfully responds to the operation instruction of the tester, and the first moment when the image change area ratio of the pan-tilt video monitoring screen viewed by the tester meets the threshold is the end time t _e . The first moment when the image change area ratio of the pan-tilt video monitoring screen viewed by the tester meets the threshold means that the difference ratio of the video change contour area of the two adjacent foreground frames is greater than 25%.

Specifically, when the tester issues a control command to the PTZ, the system will take screenshots and analyze the tester's video area S to be tested. First, the feature matching method is used to extract features from the two frames of images before and after the PTZ camera moves. The image space coordinate transformation parameters are calculated based on the extracted feature parameters, and then the threshold is set based on the parameters to eliminate the interference of camera shaking.

At the same time, the background subtraction method is used to separate the foreground and background of the video frame in the PTZ video surveillance screen to reduce the interference of factors such as lighting and shadows. Then, the contour area of the object in the extracted foreground image is calculated, and according to the proportion of the foreground image contour area, it is determined whether the PTZ camera has rotated. The first moment when the proportion of the foreground image video change contour area meets the threshold requirement is determined as the end time t _e .

S104: Determine the time difference between the end time _te and the start time _ts as the pan/tilt camera control response delay Δt= _te - _ts in the video monitoring system.

FIG. 2 is a schematic diagram of a process for eliminating interference of pan/tilt camera jitter on video picture change detection provided by an embodiment of the present invention.

The pan-tilt camera response judgment is realized by using the method based on the image movement distance detection and the object contour area in the dynamic object foreground picture, which specifically includes the following steps:

The feature matching method based on ORB key points is used to extract and compare the features of the two frames of images before and after the pan-tilt camera moves. Specifically, the feature parameters of the image are first extracted, then the image space coordinate transformation parameters are calculated, and the threshold is set to eliminate the interference of camera shaking.

S201. When performing image feature point matching, the image key points are found by extracting image features from the key points detected in the image. In this embodiment, ORB feature points are selected for image feature description. The ORB feature points are uniquely determined by finding pixels in the image that are significantly different from the surrounding pixels and calculating the BRIEF descriptor of each key point.

S202, then use the BF matching method to perform preliminary feature matching. The algorithm calculates the Hamming distance between each descriptor in the training descriptor set and the query descriptor;

S203 , sorting all Hamming distances in ascending order, and selecting the first 90% of the descriptors in the sorted sequence as matching results.

S204, calculating the conversion matrix, determining the horizontal and vertical movement distances of the image, and eliminating the influence of camera shaking on the test results by using a parallax method.

It can be implemented that the Hamming distance is the number of steps required to transform a set of binary data into another set of data. Obviously, this value can measure the difference between two pictures. The smaller the Hamming distance, the higher the similarity.

The transformation matrix is implemented using a homography matrix, which is an important transformation matrix in computer vision, used to describe the projection relationship of the same object observed by two cameras at different perspectives or different postures on a plane. In practical applications, homography matrices are widely used in image alignment, image registration, three-dimensional reconstruction, virtual reality and other fields.

The homography matrix is solved as follows:

Suppose a point p ₁ =(x ₁ ,y ₁ ,1) ^T in the first image, and its corresponding point p ₂ =(x ₂ ,y ₂ ,1) ^T in the second image. The homography transformation relationship between the two images can be expressed as: p ₂ ~Hp ₁ .

Where p ₁ and p ₂ are the position representations of the same point in two different images; H is the homography matrix; “～” indicates that the two vectors are of equal proportion.

Since _p2 and _p1 are both homogeneous coordinates, H is a 3×3 homogeneous matrix, that is:

Substituting the coordinates of _p2 and _p1 , we get:

This embodiment uses a homogeneous coordinate system for calculation, that is, multiplying h _ij by any non-zero constant k does not change the result of the equation, so the homography matrix H has only 8 degrees of freedom. In the following calculation, this embodiment sets h33=1:

By solving the homography matrix H using the least squares method, the mapping relationship between the two images can be obtained. Finally, the distances of horizontal and vertical image movement are calculated based on the homography matrix H.

FIG3 is a schematic diagram of a process for detecting image changes in a PTZ video monitoring screen provided by this embodiment.

The method of identifying moving objects and calculating contour areas based on the OpenCV library is used to roughly determine the function of the camera's successful response, which specifically includes the following steps:

S301. Using OpenCV to identify moving objects is to separate the dynamic foreground from the static background. When the current image is compared with the assumed static background and an area with obvious changes is found, it can be considered that a moving object appears in the area.

However, in actual situations, due to the large interference of factors such as lighting and shadows, it is easy to cause false detection by directly comparing pixels. Therefore, this embodiment uses a background subtraction algorithm based on the OpenCV library to separate the foreground and background of the PTZ video monitoring screen image, so as to distinguish the changes caused by moving objects and other factors.

S302, before calculating the contour area, first convert the image into a grayscale image, and perform Gaussian blur processing and binarization to obtain a clear binary image to reduce background interference;

S303, using the finfContours() function and drawContours() function of the OpenCV library to draw the contours of the objects in the foreground image, and finally calculating the contour area C of the video change through the contourArea() function.

If the difference in contour area between the two adjacent foreground video frames exceeds 25%, the PTZ camera is considered to have responded successfully.

According to the requirements of the "Technical Specifications for Railway Integrated Video Surveillance Systems", the response delay of the PTZ camera control should not exceed 500 milliseconds. In the experiment, this embodiment sets the waiting time after the test to 500 milliseconds. If the video surveillance area does not change during this period, it is considered that the PTZ camera response has failed. This embodiment takes the railway video monitoring system of a certain station as the object, and uses the actual operation screen recording video to test the PTZ camera control.

The following is the data result of a delay test. After selecting the area to be detected, the operator clicks "Start" to test, and obtains the start system timestamp _ts = 1683114014127 and the end system timestamp _te = 1683114014316. Therefore, the delay of the PTZ camera control response is t = _{te -ts} = 189ms. The test delay meets the requirements of the "Technical Specifications for Railway Integrated Video Surveillance Systems".

The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (8)

1. A method for measuring the response delay of a pan-tilt camera of a video monitoring system, characterized in that it comprises the following steps: Acquire test point information, and determine a video area to be detected based on the test point information; Issue control instructions to the PTZ based on the mouse click operation, and the moment of capturing the mouse click operation is the start moment; Based on feature matching and background subtraction algorithms, detecting video changes in the video area to be detected and performing contour calculation to obtain a video change contour area; A preset threshold value is used, and the first moment when the proportion of the contour area of the video change meets the threshold value is the end moment; Based on the time difference between the end time and the start time, the response delay of the pan-tilt camera is obtained. 2. The method for measuring the response delay of a pan-tilt camera of a video monitoring system according to claim 1, characterized in that: The process of determining the start time includes: when the test starts, the video monitoring system monitors the mouse click operation. When the pan-tilt camera control based on the mouse click operation is monitored, the current video monitoring system timestamp is recorded, and the time when the pan-tilt camera control instruction is issued is determined as the start time. 3. The method for measuring the response delay of a pan-tilt camera of a video monitoring system according to claim 1, characterized in that: The process of determining the end time includes: in the video area to be detected, based on the feature matching algorithm, feature extraction is performed on the two frames of images before and after the movement of the pan-tilt camera, the image space coordinate transformation parameters are calculated based on the extracted feature parameters, and the interference of the pan-tilt camera jitter is eliminated based on the transformation parameters; then, a background subtraction algorithm based on the OpenCV library is used to separate the foreground and background of the video frame in the pan-tilt video monitoring screen, and the object contour of the foreground image is obtained. The video change contour area is calculated based on the object contour of the foreground image, and a threshold is preset. The first moment when the video change contour area meets the threshold is the end time. 4. The method for measuring the response delay of a pan-tilt camera of a video monitoring system according to claim 3, characterized in that: The process of eliminating the interference of PTZ camera jitter includes: The feature matching method based on ORB key points is used to describe the features of the image. The pixels that are significantly different from the surrounding pixels are found as key points in the image, and the descriptors of each key point are calculated. Perform feature matching on the image by using the Hamming distance between each descriptor in the descriptor set and the query descriptor; Sort all Hamming distances from small to large, and select the first 90% of the descriptors in the sorted sequence as the matching results; Calculate the homography matrix to determine the horizontal and vertical movement distances of the image, and eliminate the interference of the gimbal camera shake through the parallax method. 5. The method for measuring the response delay of a pan-tilt camera of a video monitoring system according to claim 4, characterized in that: The process of obtaining the moving distance based on the homography matrix includes: obtaining the homography matrix between the two images based on the position representation of the same point in two different images; solving the homography matrix based on the least squares method to obtain the mapping relationship between the two images, and obtaining the moving distance of the horizontal and vertical images based on the mapping relationship. 6. The method for measuring the response delay of a pan-tilt camera of a video monitoring system according to claim 3, characterized in that: Before obtaining the contour area of the video change, the foreground image is converted into a grayscale image, and pre-processed with Gaussian blur and binarization. 7. The method for measuring the response delay of a pan-tilt camera of a video monitoring system according to claim 6, characterized in that: The process of obtaining the video change contour area includes: depicting the object contour of the preprocessed foreground image based on the finfContours() function and the drawContours() function of the OpenCV library, and then calculating the video change contour area based on the object contour through the contourArea() function. 8. The method for measuring the response delay of a pan-tilt camera of a video monitoring system according to claim 1, characterized in that: The test point information includes the test point name and the number of test data.