CN102413320A - Method for realizing wireless network intelligent video monitoring system - Google Patents

Abstract

The invention provides a method for realizing a wireless network intelligent video monitoring system, which comprises the following steps: (1) collecting videos in real time; (2) video image coding and real-time transmission; (3) data forwarding; (4) receiving, decompressing and displaying the video; (5) detecting a moving object; (6) the angle of the camera can be adjusted at will. The invention can realize the real-time monitoring of a video monitoring system on a scene in a certain range, and the angle of the camera can be adjusted at will according to the requirements of workers, thereby enlarging the video monitoring range; the detection and framing of moving objects are added, and the functions of storing and recording specific images are added, so that the intelligence of a monitoring system is enhanced, and the workload of workers is reduced; the video acquisition system is connected with the data forwarding module through wifi, and the data forwarding module is connected with the monitoring system through a public network, so that the building steps of the system are simplified, the building cost of the system is reduced, and the expandability of the system is enhanced.

Description

Implementation method of a wireless network intelligent video surveillance system

technical field

The invention belongs to the field of security technology, and in particular relates to a method for realizing a wireless network intelligent video monitoring system.

Background technique

       In the past ten years, Internet technology and embedded technology have achieved rapid development. The Internet of Things based on the above two technologies has penetrated into various fields of daily life and production, improving the quality of production and life. A video monitoring technology integrating computer technology, network technology and digital video technology emerged as the times require. With the integration of automatic control, multimedia technology and 3G technology with video monitoring system, monitoring technology has also developed rapidly. The monitoring system uses the network to connect scattered and independent image collection points to achieve unified monitoring, unified management, and resource sharing across regions, and provides users with a brand-new, intuitive management tool that expands the visual range. At present, network video surveillance has penetrated into various fields of education, government, entertainment venues, hospitals, hotels, sports venues, and urban security.

Most of the existing video surveillance systems are mainly based on analog image information, which cannot meet the application requirements of real-time video surveillance. The degree of intelligence is low, and there are certain technical defects. First, each channel of video needs to have its own independent transmission line, the network utilization rate is low, and there is a lot of bandwidth waste. Second, the video surveillance system often only realizes the real-time playback of images, requiring specialized personnel to keep watching the monitoring screen, the workload is heavy, and the system intelligence is not high. Third, the construction of the monitoring system needs to arrange special network lines, which is complicated and costly.

Contents of the invention

Aiming at the defects existing in the prior art, the object of the present invention is to provide a method for realizing a wireless network intelligent video surveillance system, which can expand the scope of video surveillance and realize the detection of moving objects under the premise of making full use of bandwidth and system performance , improve the intelligence of the system. The monitoring system of the present invention can realize real-time monitoring of scenes within a certain range, and adjust the monitoring angle arbitrarily according to the orders of the staff, realize the detection and framing of moving objects within the monitoring range, and enhance the intelligent level of the system. The video data is transmitted from the acquisition system to the data forwarding module through wifi (wireless fidelity wireless fidelity), and then the data forwarding module transmits the data to the monitoring center through the public wired network, which simplifies the system construction process and enhances the scalability of the system.

For achieving the above object, technical scheme of the present invention is:

A wireless network intelligent video surveillance reference hardware system, including USB camera, video acquisition board, data forwarding board, monitoring center PC, steering gear.

The camera in the system completes the collection of video images, the video acquisition board completes the compression of video data and the execution of control commands, the data forwarding board completes the forwarding of video data and control commands, and the PC in the monitoring center completes data decompression and detection of moving objects. The steering gear controls the camera monitoring angle.

A method for realizing a wireless network intelligent video surveillance system, comprising the following steps:

(1) Real-time video capture, equipped with an arm (advanced risc machines a processor platform) platform with embedded linux, through the video capture driver program that comes with the system, the initialization of the USB camera is realized, and the collected video data is provided by Kernel space is mapped to user space.

(2) Video image encoding and real-time transmission, use the function provided by the open source video encoding library xvidcore to compress the video data into mpeg4 (moving picture experts group-4 fourth generation standard of moving picture experts group) standard data, and compress the compressed data Encapsulate it into an rtp (real-time transport protocol, real-time transport protocol) package, and send the data out at the respective designated ports.

(3) Forwarding of video data and control commands, the forwarding node is also an arm platform equipped with embedded linux, which opens two threads and one thread for video data and control commands respectively, and sends video data from the local wireless network to the remote monitoring center , and send the control command from the monitoring center to the video acquisition node of the local wireless network.

(4) For video reception and decompression, the PC in the remote monitoring center uses the same open source library to decompress the data from the designated port into RGB image format and display it on the screen.

(5) Moving object detection, converting the decompressed RGB image format into a YUV image format, and subtracting data from the current image and the background image in the YUV space, performing threshold segmentation on the subtracted data, and obtaining moving objects in the image data in space.

(6) The monitoring center adjusts the monitoring angle of the monitoring camera in real time, and the system sends the control command of the monitoring center to the video acquisition system through the data forwarding node through the socket interface to control the rotation of the steering gear equipped with the camera.

The step (1) real-time video acquisition is to use the video acquisition driver v4l2 (video for linux 2) that comes with embedded linux to collect the data in the USB camera and map it to the user storage space that the application program can operate. The specific steps are:

1) Cut out the linux kernel, compile the USB camera driver program adopted by the present invention into the kernel, and burn it to the flash of the development board;

2) Use the interface function provided by v4l2 to initialize the camera respectively, and set the image format and size of the captured video;

3) Map the video data stored in the kernel space to the user space where the application can operate through the mmap (memory map memory mapping) function;

4) The selected chipset has a jpeg (joint photographic experts group) hardware compressor inside, and calls the jpeg decoding function that comes with the system to decode the video data;

After memory mapping and jpeg decoding, the data collected from the camera is stored in the respective buffers in RGB format and frame units. The buffer stores 3 frames of images, and the data collection and processing are carried out in two threads.

The video compression and sending in the step (2) refers to using an open source library to encode the data into mpeg4 standard data and sending it out using the rtp transfer function at the set port. Specific steps are as follows:

1) Cross-compile the open source video encoding library xvidcore and the open source real-time transmission library jrtplib, and copy the generated library files to the embedded linux file system.

2) Create an encoding routine and a transmission thread in the main thread, and specify the data sending port number.

3) In the video capture thread, call the corresponding encoding thread to process the image data stored in the user space, and encode it into video data conforming to the mpeg4 standard.

4) Since the volume of the compressed data is still larger than the volume of the smallest sending unit, it is necessary to cut the compressed data according to the specified size, group the cut data according to the rtp packet format, and call the sending thread function to pack the completed Data is sent out on the specified port.

5) Through the flag bit of the rtp data packet, specify the data packets belonging to the same frame of image, which is convenient for the monitoring center to group the data.

Use the v4l2 program to continuously collect camera data, compress and send it at the same time. The driver of the wireless network card is transplanted into the video acquisition system, which supports the IEEE 802.11b/g protocol, and the bandwidth is 54M, which can meet the needs of real-time video transmission and can communicate with the network card that supports this protocol. Within a certain range, the video acquisition system can be installed in any position, which simplifies the construction process of the monitoring system.

The forwarding node of the video data and control command in the step (3), the hardware and software platform of the forwarding node is also arm and embedded linux, the forwarding node supports wired and wireless communication at the same time, and realizes the connection between the local wireless monitoring network and the remote monitoring center connect. The specific implementation steps are as follows:

1) Two threads are opened for video data forwarding, and one thread is opened for control command forwarding. The threads communicate through the shared storage area, and the shared storage area of the video forwarding thread is 3 video frames in size.

2) In the main process, create two rtp threads, and specify the IP address for receiving and sending, and the port number for receiving and sending. The receiving IP address is the IP address of the video acquisition system, and the sending IP address is the IP address of the control center PC. The receiving port number corresponds to the sending port number set by the video acquisition system, and the sending port number corresponds to the receiving port number set by the control center.

3) Create two socket threads in the main process, and specify the receiving and sending IP address and receiving and sending port number, the receiving IP address is the IP address of the control center, the sending IP address is the video acquisition IP address, and the receiving port number is set by the control center The sending port number corresponds to the sending port number, and the sending port number corresponds to the receiving port number set by the video capture system.

4) In the video data forwarding thread, use the semaphore to control the access of the two threads to the shared storage area. The access to the shared storage area adopts the queue method. The data received by the receiving thread is placed at the head of the queue, and the thread of the sending queue takes The data at the end of the queue is sent, and this forwarding process is carried out in an orderly manner, while ensuring the smooth transmission of data.

5) In the forwarding thread of the control command, since the byte size of the control command is relatively small, when the command data is received, the data header is detected first, and when it is determined to be a control command, the sending thread is executed to send the control command to the video receiving system , to control the adjustment of the camera acquisition angle.

The function of introducing the data forwarding node in the present invention is to make full use of the advantages of wifi communication, simplify the construction process of the monitoring system, and facilitate the reconstruction of the system. The video acquisition system supports wired network at the same time. When the monitoring system is relatively simple, the data forwarding node can be removed, and the video acquisition node directly communicates with the remote control center through the public network.

The video reception, decompression and display functions of the step (4) are completed by the PC in the monitoring center. The realization of this function does not require high configuration of the PC, and a PC with general configuration can realize real-time monitoring and control of the monitoring terminal. control. The specific implementation steps are as follows:

1) The development platform of the monitoring center software is VC++6.0, build an MFC project, and transplant the open-source video coding library xvidcore and the open-source real-time transmission library jrtplib in the project.

2) Add communication connection class, data receiving class, data decompression class and image processing class respectively in the program.

3) In the main class, realize the initialization of relevant data in the three classes, and establish a link with the data forwarding node (data collection node).

4) In the communication connection class, establish an rtp thread and a socket thread to realize the rtp link with the data forwarding node (data collection node) and the socket binding of the control command forwarding.

5) In the data receiving class, in the receiving thread, continuously monitor and receive the information of the specified port. After receiving the data, save the data in the buffer, and identify the flag bits in the rtp header, and group the data. When a complete frame of data is received, the data is sent to the data processing class in the form of a message for processing.

6) In the data processing class, complete the data decompression, display, screenshot and video recording, and moving object detection. For the data sent from the data receiving class, first call the function in the encoding library to decompress and get the RGB image. Whether to perform motion detection is determined by a bool value in this class, and the bool value can be set by the staff through the monitoring interface. When moving object detection is not performed, the decompressed RGB image is directly displayed on the monitoring interface in the form of a picture. When moving object detection is required, the decompressed RGB image is first converted into a YUV image, and the image is processed in the YUV space. The frame difference operation detects the outline of the moving object.

7) When a screenshot or video command is detected, the function in the data operation class will save the compressed image as a picture in bmp format named after the current time or open a video file named after the current time, and decompress it Finally, the picture is obtained and stored in the opened video file until an order to stop recording is received.

Since the control center needs to monitor the video, the decompression, display and moving object detection of the image data should be realized in the image processing class.

In the step (5), the detection of the moving object is realized, and the detection of the moving object is divided into the optical flow method and the frame difference method. The present invention uses the frame difference method to operate the image data in the YUV color space, and performs macro on the image Block division reduces the amount of data calculation and increases real-time performance. Specific steps are as follows:

1) After receiving the motion detection command, convert the decompressed RGB image format into a YUV image format.

2) In the motion detection based on the frame difference method, the background update strategy occupies an important position. The present invention proposes the following background update strategy on the basis of experimental analysis: after starting the motion detection, the current frame is used as the background frame to calculate the frame difference method. The decompressed image at that time is the background frame; when a moving object is detected, the background update is stopped, and the original background frame is used as the background frame.

3) Since the storage order of YUV is Y1Y2...YnU1U2...UnV1V2...Vn, in order to reduce the amount of calculation, the data is firstly divided into macroblocks. The invention defines that a macroblock is composed of 8×8 pixels. For this The 320 × 240 video format adopted by the invention will obtain 40 × 30 macroblock matrices. The size of the array in each macroblock matrix in the Y space is the average value of Y corresponding to 64 pixels. For the U and V spaces, The size of the array in each macroblock is the average of U and V of the corresponding 16 pixels. After the calculation, the macroblock matrices of the three-dimensional color space are respectively My, Mu, and Mv. Using these three macroblock matrices to represent image data for frame difference calculation can reduce the amount of calculation.

4) In order to facilitate the detection and framing of multiple moving objects in an image, the storage of the macroblock matrix elements is in the form of columns first and then rows, that is, the first column of pixel data is stored from top to bottom, and then the second column of pixel data is stored. Two columns of pixel data. The background frame data and the foreground frame data are respectively processed as above.

5) When calculating the frame difference method, subtract the YUV three-dimensional macroblock matrix to obtain the difference matrix. Binarize the obtained three difference matrices respectively, take a threshold of 15 for the difference matrix of Y space, and take a threshold of 5 for the difference matrix of U and V spaces.

After the binarization is completed, the coordinate values of the non-zero elements in each macroblock matrix are stored, and the coordinate values in the U and V spaces are ORed, and then the coordinate values obtained after the OR operation and the coordinate values of the Y space are used to obtain And operation, finally get the coordinate value of the moving object range in the image space.

6) After the detection of the moving object is completed, the framing of the moving object can be realized. Two framing forms can be selected, one is macro block framing, which can well reflect the outline of the moving object, and the other framing form is rectangular framing. When performing rectangular frame timing, first calculate the maximum and minimum values of the horizontal and vertical coordinates of each moving object within an image range, and use these four values as the endpoints to make a rectangle to achieve separate framing of the moving objects.

7) When the bool value indicating the detection of moving objects is false, or when a control command is detected and the camera rotates, the detection of moving objects will be stopped.

Experiments show that the frame difference method proposed by the present invention can effectively and accurately detect moving objects within the image range, and can frame multiple moving objects separately, effectively improving the intelligence level of the monitoring system and ensuring the safety of security sex. Two channels of video data can be used for motion detection at the same time or separately.

In the step (6), the control of the local video acquisition system by the remote monitoring center is realized, because the real-time requirement is not very high, and the control command is transmitted by socket programming. Specific steps are as follows:

1) Establish a socket routine in the communication class, and realize the binding with the data forwarding node (video acquisition system).

2) Create a command format for effective transmission of control commands. In the present invention, the command format is as follows:

00001010 ××××××××

The command header is defined as 0x0a, and the next byte is defined as the command content.

3) When it is detected that the command is to be sent, a complete command is formed according to the command format, and sent through the socket sending function to realize the adjustment of the camera angle.

Description of drawings

       FIG. 1 is a structural block diagram of the reference hardware system of the present invention.

Figure 2 is the experimental platform used in the present invention.

Figure 3 is a flow chart of the program of the present invention.

Figure 4 is a flowchart of the data forwarding module.

Figure 5 is a flow chart of the moving object detection module.

Figure 6 is one of the screenshots of the experimental results.

Figure 7 is the second screenshot of the experimental results.

Detailed ways

The implementation of the present invention will be described in detail below in conjunction with the accompanying drawings:

As shown in Figure 1, a wireless network intelligent video surveillance reference hardware system includes a USB camera, a video acquisition board, a data forwarding board, a monitoring center PC, and a steering gear.

The camera in the system completes the collection of video images, the video acquisition board completes the compression of video data and the execution of control commands, the data forwarding board completes the forwarding of video data and control commands, and the PC in the monitoring center completes data decompression and detection of moving objects. The steering gear controls the camera monitoring angle.

As shown in Fig. 2, the experimental platform of the present invention is made up of two arm development boards, is transplanted with embedded linux operating system (version is 2.6. The peripheral interfaces include USB wireless network card (the ASUS wl-167g wireless network card is used), and Ethernet interface. Wherein it is connected with the video acquisition module through the USB wireless network card, and connected with the remote control center through the Ethernet module. The development board on the right is used as a video acquisition module. The peripheral interfaces used include USB wireless network card, USB camera (the camera adopts Vimicro’s zc301 chipset), built-in hardware jpeg compressor, and pwm (pulse width modulation) interface And the serial port, which is connected to the data forwarding module through the USB wireless network card, uses the USB camera to collect video images, uses the pwm interface to control the rotation of the steering gear, realizes the angle adjustment of the camera, connects to the console through the serial port line, and uses it during system debugging.

As shown in Figure 3, a method for realizing a wireless network intelligent video surveillance system includes the following steps:

(1) Real-time video capture, the arm platform equipped with embedded linux, through the video capture driver that comes with the system, realizes the initialization of the USB camera, and maps the captured video data from the kernel space to the user space.

(2) Video image encoding and real-time transmission, use the function provided by the open source video encoding library xvidcore to compress the video data into mpeg4 standard data, encapsulate the compressed data into rtp packets, and send the data out at the respective designated ports.

(3) Forwarding of video data and control commands, the forwarding node is also an arm platform equipped with embedded linux, and two threads are opened for two channels of video data and one channel of control commands, and the video data is sent from the local wireless network to the remote monitoring center , to send the control command from the monitoring center to the video acquisition node of the local wireless network.

(4) For video reception and decompression, the PC in the remote monitoring center uses the same open source library to decompress the data from the designated port into RGB image format and display it on the screen.

(5) The moving object detection module converts the decompressed RGB image format into a YUV image format, and uses the current image and the background image to perform data subtraction in the Y space, and performs threshold segmentation on the subtracted data to obtain the moving object in the image data in space.

(6) The monitoring center can adjust the monitoring angle of the monitoring camera in real time, and the system can send the control command of the monitoring center to the video acquisition system through the data forwarding node through the socket interface to control the rotation of the steering gear equipped with the camera.

As shown in Figure 4, step (3) is the forwarding of video data and control commands, the forwarding node of the video data and control command in step (3), the hardware and software platform of the forwarding node is also arm and embedded linux, forwarding The node supports wired and wireless communication at the same time, realizing the connection between the local wireless monitoring network and the remote monitoring center. The specific implementation steps are as follows:

1) Two threads are opened for video data forwarding, and one thread is opened for control command forwarding. The threads communicate through the shared storage area, and the shared storage area of the video forwarding thread is 3 video frames in size.

2) In the main process, create two rtp threads, and specify the IP address for receiving and sending, and the port number for receiving and sending. The receiving IP address is the IP address of the video acquisition system, and the sending IP address is the IP address of the control center PC. The receiving port number corresponds to the sending port number set by the video acquisition system, and the sending port number corresponds to the receiving port number set by the control center.

3) Create two socket threads in the main process, and specify the receiving and sending IP address and receiving and sending port number, the receiving IP address is the IP address of the control center, the sending IP address is the video acquisition IP address, and the receiving port number is set by the control center The sending port number corresponds to the sending port number, and the sending port number corresponds to the receiving port number set by the video capture system.

4) In the video data forwarding thread, use the semaphore to control the access of the two threads to the shared storage area. The access to the shared storage area adopts the queue method. The data received by the receiving thread is placed at the head of the queue, and the thread of the sending queue takes The data at the end of the queue is sent, and this forwarding process is carried out in an orderly manner, while ensuring the smooth transmission of data.

5) In the forwarding thread of the control command, since the byte size of the control command is relatively small, when the command data is received, the data header is detected first, and when it is determined to be a control command, the sending thread is executed to send the control command to the video receiving system , to control the adjustment of the camera acquisition angle.

The function of introducing the data forwarding node in the present invention is to make full use of the advantages of wifi communication, simplify the construction process of the monitoring system, and facilitate the reconstruction of the system. The video acquisition system supports wired network at the same time. When the monitoring system is relatively simple, the data forwarding node can be removed, and the video acquisition node directly communicates with the remote control center through the public network.

As shown in Figure 5, the detection of moving objects is realized in step (5). The detection of moving objects is divided into optical flow method and frame difference method. The present invention uses frame difference method to operate image data in YUV color space, and The image is divided into macroblocks, which reduces the calculation amount of data and increases real-time performance. Specific steps are as follows:

1) After receiving the motion detection command, convert the decompressed RGB image format into a YUV image format.

2) In the motion detection based on the frame difference method, the background update strategy occupies an important position. The present invention proposes the following background update strategy on the basis of experimental analysis: after starting the motion detection, the current frame is used as the background frame to calculate the frame difference method. The decompressed image at that time is the background frame; when a moving object is detected, the background update is stopped, and the original background frame is used as the background frame.

3) Since the storage order of YUV is Y1Y2...YnU1U2...UnV1V2...Vn, in order to reduce the amount of calculation, the data is firstly divided into macroblocks. The invention defines that a macroblock is composed of 8×8 pixels. For this The 320 × 240 video format adopted by the invention will obtain 40 × 30 macroblock matrices. The size of the array in each macroblock matrix in the Y space is the average value of Y corresponding to 64 pixels. For the U and V spaces, The size of the array in each macroblock is the average of U and V of the corresponding 16 pixels. After the calculation, the macroblock matrices of the three-dimensional color space are respectively My, Mu, and Mv. Using these three macroblock matrices to represent image data for frame difference calculation can reduce the amount of calculation.

4) In order to facilitate the detection and framing of multiple moving objects in an image, the storage of the macroblock matrix elements is in the form of columns first and then rows, that is, the first column of pixel data is stored from top to bottom, and then the second column of pixel data is stored. Two columns of pixel data. The background frame data and the foreground frame data are respectively processed as above.

5) When calculating the frame difference method, subtract the YUV three-dimensional macroblock matrix to obtain the difference matrix. Binarize the obtained three difference matrices respectively, take a threshold of 15 for the difference matrix of Y space, and take a threshold of 5 for the difference matrix of U and V spaces. The formula is as follows:

After the binarization is completed, the coordinate values of the non-zero elements in each macroblock matrix are stored, and the coordinate values in the U and V spaces are ORed, and then the coordinate values obtained after the OR operation and the coordinate values of the Y space are used to obtain And operation, finally get the coordinate value of the moving object range in the image space.

6) After the detection of the moving object is completed, the framing of the moving object can be realized. Two framing forms can be selected, one is macro block framing, which can well reflect the outline of the moving object, and the other framing form is rectangular framing. When performing rectangular frame timing, first calculate the maximum and minimum values of the horizontal and vertical coordinates of each moving object within an image range, and use these four values as the endpoints to make a rectangle to achieve separate framing of the moving objects.

7) When the bool value indicating the detection of moving objects is false, or when a control command is detected and the camera rotates, the detection of moving objects will be stopped.

Experiments show that the frame difference method proposed by the present invention can effectively and accurately detect moving objects within the image range, and can frame multiple moving objects separately, effectively improving the intelligence level of the monitoring system and ensuring the safety of security sex. Two channels of video data can be used for motion detection at the same time or separately.

As shown in Figure 6 and Figure 7, it is the experimental part of the present invention, that is, the functional verification part.

As shown in Figure 6, it is a screenshot of the monitoring interface without the function of moving object detection enabled. As shown in Figure 6(a), it is the monitoring image at the time of initialization, and Figure 6(b) is a screenshot of the monitoring interface after adjusting the camera angle, which can verify that the basic functions of the monitoring system have been realized, and the control center will monitor the camera Angle control function can also be realized.

As shown in Figure 7, it is a screenshot of the monitoring interface with the moving object detection function enabled. As shown in Figure 7(a), it is a screenshot of the interface when one person passes through the monitoring scene, and as shown in Figure 7(b), it is a screenshot of the interface when two people pass through the monitoring scene.

Figure 7(a) and Figure 7(b) show that the moving object is framed by a rectangular frame, and Figure 7(c) shows that the moving object is framed by the calculated macroblock.

As shown in Figure 6 and Figure 7, the functions of this intelligent video surveillance system have been fully realized.

Claims (7)

1. an implementation method of a wireless network intelligent video surveillance system, is characterized in that, comprises the following steps: (1) Real-time video capture, the arm platform equipped with embedded linux, through the video capture driver that comes with the system, realizes the initialization of the USB camera, and maps the captured video data from the kernel space to the user space; (2) Video image coding and real-time sending, use the function provided by the open source video coding library xvidcore to compress the video data into mpeg4 standard data, encapsulate the compressed data into rtp packets, and send the data out at the respective designated ports; (3) Forwarding of video data and control commands, the forwarding node is also an arm platform equipped with embedded linux, which opens two threads and one thread for video data and one control command respectively, and sends video data from the local wireless network to remote monitoring center, and send the control command from the monitoring center to the video acquisition node of the local wireless network; (4) For video reception and decompression, the PC in the remote monitoring center uses the same open source library to decompress the data from the designated port into RGB image format and display it on the screen; (5) Moving object detection, converting the decompressed RGB image format into a YUV image format, subtracting the data of the current image and the background image in the YUV space, performing threshold segmentation on the subtracted data, and obtaining the moving object in the image data in space; (6) The monitoring center adjusts the monitoring angle of the monitoring camera in real time, and the system sends the control command of the monitoring center to the video acquisition system through the data forwarding node through the socket interface to control the rotation of the steering gear equipped with the camera. 2. the realization method of a kind of wireless network intelligent video monitoring system according to claim 1 is characterized in that, described step (1) video real-time collection is to utilize the video collection driver v4l2 that embedded linux carries to USB camera The data in is collected and mapped to the user storage space that the application can operate; the specific steps are: Cut out the linux kernel, compile the USB camera driver into the kernel, and burn it to the flash of the development board; Use the interface function provided by v4l2 to initialize the camera respectively, and set the image format and size of the captured video; Map the video data stored in the kernel space to the user space that the application can operate through the mmap function; The selected chipset has a jpeg hardware compressor inside, and calls the jpeg decoding function that comes with the system to decode the video data; After memory mapping and jpeg decoding, the data collected from the camera is stored in the respective buffers in RGB format and frame units. The buffer stores 3 frames of images, and the data collection and processing are carried out in two threads. 3. The implementation method of a wireless network intelligent video surveillance system according to claim 1, characterized in that the video compression and sending in the step (2) refers to using an open source library to encode data into mpeg4 standard data and Use the rtp transfer function to send out at the set port; the specific steps are as follows: 1) Cross-compile the open source video encoding library xvidcore and the open source real-time transmission library jrtplib, and copy the generated library files to the embedded linux file system; 2) Create an encoding routine and a transmission thread in the main thread, and specify the data sending port number; 3) In the video capture thread, call the corresponding encoding thread to process the image data stored in the user space, and encode it into video data conforming to the mpeg4 standard; 4) Since the volume of the compressed data is still larger than the volume of the smallest sending unit, it is necessary to cut the compressed data according to the specified size, group the cut data according to the rtp packet format, and call the sending thread function to pack the completed The data is sent out on the specified port; 5) Through the flag bit of the rtp data packet, specify the data packet belonging to the same frame of image, which is convenient for the monitoring center to group the data; Use the v4l2 program to continuously collect camera data, compress and send it at the same time; the video capture system transplants the driver of the wireless network card, supports IEEE 802.11b/g protocol, and has a bandwidth of 54M, which can meet the needs of real-time video transmission. Communicate with the network card that supports this protocol; within a certain range, the video acquisition system can be installed at any location, which simplifies the construction process of the monitoring system. 4. The implementation method of a wireless network intelligent video surveillance system according to claim 1, characterized in that, the forwarding node of the video data and the control command in the step (3), the hardware and software platform of the forwarding node is also arm And embedded linux, the forwarding node supports wired and wireless communication at the same time, realizing the connection between the local wireless monitoring network and the remote monitoring center; the specific steps are as follows: 1) Open two threads for video data forwarding, open one thread for control command forwarding, the threads communicate through the shared storage area, and the shared storage area of the video forwarding thread is 3 video frames in size; 2) In the main process, create two rtp threads, and specify the receiving and sending IP address and the receiving and sending port number; the receiving IP address is the IP address of the video acquisition system, and the sending IP address is the IP address of the control center PC; The receiving port number corresponds to the sending port number set by the video acquisition system, and the sending port number corresponds to the receiving port number set by the control center; 3) Create two socket threads in the main process, and specify the receiving and sending IP address and receiving and sending port number, the receiving IP address is the IP address of the control center, the sending IP address is the video acquisition IP address, and the receiving port number is set by the control center The sending port number corresponds to the sending port number, and the sending port number corresponds to the receiving port number set by the video acquisition system; 4) In the video data forwarding thread, use the semaphore to control the access of the two threads to the shared storage area. The access to the shared storage area adopts the queue method. The data received by the receiving thread is placed at the head of the queue, and the thread of the sending queue takes The data at the end of the queue is sent, and the forwarding process is carried out in an orderly manner, while ensuring the smooth transmission of data; 5) In the forwarding thread of the control command, since the byte size of the control command is relatively small, when the command data is received, the data header is detected first, and when it is determined to be a control command, the sending routine is executed to send the control command to the video receiver The system controls the adjustment of the camera acquisition angle; The video acquisition system supports wired network at the same time. When the monitoring system is relatively simple, the data forwarding node can be removed, and the video acquisition node directly communicates with the remote control center through the public network. 5. The implementation method of a wireless network intelligent video monitoring system according to claim 1, characterized in that the video receiving, decompressing and displaying functions of the step (4) are completed by a PC in the monitoring center, and this function The realization of PC does not have high requirements for PC configuration, and a PC with general configuration can realize real-time monitoring and control of monitoring terminals; the specific steps are as follows: 1) The monitoring center software development platform is VC++6.0, build an MFC project, and transplant the open source video coding library xvidcore and open source real-time transmission library jrtplib in the project; 2) Add communication connection class, data receiving class, data decompression class and image processing class respectively in the program; 3) In the main class, realize the initialization of relevant data in the three classes, and establish links with data forwarding nodes or data collection nodes; 4) In the communication connection class, establish rtp routines and socket routines to realize the rtp link with the data forwarding node or data collection node and the socket binding for control command forwarding; 5) In the data receiving class, in the receiving thread, continuously monitor and receive the information of the specified port, after receiving the data, save the data in the buffer, and identify the flag bits in the rtp packet header, and group the data; when After receiving a complete frame of data, send the data to the data processing class in the form of a message for processing; 6) In the data processing class, complete the data decompression, display, screenshot and video recording, and moving object detection; for the data sent from the data receiving class, first call the function in the encoding library to decompress and obtain the RGB image; whether The motion detection is determined by a bool value in this class, and the bool value can be set by the staff through the monitoring interface; when the moving object detection is not performed, the RGB image obtained after decompression is directly displayed on the monitoring interface in the form of a picture, When moving object detection is required, the decompressed RGB image is first converted into a YUV image, and the frame difference operation of the image is performed in the YUV space to detect the outline of the moving object; 7) When a screenshot or video command is detected, the function in the data operation class will save the compressed image as a picture in bmp format named after the current time or open a video file named after the current time, and decompress it Then get the picture and store it in the opened video file until the command to stop recording is received; Since the control center needs to monitor the video, the decompression, display and moving object detection of the image data should be realized in the image processing class. 6. The implementation method of a wireless network intelligent video surveillance system according to claim 1, characterized in that, in the step (5), the detection of moving objects is realized, and the image data of the YUV color space is processed by using the frame difference method Perform operations and divide the image into macroblocks, which reduces the amount of data calculation and increases real-time performance; the specific steps are as follows: 1) After receiving the motion detection command, convert the decompressed RGB image format into a YUV image format; 2) In the motion detection based on the frame difference method, the background update strategy occupies an important position; on the basis of the experimental analysis, the following background update strategy is proposed: after the motion detection is started, the frame difference method is calculated with the current frame as the background frame, If no moving object is found, the background is updated every 10 frames, and the image decompressed at that time is used as the background frame for each update; when a moving object is detected, the background update is stopped, and the original background frame is used as the background frame; 3) Since the storage order of YUV is Y1Y2...YnU1U2...UnV1V2...Vn, in order to reduce the amount of calculation, first divide the data into macroblocks, and define a macroblock with 8×8 pixels, for 320×240 In the video format, a 40×30 macroblock matrix will be obtained. For the Y space, the size of the array in each macroblock matrix is the average value of the corresponding Y of 64 pixels. For U and V spaces, the array size in each macroblock The size is the average value of U and V of the corresponding 16 pixels; after calculation, the macroblock matrices of the three-dimensional color space are respectively My, Mu, and Mv, and the frame difference calculation can be reduced by using these three macroblock matrices to represent image data. Calculations; 4) In order to facilitate the detection and framing of multiple moving objects in an image, the storage of the macroblock matrix elements is in the form of columns first and then rows, that is, the first column of pixel data is stored from top to bottom, and then the second column of pixel data is stored. Two columns of pixel data; the background frame data and the foreground frame data are respectively processed as above; 5) When calculating the frame difference method, the YUV three-dimensional macroblock matrix is subtracted and calculated to obtain the difference matrix; the three difference matrices obtained are binarized respectively, and the difference matrix in the Y space is taken The threshold is 15, and the threshold is 5 for the difference matrix of U and V spaces; After the binarization is completed, the coordinate values of the non-zero elements in each macroblock matrix are stored, and the coordinate values in the U and V spaces are ORed, and then the coordinate values obtained after the OR operation and the coordinate values of the Y space are used to obtain And operation, finally get the coordinate value of the moving object range in the image space; 6) After completing the detection of moving objects, realize the framing of moving objects. Two framing forms can be selected, one is macro block framing, which can well reflect the outline of moving objects, and the other framing form is rectangular framing; When performing rectangular frame timing, first calculate the maximum and minimum values of the horizontal and vertical coordinates of each moving object within an image range, and use these four values as the endpoints to make a rectangle to achieve separate framing of the moving objects; 7) When the bool value indicating the detection of moving objects is false, or when a control command is detected and the camera rotates, the detection of moving objects will be stopped. 7. The implementation method of a wireless network intelligent video monitoring system according to claim 1, characterized in that, in the step (6), the remote monitoring center controls the local video acquisition system, because real-time requirements It is not very high, and the control command is transmitted by socket programming; the specific steps are as follows: Create a socket routine in the communication class, and realize the binding with the data forwarding node (video acquisition system); Create a command format to control the effective transmission of commands, the command format is as follows: 00001010 ×××××××× The command header is defined as 0x0a, and the next byte is defined as the command content; When it is detected that the command is to be sent, a complete command is formed according to the command format, and sent through the socket sending function to realize the adjustment of the camera angle.

Images