CN108810475A - A kind of Android video monitoring apparatus based on Onvif standards and Sip agreements - Google Patents

Abstract

An Android video monitoring device based on the Onvif standard and the Sip protocol. The hard disk video recorder establishes a connection with the Android device by responding to the hard disk video recorder module; through the Web Service interactive module, user authentication is performed and the interactive address of Rtsp is obtained; through the Rtsp interactive module Obtain the video load type and video decoding information of the Android device; the Android device encodes and subpackages the collected video data through the video module and sends it to the hard disk video recorder or other Android clients; the Android device responds to the requested Android device through the Sip module. The client establishes a connection, and then sends Rtp packets through the Android video module. The invention satisfies the real-time communication between the personnel who go out to perform tasks and the central management personnel.

Description

An Android video monitoring device based on Onvif standard and Sip protocol

technical field

The invention relates to the field of video monitoring, in particular to a device for collecting, encoding and transmitting Android video based on the Onvif standard and the Sip protocol.

Background technique

In May 2008, Axis (AXIS), Bosch (BOSCH) and Sony (SONY) jointly announced the establishment of an international open network video product standard network interface development forum, named Onvif (Open NetworkVideo Interface Forum , Open Network Video Interface Forum), and jointly formulate open industry standards based on the principles of openness and openness. The Onvif standard will define a common protocol for information exchange between network video devices, including device discovery, real-time video, audio, metadata and control information.

The Onvif specification describes the model, interface, data type and data interaction mode of network video. And reuse some existing standards, such as Web Service series standards. The goal of the Onvif specification is to implement a network video framework protocol, so that network video products produced by different manufacturers (including recording front-ends, video equipment, etc.) are fully interoperable.

Sip (Session Initiation Protocol, Session Initiation Protocol) is an IP telephony signaling protocol proposed by IETF (Internet Engineering Task Force, Internet Engineering Task Force), and is a text protocol encoded using the UTF-8 character set. Sip is a communication protocol that defines how to connect and exchange information between communication devices (computers, phones, mobile phones, etc.), and Sip is also a signaling control protocol that can configure and manage any type of peer-to-peer Communication sessions, but don't care about the media type (voice, SMS, game, video, etc.). The Sip protocol has good scalability characteristics, can easily add definitions, embed various user terminals and quickly realize new functions; it has strong interoperability and good openness.

At present, most domestic camera front-ends and hard disk video recorders meet the Onvif specification, so that videos sent by different manufacturers' camera front-ends can be stored on different manufacturers' hard disk video recorders. However, due to the influence of the location of the camera front end, it can only monitor a fixed range, which is suitable for monitoring fixed places such as schools, hospitals, roads, airports, and governments. Although the Android device can use the Sip protocol to establish a session and then make a real-time video call, it cannot save the video data in the video call.

Contents of the invention

In order to solve the problem that the existing camera front end cannot meet the real-time communication between the personnel who go out to perform tasks and the center management personnel, and the current Android device video cannot store the collected video content in the hard disk video recorder for future viewing. The Android video surveillance device of Onvif standard and Sip protocol can perform video surveillance on the Android device side, and at the same time can store the collected video data to the device of the hard disk video recorder.

The technical solution adopted by the present invention to solve its technical problems is:

An Android video monitoring device based on the Onvif standard and the Sip protocol, including a response hard disk video recorder module, a Web Service interaction module, an Rtsp (Real Time Streaming Protocol, real-time streaming protocol) interaction module, an Android video module, a Sip registration module and a Sip response module;

The hard disk video recorder establishes a connection with the Android device by responding to the hard disk video recorder module; through the WebService interactive module, user authentication is performed and the interactive address of Rtsp is obtained; the video load type and video decoding information of the Android device is obtained through the Rtsp interactive module; Through the video module, the collected video data is encoded and subpackaged and sent to the hard disk video recorder or other Android clients; the Android device registers with the Sip server through the Sip registration module, and sends heartbeat packets to the Sip server at intervals of 20 seconds for protection. Live; establish a connection with the requested Android client through the Sip response module, and then send the Rtp packet through the Android video module.

Further, open an address and port service on the Android device side to listen to the broadcast of the DVR. When you click to search the network camera on the web client of the DVR, the Android device that is in the same LAN and meets the Onvif standard will receive the The Probe message initiated by the DVR needs to parse out the Uuid in the message body after getting the message, and then carry the Uuid and the Web Service address to obtain the Android device information in the reply message, and the DVR will proofread after receiving the message from the Android device. If the sent Uuid is consistent with the Uuid in the received message body, add the Android device to the camera list.

Further, the hard disk video recorder will call the Web Service method to the Android device according to the searched IP information of the Android device. The hardware resources of the device are limited, so a lightweight Http server is built on the Android device side, because the underlying layer of Http uses Socket for communication, and Socket is a lightweight remote communication, here we choose to encapsulate Socket messages into Http messages for sending . First perform user authentication on the Android device, then obtain the hardware information and supported video resolution information of the Android device, and improve the security of the Android device through two authentications during the interaction between the DVR and the Android device. The device side supports three video resolutions to meet video surveillance under different network conditions.

Furthermore, the Android device side turns on the camera to collect video data, calls the Android native Api for H264 hard coding on the collected raw data, and the encoded data is H264 raw data. When the length of one frame of encoded data is greater than Rtp The maximum length of the packet must be fragmented and encapsulated into an Rtp packet. First cache the fragmented Rtp packets in the queue, one thread stores Rtp packets in the queue, and one thread fetches Rtp packets from the queue and sends them at a certain rate, so as to realize sending Rtp packets at a uniform speed.

When other Android clients come to request the Android device, establish a Sip session, and judge from the Sip session whether a connection needs to be established with the DVR, and if a connection needs to be established, the Android response DVR module is opened, so as long as the Android The device is already in the camera list of the DVR. When the Android device transmits video data to the Android client that initiated the request, it will also send the same data to the DVR, realizing the storage of video data on the Android device and Web client The video monitoring of the Android device from the Android client and the Android client. Described response hard disk video recorder module, when hard disk video recorder initiates multicast, when searching for all network cameras that meet the Onvif standard in the same local area network, the Android device end (the Android device end being the Android end of the requested video) that meets the Onvif standard will receive Receive the detection information, and encapsulate the information such as the IP (Internet Protocol, the protocol for interconnection between networks) of the device into a format that conforms to the Soap (Simple Object Access Protocol, Simple Object Access Protocol), and send it to the hard disk video recorder through the Socket. At this time, The hard disk video recorder has found the Android device that meets the Onvif standard; if you need to search for the Android device across network segments, you need to provide routing support.

The hard disk video recorder will call the WebService method to the Android device according to the searched IP information of the Android device. Here, the user authentication of the Android device is performed first, and then the hardware information and the supported video resolution information of the Android device are obtained.

The hard disk video recorder will interact with the Android device through Rtsp. Here, it will first authenticate the user to the Android device, and then obtain the load type, data transmission method, address and port of the video stream on the Android device, and the SSRC (Synchronization source, synchronization source) to indicate the information; at the same time, the Android device will get the information to which port the DVR should send.

When the Android device receives the video playback information sent by the hard disk video recorder, it will open the Android camera, encode the data collected by the camera in H264, and then perform Rtp (Real-time Transport Protocol, real-time transport protocol) encapsulation. Good Rtp packets are put into the cache, and then sent to the hard disk video recorder at a certain rate.

Preferably, the Android device end registers to the Sip server by the Sip registration module, and after the registration is successful, it sends a heartbeat message to the Sip server, and the Sip server just knows which Android device ends are online, so that when other Android clients (Android clients are requesting video) Android end) to this Android device end when requesting video, other Android client ends send request video message to Sip server first, and Sip server detects that the requested Android device end is online, and just forwards the message to this Android device end, like this A Sip session is established between the Android device and the Android client. The Android device judges from the Sip message whether it needs to establish a connection with the DVR. If it needs to establish a connection, it opens the response DVR module of the Android device. The device is already in the camera list of the DVR, and the Android device will also send the same data to the DVR while transmitting the video data to the Android client that initiates the request, thus realizing the real-time connection between the Android device and the client. Preservation of video and video data.

The technical idea of the present invention is: Android, as a Linux free and open source operating system, provides a good foundation for customization, and with the development of mobile networks and the popularization of mobile terminals, remote video on mobile networks Communication becomes possible. Personnel who go out to perform tasks will carry their mobile phones with them. At this time, the center management system can use the mobile phone to conduct remote command and dispatch or check the situation on the spot and take corresponding measures. At the same time, the video information will be saved on the hard disk video recorder for future viewing. The combination of DVR and Android device not only meets the requirements of portability, but also can be stored in the DVR like the camera, and the video of the Android device can be directly viewed with the web client of the DVR, which has high engineering significance and market value.

The Android monitoring front-end device of the present invention supports three different resolution digital code streams, which are respectively 720P (1280*720), D1 (720*576), and three resolutions of CIF (352*288). Choose different resolutions according to the network quality of the Android device.

The beneficial effects of the present invention are mainly manifested in: (1) the Android device meets the Onvif standard, that is, the video of the Android device can be directly viewed on the terminal of the monitoring network camera, and stored in the hard disk video recorder. (3) Android devices use the Sip protocol to conduct point-to-point video calls between Android devices. (2) Digital code streams with multiple resolutions can satisfy video viewing under different network conditions.

Description of drawings

Figure 1 is a software framework design diagram of an Android video surveillance device based on the Onvif standard and the Sip protocol.

Figure 2 is the response to the DVR module.

Figure 3 is the Web service interaction module.

Figure 4 is the RTSP interactive module.

Figure 5 is the Android video module.

Figure 6 is the Sip registration module.

Figure 7 is a Sip response module.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings.

With reference to Fig. 1, a kind of Android video monitoring device based on Onvif standard and Sip agreement, described device is two-way video sending device, comprises response hard disk video recorder module, Web Service interactive module, Rtsp interactive module, Android video module, Sip registration module and Sip response module;

The hard disk video recorder establishes a connection with the Android device by responding to the hard disk video recorder module; through the WebService interactive module, user authentication is performed and the interactive address of Rtsp is obtained; the video load type and video decoding information of the Android device is obtained through the Rtsp interactive module; Through the video module, the collected video data is encoded and subpackaged and sent to the hard disk video recorder or other Android clients; the Android device registers with the Sip server through the Sip registration module, and sends heartbeat packets to the Sip server at intervals of 20 seconds for protection. Live; establish a connection with the requested Android client through the Sip response module, and then send the Rtp packet through the Android video module. Further, open an address and port service on the Android device side to listen to the broadcast of the DVR. When you click to search the network camera on the web client of the DVR, the Android device that is in the same LAN and meets the Onvif standard will receive the The Probe message initiated by the DVR needs to parse out the Uuid in the message body after getting the message, and then carry the Uuid and the Web Service address to obtain the Android device information in the reply message, and the DVR will proofread after receiving the message from the Android device. If the sent Uuid is consistent with the Uuid in the received message body, add the Android device to the camera list.

Further, the hard disk video recorder will call the Web Service method to the Android device according to the searched IP information of the Android device. The hardware resources of the device are limited, so a lightweight Http server is built on the Android device side, because the underlying layer of Http uses Socket for communication, and Socket is a lightweight remote communication, here we choose to encapsulate Socket messages into Http messages for sending . First perform user authentication on the Android device, then obtain the hardware information and supported video resolution information of the Android device, and improve the security of the Android device through two authentications during the interaction between the DVR and the Android device. The device side supports three video resolutions to meet video surveillance under different network conditions.

Furthermore, the Android device side turns on the camera to collect video data, calls the Android native Api for H264 hard coding on the collected raw data, and the encoded data is H264 raw data. When the length of one frame of encoded data is greater than Rtp The maximum length of the packet must be fragmented and encapsulated into an Rtp packet. First cache the fragmented Rtp packets in the queue, one thread stores Rtp packets in the queue, and one thread fetches Rtp packets from the queue and sends them at a certain rate, so as to realize sending Rtp packets at a uniform speed.

When other Android clients come to request the Android device, establish a Sip session, and judge from the Sip session whether a connection needs to be established with the DVR, and if a connection needs to be established, the Android response DVR module is opened, so as long as the Android The device is already in the camera list of the DVR. When the Android device transmits video data to the Android client that initiated the request, it will also send the same data to the DVR, realizing the storage of video data on the Android device and Web client The video monitoring of the Android device from the Android client and the Android client.

Figure 2 shows the interactive process of establishing a connection between the DVR and the Android device. First, analyze the multicast address and port sent by the DVR through packet capture, and then open a service on the Android device to monitor the broadcast address and port of the DVR. When you click to search the network camera on the web client of the DVR, the Android device that is in the same LAN and meets the Onvif standard will receive the Probe message initiated by the DVR. After getting the message, you need to parse out the Uuid in the message body , and then carry this Uuid and the Web Service address for obtaining Android device information in the reply message. After receiving the message from the Android device, the DVR checks the sent Uuid and the Uuid in the received message body. If they are consistent, the Android The device side is added to the camera list.

Referring to Fig. 3, the interaction between the hard disk video recorder and the Android device is carried out in the form of Web Service. Web Service is a network-based, distributed modular component that performs specific tasks. Web Service mainly utilizes Http (Hyper Text Transport Protocol, hypertext transfer protocol) and Soap to transmit data on the Web. Web users can use Soap and Http to call remote objects through the Web. And Soap is a kind of agreement based on XML (ExtensibleMarkup Language, extensible markup language), and a Soap message is exactly a common XML document, comprises following element: necessary Envelope element, can mark this XML document as a Soap message; The optional Header element contains header information; the required Body element contains all call and response information; the optional Fault element provides information about errors that occurred while processing this message. Because the Web Service is equivalent to an Http server, it needs to reply the Http message when receiving the Http message, but due to the limited hardware resources of the Android device, it can only build a lightweight Http server on the Android device side, because the bottom layer of Http uses Socket communicates, and Socket is a lightweight remote communication. Here, the Socket message is selected to encapsulate the Http message for sending. After going through the steps in Figure 2, click on the searched Android device, enter the account password of the Android device, and then request the DeviceInformation on the Android device from the Web Service address obtained in the steps in Figure 2. At this time, the message body does not carry Authentication message; the Android device will reply with an Unauthorized message with the key after receiving the message; the DVR will send the account number and encrypted password information to the Android device after receiving the message; the Android device will confirm that the password is correct After that, it will send its own hardware information to the DVR; then the DVR will request the Web Service address of Media from the Android device; after the Android device receives the message, it will send the corresponding Web Service address to the DVR; After the DVR receives the message, it will send another authentication message to the address; after receiving the message, the Andorid device will generate a new key and send it to the DVR because the message header does not contain the authentication message; the DVR receives After the message with the key, the account number and encrypted password information will be sent to the Android device; after receiving the message, the Android device will send the resolution information of the video supported by the device to the DVR. Here, two authentications are performed during the interaction between the DVR and the Android device, which can improve the security of the Android device.

Referring to Figure 4, after the steps in Figure 3 are executed, a resolution can be selected on the web client of the DVR, and a Describe message will be sent to the Android device after the resolution is selected, and the message body does not carry authentication message; the Android device receives a Describe message without an authentication message, and replies with a message of authentication failure, and carries its own key information; the DVR receives a message of authentication failure with a key, and passes the agreement The encryption algorithm encrypts the password of the Android device, and then sends it back to the Android device; the Android device decrypts the received encrypted information, and if the decryption result is consistent with its own password, it sends a Describe message of the video, which contains the video Load type, video decoding information, etc.; after receiving the Describe message, the DVR will reply with a port message with its own Rtp packet received; after receiving the message, the Android device will also reply with a port message with its own Rtp packet sent, so that the Android device The end will know which port to send the Rtp packet to the DVR; after the DVR receives the port message from the Android device, it will know which port the Andorid device sends the Rtp packet from, and then send a message to the Android device to start playing; the Android device After the end receives the message to start playing, it starts a thread to send the Rtp packet.

Referring to Figure 5, after the Android device receives the message of starting to send Rtp packets, it will turn on the camera of the Android device to collect video data. The data is hard-coded in H264. The hard-coded data is H264 bare data and cannot be directly transmitted. When the length of a frame of encoded data is greater than the maximum length of the Rtp packet, it must be fragmented and encapsulated into an Rtp packet. Considering that the data size of the main frame and sub-frame of the video obtained through H.264 compression encoding is quite different, and the underlying hardware of the mobile phone collects video at the same rate, so the dwell time of each frame is the same. If the frame rate is 25, the dwell time of each frame is 1/25 second. If the strategy of directly packaging and sending after encoding is adopted, the packet sending rate during the main frame is high because the data in the main frame is large and there are many fragmented packets. In contrast, because the sub-frame data is small and there are few fragmented packets, the packet sending rate during the sub-frame is low. And because the data size between the sub-frames is not equal, there is also a difference in the packet sending rate between the sub-frames. It can be seen that the packet sending rate of this kind of packet sending mechanism is unstable, and the packet sending rate is determined by the size of the frame. This will cause a packet sending peak period, causing network congestion, resulting in an increase in the packet loss rate and affecting the image quality of the receiving end. Fragmented Rtp packets are cached in the queue, one thread stores Rtp packets in the queue, and one thread fetches Rtp packets from the queue and sends them at a certain rate. The Android device-side video module sends Rtp packets to the streaming media server or hard disk video recorder as needed.

Referring to Figure 6, the Android device first needs to perform device registration, and needs to send a Register message to the Sip server. After receiving the Sip message, the server will analyze the Sip message, reply 200 OK to the Android device, and carry the encrypted seed required for the second step of registration in the message body. After receiving the 200 OK reply from the server, the Android device parses out the encrypted seed, obtains the password through the MD5 algorithm, and sends the Register message carrying the password to the server. After receiving the message, the server parses out the password and passes the verification. , if the password is correct, it will reply 200 OK to the Android device, and if it is wrong, it will reply 401 to the Android device, indicating that the login failed. In order to keep the Android device and the server online and provide a reminder, the Android device sends a Register message to the Sip server every 20 seconds, and the field in the message body is HeartBeat. When the server receives the Sip message and parses out the HeartBeat field, it judges that the message is a heartbeat keep-alive registration message, and replies 200 OK to the Android device as a response.

Referring to Figure 7, the Android video requester first sends a Request message to the Android device through the Sip server. This request message includes whether the video request needs to store the video data in the hard disk video recorder and the port and Ip address for sending the Rtp packet. When the Android device receives the message, if it needs to store the video after parsing the message, it will open the response DVR module, connect the DVR, and then reply a RequestResponse message to indicate that it has received the message. When the Android client receives the RequestResponse message, it will Send a Media message requesting this video data to the Android device. After receiving the message, the Android device sends the resolution and frame rate of this video call back to the requesting end, and then starts the Android video module. The video module is based on It is necessary to send Rtp data packets to the hard disk video recorder and the Android video request end, thus realizing the storage of real-time video and video data. Finally, when the Android client is about to end the video, it sends an end message to the Android device, and the Android device closes the video module after receiving the message.

Claims (5)

1. a kind of Android video monitoring apparatus based on Onvif standards and Sip agreements, it is characterised in that：Described device is two Road video transmission device, including response hard disk video recorder module, Web Service interactive modules, Rtsp interactive modules, Android video modules, Sip registration modules and Sip respond modules；

Hard disk video recorder establishes connection by responding hard disk video recorder module with Android device end；It is handed over by Web Service Mutual module carries out the authentication of user and obtains the interaction address of Rtsp；Android device is obtained by Rtsp interactive modules The video load type and video decoded information at end；Android device end by video module by collected video data into Row encodes subpackage and is sent to hard disk video recorder or other Android user terminals；Android device end mould is registered by Sip Block sends heartbeat packet to Sip servers according to 20 seconds intervals and carries out keep-alive to Sip server registrations；Mould is responded by Sip Block and the Android user terminals of request, which are established, to be connected, and then sends Rtp packets by Android video modules.

2. a kind of Android video monitoring apparatus based on Onvif standards and Sip agreements as described in claim 1, feature It is：The address and miniport service that one is monitored hard disk video recorder multicast are opened at Android device end, when in hard disk video recorder Webpage client on when clicking search web camera, in the same LAN and meet the Android of Onvif standards Equipment just will receive the Probe message of hard disk video recorder initiation, get the Uuid that parse after message inside message body, so This Uuid is carried in the message of reply afterwards and obtains the addresses Web Service of Android device information, hard disk video recorder The Uuid in the Uuid sent out and the message body received is proofreaded after the message for receiving Android device end, if unanimously, Android device end is added in camera list.

3. a kind of Android video monitoring apparatus based on Onvif standards and Sip agreements as claimed in claim 1 or 2, It is characterized in that：HD recording chance carries out Web according to the Android device end IP information searched, to Android device end The calling of Service methods, Web Service are equivalent to a Http server, need to reply when receiving Http message Http message, but since the hardware resource of Android device is limited, so building a lightweight at Android device end Http servers, because the bottom of Http is communicated using Socket, and Socket is a kind of telecommunication of lightweight, this In selection by Socket message encapsulation Http message send；User authentication is first carried out at Android device end, is then obtained The hardware information at Android device end and the video resolution information of support, by hard disk video recorder and Android device end into Authentication twice during row interaction, improves the safety at Android device end, and three kinds of videos are supported at Android device end Resolution ratio, to meet the video monitoring under heterogeneous networks situation.

4. a kind of Android video monitoring apparatus based on Onvif standards and Sip agreements as claimed in claim 1 or 2, It is characterized in that：The acquisition that camera carries out video data is opened at Android device end, is called to acquiring the initial data come Api primary Android carries out H264 hard codeds, and the data after coding are H264 uncorrected datas, frame data after coding Length is more than the maximum length of Rtp packets, to be also packaged into Rtp packets by fragment, the good Rtp packets of fragment are first cached to queue In, a thread toward depositing Rtp packets inside queue, a thread according to certain rate toward the transmission of Rtp packets is taken in queue, to realize At the uniform velocity send Rtp packets.

5. a kind of Android video monitoring apparatus based on Onvif standards and Sip agreements as claimed in claim 1 or 2, It is characterized in that：When asking the Android device end, Sip sessions are established, and from Sip sessions when other Android user terminals Judge whether to need to establish with hard disk video recorder to connect, if necessary to establish connection, is switched on the response hard disk of Android Recorder module, as long as then the Android device end is in the camera list of hard disk video recorder, which sets Standby end also can send identical number while to the Android user terminal transmitting video datas for initiating request to hard disk video recorder According to, realize Android device end video data storage and Web client and Android user terminals to Android device The video monitoring at end.