CN116600162A

CN116600162A - Progressive audio and video time stamp synchronization method and system

Info

Publication number: CN116600162A
Application number: CN202310736575.9A
Authority: CN
Inventors: 李念强; 张宇
Original assignee: University of Jinan
Current assignee: University of Jinan
Priority date: 2023-06-21
Filing date: 2023-06-21
Publication date: 2023-08-15

Abstract

The invention provides a brand new progressive audio and video time stamp synchronization method and system. The method has the advantages of rapidness, accuracy, stability, reliability and the like, and is suitable for various network environments and scenes. By means of establishing a reference node and a reference clock, dynamically adjusting the clock synchronization state of the node and the like, the audio and video time stamp synchronization can be more accurately realized. The time stamp synchronization in audio and video data transmission can be realized efficiently, reliably and accurately, and the user's looking and feel experience is improved.

Description

Progressive audio and video time stamp synchronization method and system

Technical Field

The invention relates to the technical field of audio and video, in particular to an audio and video time stamp synchronization technology. In particular to a progressive audio and video time stamp synchronization method and system.

Background

With the wide application of multimedia data, audio and video technologies are rapidly developed. In the process of audio and video data transmission, processing and playing, the synchronization of audio and video time stamps is very critical. When the audio and video time stamps are not synchronous, the problems of image jitter, intermittent and the like can be caused, and the viewing experience of a user is affected. Therefore, achieving accurate audio and video timestamp synchronization is one of the hot spots of current audio and video technology research. With the increasing development of wireless communication technology, the real-time transmission requirement of multiple paths of audios and videos in the industries of video conferences, online education and the like is more and more urgent. However, since clocks of each transmission device are different, a time stamp deviation of an audio and video signal is easily caused, thereby affecting the effect of video playback and sound synchronization. The main methods currently solving this problem include synchronizing clocks using NTP (network time protocol) or adding time stamp information in the transmitted data packets. Some audio and video time stamp synchronization methods based on algorithms such as time domain filtering, statistics, deep learning and the like. However, these methods still have some problems, such as low synchronization accuracy for distributed video streams, very difficult signal discrimination, etc. Therefore, a more efficient and accurate audio-video timestamp synchronization method is needed.

Disclosure of Invention

The invention provides a progressive audio and video time stamp synchronization method and system. The method aims to realize accurate synchronization in audio and video data transmission, overcome the defects and the shortcomings existing in the prior art, and improve the efficiency and the accuracy of audio and video time stamp synchronization.

Specifically, the progressive audio and video time stamp synchronization method and system of the invention comprise the following steps.

S11: and determining an audio and video data transmission route, and selecting a specific reference node as a standard of the whole synchronous system. On a specific reference node, a time synchronization system is established and the offset between the clocks to this node and the other nodes is obtained. The offset may be obtained through a network or other communication means.

And S12, synchronizing the clocks of each node by using a set of accurate reference clocks. To optimize the synchronization effect, the clock rate may be adjusted in some nodes. This allows better maintenance of the synchronisation results.

And S13, processing each audio and video data stream according to the time stamp information received from the specific reference node so as to keep synchronization in the transmission process. The specific method comprises the following steps:

1. receiving an audio and video data stream and acquiring time stamp information of the data stream;

2. correcting the time stamp of each data packet;

3. the synchronization error is smoothed by interpolation or the like.

And S14, when a new node is added into the network, the clock synchronization state of the node can be dynamically adjusted through an incremental updating algorithm or other methods, so that progressive improvement is realized. The system clock bias is reduced while maintaining timestamp synchronization.

The invention provides a progressive audio and video time stamp synchronization method and a progressive audio and video time stamp synchronization system, which have the following advantages:

by determining a particular reference node and establishing a reference clock, it is possible to allow all nodes to perform more accurate time synchronization while reducing system clock skew.

By adopting the expansion type synchronization method, when a new node joins the network, the node synchronization state can be dynamically adjusted by incremental updating algorithm and the like, so that progressive improvement is realized.

Introducing machine learning techniques, such as deep learning, can further optimize the synchronization efficiency and accuracy of the algorithm.

The method is suitable for various network environments and can cope with different scenes and complex network topological structures.

Based on the method, high-efficiency, reliable and accurate audio and video data transmission can be realized, and the user's looking and feel experience is improved.

Drawings

Fig. 1 is a flowchart of a progressive audio and video time stamp synchronization method.

Fig. 2 is a schematic diagram of a progressive audio/video timestamp synchronization method and system implementation mode.

Fig. 3 is a block diagram of a PLL-based clock synchronization system.

Detailed Description

The following describes specific embodiments of the present invention in connection with specific examples.

The progressive audio and video time stamp synchronization method shown in fig. 1 specifically comprises the following steps:

s1: selecting a specific node in an audio and video data transmission route as a reference node, and establishing a time synchronization system;

s2: synchronizing and correcting the clocks of each node by using a set of accurate reference clocks, and optimizing clock synchronization results;

s3: processing each audio and video data stream according to the time stamp information received from the specific reference node so as to keep synchronization in the transmission process, including receiving the audio and video data stream, correcting the time stamp of each data packet, and smoothing the synchronization error by interpolation and other methods;

s4: when a new node is added to the network, the clock synchronization state of the node can be dynamically adjusted by an incremental update algorithm or other methods, thereby realizing progressive improvement and reducing the system clock deviation while maintaining the time stamp synchronization.

As shown in fig. 2, this embodiment mode includes one reference node (S) and three working nodes (A, B, C). The reference node sends a clock synchronization message to the working node via a network or other communication means, and the working node receives and uses the message to adjust its own internal clock. In order to make the clock synchronization more accurate, the reference node and the working node can perform periodic state monitoring, so that the clock synchronization is adjusted according to the monitoring data.

As shown in fig. 3, the time synchronization system in this embodiment mode includes a clock Phase Locked Loop (PLL). The PLL calculates the clock offset and rate adjustment value to be adjusted by comparing the time difference between the local clock and the reference clock and the rate of change of the reference clock. Thus, clock synchronization can be more accurate and stable.

In this embodiment, the node synchronization state may be dynamically adjusted by an incremental update algorithm as new nodes join the network, thereby achieving progressive improvement. In particular, when a new node is connected into the network, there may be some skew between its internal clock and the reference clock. At this time, the clock offset of the node may be calculated by an incremental update algorithm and transmitted to the reference node. After receiving the information, the reference node calculates the relative time difference between the node and other nodes and sends corresponding clock adjustment information. When the node receives the clock adjustment information, the node can synchronize according to the clock of the reference node, so that the time synchronism of the whole system is maintained.

The incremental update algorithm may be implemented in a variety of ways, such as Kalman filtering, linear prediction, neural networks, and so forth. In this embodiment, an incremental update algorithm based on linear prediction is used. The algorithm predicts the clock offset of the nodes by analyzing the track data among the nodes and adjusts the clock offset by taking the clock offset as a reference value. The algorithm can reduce the system clock deviation and improve the synchronization efficiency and precision while keeping time synchronization.

Claims

1. A progressive audio-video time stamp synchronization method, the method may further comprise the steps:

S1: Select a specific node as a reference node in the audio and video data transmission route, and establish a time synchronization system;

S2: By using a set of accurate reference clocks, the clocks of each node are synchronized and corrected, and the clock synchronization results are optimized;

S3: According to the timestamp information received from a specific reference node, process each audio and video data stream to keep it synchronized during transmission, including receiving audio and video data streams, and performing timestamp processing on each data packet Correction, and smoothing of synchronization errors through methods such as interpolation;

S4: When a new node is added to the network, the clock synchronization status of the node can be dynamically adjusted through an incremental update algorithm or other methods, so as to achieve gradual improvement and reduce the system time while maintaining timestamp synchronization. clock skew.

2. The method according to claim 1, characterized in that, selecting a specific node as a reference node in the audio-video data transmission route, and setting up a time synchronization system, specifically, determining the audio-video data transmission route, A specific reference node is selected as the basis for the entire synchronization system. On a specific reference node, establish a time synchronization system, and obtain the offset between the clocks of this node and other nodes. The offset can be obtained through the network or other communication means.

3. The method according to claim 1, wherein the incremental update algorithm can be realized by any one of methods such as Kalman filter, linear prediction, and neural network.

4. A progressive audio and video time stamp synchronization system, including:

A specific reference node is used to determine the transmission route of audio and video data and establish a time synchronization system;

A set of precise reference clocks is used to synchronize and correct the clocks of each node;

An audio and video data processor, which is used to process each audio and video data stream according to the time stamp information received from a specific reference node, so as to keep it synchronized during transmission;

An incremental update module that dynamically adjusts the clock synchronization status of nodes to achieve incremental improvements and reduce system clock skew while maintaining timestamp synchronization.

5. The system according to claim 3, wherein said audio-video data processor comprises a receiver, a time stamp corrector and a synchronization error smoother.

6. The system according to claim 3, wherein the incremental update module is implemented by an incremental update algorithm based on linear prediction.

7. The system of claim 3, wherein said incremental update module includes a trajectory analyzer, a predictor, and a clock adjuster.