KR102819600B1 - A broadband network-based audio streaming broadcasting system that supports time synchronization of audio packets - Google Patents

Abstract

The present invention discloses a broadband network-based audio streaming broadcasting system that supports time synchronization of audio packets. The present invention is configured to support the same time synchronization of audio packets between an audio streaming server, an audio streaming client, and a control client, thereby enabling the use of a general external Internet network and a local broadband network together, and while transmitting IP stream audio through multicast, avoids the speed and data errors inherent in a general Internet network, thereby enabling an IP broadcasting environment to be built and used without building a separate local network, thereby efficiently operating audio data and AI-based voice recognition data, etc. even in a general broadband network.

Description

{A broadband network-based audio streaming broadcasting system that supports time synchronization of audio packets}

The present invention relates to audio streaming technology, and more particularly, to a wide area network-based audio streaming broadcasting system that supports time synchronization of audio packets so that audio packets streamed through a general wide area network (WAN) rather than a closed network can be broadcast while being time synchronized.

In general, broadcasting systems broadcast by amplifying audio signals from an audio transmission section through a power amplifier and transmitting the amplified signals as analog audio signals to speakers connected via cables. However, IP audio broadcasting systems use a wideband network to use a multicast-only router and a closed network, because there are situations where broadcasting is impossible with protocols other than multicast.

In other words, the above-mentioned broadband network-based multicast transmission is an efficient method that enables data transmission to all receiving hosts with only one transmission request from the transmitting host during the data transmission process.
However, the multicast transmission method has a problem of time synchronization between clients of the audio signal due to differences in network speeds between multiple routers and various network equipment, and this synchronization error causes an echo phenomenon and an overload due to a request for data retransmission, and when transmitting and receiving audio using multicast transmission in the above-mentioned broadband network, it affects the network speed of various types of network equipment connected to multiple routers, or a process of continuously checking and detecting data errors in the network occurs in order to extract the desired data, and in severe cases, the router itself goes down, causing a state in which the entire Internet network environment does not operate.

delete

That is, when using multicast transmission in a broadband network where a general Internet network is used, a synchronization problem may occur during the process of sending and receiving audio signals between equipment used with multiple routers, and packet synchronization errors may occur. Therefore, continuous confirmation requests must be made until the packets are accurately synchronized. However, if packet synchronization is not achieved, data packet errors occur, and not only normal audio transmission and reception are not possible, but even the use of the existing Internet network becomes impossible.

Accordingly, most conventional IP multicast audio transmission technologies had no choice but to solve the synchronization problem of audio data by transmitting and receiving network audio using multicast protocols over closed networks.

Meanwhile, another problem was that when using multiple audio stream clients, the delay time for each client's sound transmission was different, and there was a problem that the sound was transmitted with an echo phenomenon through speakers installed in the same section.

For example, in cases where multiple IP network audio devices are installed in a single building, the signal synchronization problem is solved by transmitting audio signals to IP network audio transmitting speakers using a multicast protocol in a closed network. The reason for this is that when general Internet network lines are used, if accurate transmission data packet synchronization is not achieved due to various reasons such as the number and type of client IP network output devices and installation distance, an overload occurs due to data analysis and retransmission requests to continuously achieve accurate data synchronization, which significantly disrupts existing Internet network signal processing and makes normal Internet operation impossible.

That is, in order to transmit audio data using a conventional IP network audio transmitter, a protocol other than multicast cannot be used. This is because multiple receiving streams are required from a single transmitting host, and audio data signals are processed from multiple hosts with a single data transmission.

Publication of Patent Publication No. 10-2001-0067959 (published on July 13, 2001) Publication of Patent Publication No. 10-2006-0087434 (Published on August 2, 2006)

The problem to be solved by the present invention is to provide a broadband network-based audio streaming broadcasting system that supports time synchronization of audio packets by configuring it to support the same time synchronization of audio packets between an audio streaming server, an audio streaming client, and a control client, thereby enabling the use of a general external Internet network and a local broadband network together, and avoiding the speed and data errors inherent in the general Internet network while transmitting IP stream audio through multicast, thereby enabling the use of an IP broadcasting environment without building a separate local network, thereby efficiently operating audio data and AI-based voice recognition data, etc. even in a general broadband network.
In addition, the system eliminates the echo phenomenon of an audio signal by enabling audio data to be played at the same time in one or more clients connected to an audio streaming server from information for time synchronization included in the header of an audio packet transmitted from an audio streaming server, and provides a stable broadcasting environment through processing silence or drop when a packet is lost.

The present invention provides a broadband network-based audio streaming broadcasting system supporting time synchronization of audio packets, which is a means for solving the problem, comprising: an audio streaming server which selectively receives audio packets from a sound source provider and transmits them through a broadband network; at least one audio streaming client which is connected to the audio streaming server and receives audio packets transmitted from the audio streaming server through a broadband network; and a control client which controls the audio streaming server or the audio streaming client; wherein streaming of the audio packets received through TCP or WebSocket of the broadband network by at least one audio streaming client which is connected to the audio streaming server is controlled so as to achieve time synchronization.
The audio streaming server comprises: a reception selection unit for selectively receiving audio packets provided from an external source; a first buffer unit for temporarily storing audio packets selectively received by the reception selection unit; an encoding processing unit for encoding audio packets temporarily stored by the first buffer unit; a first synchronization processing unit for inserting time synchronization information for time synchronization into a header of an audio packet encoded by the encoding processing unit; and a streaming processing unit for transmitting the audio packet into which time synchronization information has been inserted by the first synchronization processing unit to the audio streaming client.

delete

In addition, the audio packets that are synchronized by the first synchronization processing unit are RAW PCM or compressed PCM using lossless compression (FLAC/ALAC) or encoded data using a lossy compression technique.

In addition, the audio streaming client includes: a second buffer unit for temporarily storing audio packets transmitted from the streaming processing unit after receiving them; a decoding processing unit for decoding audio packets temporarily stored by the second buffer unit; a second synchronization processing unit for extracting time synchronization information from audio packets decoded by the decoding processing unit and performing time synchronization; an error check unit for checking whether there is an error in the audio packets for which time synchronization is performed by the second synchronization processing unit and correcting an error portion if an error occurs; and a reproduction processing unit for reproducing the audio packets for which an error has been corrected by the error check unit at the same time through time synchronization.

Additionally, the error checking unit silences the error portion or drops the entire audio packet when an error occurs in the audio packet.

Additionally, the audio packets played from the playback processing unit are configured to be played based on time synchronization information inserted into the header of the audio packets from the first synchronization processing unit.

In addition, the audio packets played by the playback processing unit are transmitted to an external device through an amplifier unit after DAC playback, and the external device is a speaker.

In addition, when an underrun or overrun occurs in the audio streaming server or the audio streaming client, audio packets temporarily stored in the first buffer section and the second buffer section are removed and audio packets for the next streaming are managed to be stored within ±2ms.

In addition, the control client is configured to control the volume of the audio packet, select input audio, and provide status information and control of the audio streaming clients by connecting to the audio streaming client or controlling it through a WEB interface device.

In this way, the broadcasting system of the present invention is configured to support the same time synchronization for audio packets between the audio streaming server, the audio streaming client, and the control client, thereby enabling the use of a general external Internet network and a local broadband network together, and by transmitting IP stream audio through multicast, avoiding the speed and data errors inherent in the general Internet network, thereby enabling the use of an IP broadcasting environment without building a separate local network, so that the effect of efficiently operating audio data and AI-based voice recognition data, etc., can be expected even in a general broadband network.
In addition, it supports accurate playback time synchronization between one or more clients connected to an audio streaming server without multicasting via TCP or WebSocket protocol on a general Internet network, and provides a stable streaming environment by allowing one or more clients to play audio data at the same time through silence or drop processing in case of packet loss, so that an efficient IP-based broadcasting system can be configured on a general broadband network without separately constructing a local network or closed network.
The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

delete

FIG. 1 is a schematic block diagram of a broadband network-based audio streaming broadcasting system that supports time synchronization of audio packets according to an embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described with reference to the attached drawings.

FIG. 1 is a schematic block diagram of a broadband network-based audio streaming broadcasting system that supports time synchronization of audio packets according to an embodiment of the present invention.

Referring to the attached FIG. 1, a wideband network-based audio streaming broadcasting system supporting time synchronization of audio packets according to an embodiment of the present invention proposes a technical solution for operating audio data and/or AI-based voice recognition data, which are audio packets, through multicasting even in a WAN (WIDE AREA NETWORK) network, which is a general wideband network, in order to solve the problems of existing IP audio transmission technologies that must use a closed network and only a multicast method, and includes an audio streaming server (10), an audio streaming client (20), and a control client (30).

The above audio streaming server (10) selectively receives audio packets from a sound source provider (100) and then transmits them to one or more connected audio streaming clients (20) via a broadband network, and includes a reception selection unit (11), a first buffer unit (12), an encoding processing unit (13), a first synchronization processing unit (14), and a streaming processing unit (15).

The above-mentioned reception selection unit (11) is configured to selectively receive audio packets provided from the above-mentioned sound source providing unit (100).

The above first buffer unit (12) is configured to temporarily store audio packets selected and received by the reception selection unit (11) for streaming.

Here, when an underrun or overrun occurs in the audio streaming server (10) or the audio streaming client (20), the audio packets temporarily stored in the first buffer unit (12) are removed and audio packets for the next streaming are managed to be stored within ±2ms. This is to manage the transmission demonstration time according to the playback of the audio packets while ensuring that there are no problems with listening.

The above encoding processing unit (13) is configured to encode the audio packet temporarily stored by the first buffer unit (12).

The above first synchronization processing unit (14) is configured to insert time synchronization information for time synchronization into the header of the audio packet encoded by the encoding processing unit (13), and the audio packet subject to synchronization processing may be RAW PCM or compressed PCM using lossless compression (FLAC/ALAC) or encoded data using a lossy compression technique.

The above streaming processing unit (15) is configured to transmit the audio packet into which time synchronization information has been inserted by the first synchronization processing unit (14) to one or more connected audio streaming clients (20).

That is, when the streaming processing unit (15) transmits the audio packet to one or more of the audio streaming clients (20) via TCP or websocket of a broadband network, the audio packet can be played at the same time in synchronization in one or more of the audio streaming clients (20) connected to the audio streaming server (10).

The above audio streaming client (20) is connected to at least one audio streaming server (10) and receives audio packets transmitted from the audio streaming server (10) through a broadband network, and includes a second buffer unit (21), a decoding processing unit (22), a second synchronization processing unit (23), an error check unit (24), and a playback processing unit (25).

The above second buffer unit (21) is configured to temporarily store audio packets transmitted from the streaming processing unit (15) after receiving them.

Here, when an underrun or overrun occurs in the audio streaming server (10) or the audio streaming client (20), the audio packets temporarily stored in the second buffer unit (21) are removed and audio packets for the next streaming are managed to be stored within ±2ms. This is also to manage the transmission demonstration time according to the playback of the audio packets while ensuring that there are no problems with listening.

The above decoding processing unit (22) is configured to decode the audio packet temporarily stored by the second buffer unit (22).

The above second synchronization processing unit (23) is configured to extract time synchronization information from the audio packet decoded by the decoding processing unit (22) and perform time synchronization during playback.

That is, the second synchronization processing unit (23) extracts time synchronization information inserted into the header of the audio packet from the first synchronization processing unit (14), and accordingly, when one or more of the audio streaming clients (20) connected to the audio streaming server (10) receive the audio packet through TCP or websocket of a broadband network, the streaming of the received audio packet can be played at the same time through time synchronization.

The above error check unit (24) is configured to check for errors in the audio packets for which time synchronization is performed by the second synchronization processing unit (23), and to correct the error portion if an error occurs.

That is, when an error occurs in the audio packet, the error checking unit (24) silences the error portion or drops the entire audio packet.

The above playback processing unit (25) is configured to play back the audio packets whose errors have been corrected by the above error checking unit (24) at the same time through time synchronization.

That is, the audio packets played from the playback processing unit (25) can be configured to be played based on the time synchronization information inserted into the header of the audio packets from the first synchronization processing unit (14), and the audio packets played by the playback processing unit (25) can be transmitted to an external device (40), which is a speaker, through an amplifier unit (26) after DAC playback is performed, so that the playback can be performed.

The above control client (30) controls the audio streaming server (10) and/or the audio streaming client (20), and controls the audio streaming client (20) by accessing the audio streaming client (20) or controlling it through a web interface device to control the volume of the audio packet, select input audio, and provide status information and control one or more of the audio streaming clients (20).

In this way, a broadband network-based audio streaming broadcasting system that supports time synchronization of audio packets according to an embodiment of the present invention inserts time synchronization information into the header of the transmitted audio packet when transmitting an IP stream audio packet for multicasting for streaming from an audio streaming server (10) to one or more audio streaming clients (20) using a general broadband network, as shown in the attached FIG. 1.

Then, since audio packets are played at the same time that time synchronization is achieved through time synchronization information added to the header of the received audio packet in one or more of the above audio streaming clients (20), it is possible to build and use an IP broadcasting environment that manages transmission delay time without building a separate local network, avoiding the speed and data errors inherent in a typical Internet network, and this can solve the problem of existing IP audio transmission technology that only uses a closed network and must use only a multicasting method.

Although the technical idea of a broadband network-based audio streaming broadcasting system supporting time synchronization of audio packets of the present invention has been described above with the attached drawings, this is merely an example of the best embodiment of the present invention and does not limit the present invention.

Therefore, the present invention is not limited to the specific preferred embodiments described above, and anyone having ordinary skill in the art to which the present invention pertains may make various modifications without departing from the gist of the present invention claimed in the claims, and such modifications are within the scope of the claims.

10; Audio streaming server 11; Receiver selector
12; 1st buffer section 13; Encoding processing section
14; 1st synchronization processing unit 15; Streaming processing unit
20; Audio streaming client 21; Second buffer
22; Decoding processing unit 23; Second synchronization processing unit
24; Error check section 25; Playback processing section
26; Amplifier 30; Control Client
40; External device 100; Sound source provider

Claims (9)

In a broadband network-based audio streaming broadcasting system supporting time synchronization of audio packets,
An audio streaming server which selectively receives audio packets from a sound source provider through a receiving selection unit, temporarily stores the selectively received audio packets in a first buffer unit, encodes them through an encoding processing unit, and transmits them through a broadband network, inserts time synchronization information for time synchronization into the header of each audio packet during transmission, and performs data transmission for the same time synchronization between one or more connected audio streaming clients;
An audio streaming client connected to at least one of the above audio streaming servers via a broadband network, receiving audio packets and time synchronization information transmitted from the audio streaming server, and performing time synchronization to play the received audio packets at the same time by referring to the time synchronization information;
A control client for controlling the audio streaming server and one or more of the audio streaming clients, and capable of managing volume adjustment, input audio selection, and client status information;
The audio streaming server further includes a first synchronization processing unit that inserts the time synchronization information into the header of the audio packet encoded by the encoding processing unit, and a streaming processing unit that transmits the audio packet into which the time synchronization information has been inserted by the first synchronization processing unit to one or more of the audio streaming clients, wherein the transmitted audio packet is configured in one of RAW PCM or compressed PCM using lossless compression (FLAC/ALAC) or encoded data format using a lossy compression technique, and is configured to be transmitted via the TCP or WebSocket protocol of a broadband network.
One or more of the above audio streaming clients include a second synchronization processing unit which extracts time synchronization information from the audio packets that are decoded when decoded by the decoding processing unit after temporarily storing the received audio packets in the second buffer unit and then performs time synchronization based on the extracted time synchronization information, and an error check unit which controls to maintain time synchronization between clients by silencing an error section or dropping the corresponding packet when packet loss or error occurs.
A broadband network-based audio streaming broadcasting system supporting time synchronization of audio packets, characterized in that one or more of the above audio streaming clients play audio packets whose time synchronization has been completed through a playback processing unit, and the played audio packets are output to external speakers through an amplifier unit after DAC conversion. delete delete delete delete delete delete In paragraph 1,
A broadband network-based audio streaming broadcasting system that supports time synchronization of audio packets, characterized in that when an underrun or overrun occurs in the audio streaming server or one or more of the audio streaming clients, audio packets temporarily stored in the first buffer section and the second buffer section are removed and audio packets for the next streaming are managed so as to be stored within ±2 ms. In paragraph 1,
A broadband network-based audio streaming broadcasting system that supports time synchronization of audio packets, characterized in that the control client is configured to perform volume control of the audio packets, input audio selection, and provision of status information and control of audio streaming clients by connecting to the audio streaming client or controlling it through a web interface device.