GB2480818A

GB2480818A - Transmitting and receiving multimedia data using an in-band channel and an out-of-band control channel

Info

Publication number: GB2480818A
Application number: GB1009127A
Authority: GB
Inventors: James Chung-How; Andrew Nix; David Bull
Original assignee: GLOBAL PROVISION Ltd; PROVISION COMM TECHNOLOGIES Ltd; GI Provision Ltd
Current assignee: GLOBAL PROVISION LIMITED
Priority date: 2010-06-01
Filing date: 2010-06-01
Publication date: 2011-12-07
Also published as: GB201009127D0

Abstract

Multimedia data is transmitted from a transmitter to a receiver over an air interface having a predetermined bandwidth. A first data stream 11a-11n comprising multimedia data items and control data items is received by server 10. Control data items such as the transport stream and codec configuration data are extracted by extraction mechanism 18a-18n from the first data stream to produce a multimedia data stream 17a-17n and a control data stream 15a-15n. The multimedia data stream is transmitted to a receiver (26, figure 3) over a first channel and the control data stream is transmitted over a second channel different to the first channel, and the first channel is an in-band channel of the air interface, and the second channel is an outof-band channel.

Description

CONFIGURATION DATA TRANSMISSION MECHANISM

The present invention relates to a configuration data transmission mechanism. In particular, the present invention relates to a transmission mechanism for configuration data which enables reliable delivery across an unreliable network.

When digital media data, such as audio and/or video media data, are being transmitted from a server to an end user using a streaming application, the maximisation of quality of the media output presented to the end user is a high priority. However, when bandwidth is limited, it can be difficult to guarantee quality of service, particularly if the network over which the data are being transmitted is unreliable.

It is common for an MPEG-2 Transport Stream to be used as a digital container for transporting media data streams over network systems. An MPEG-2 Transport Stream consists of encapsulated Packetized Elementary Streams (PES) which contain the media data streams, Each Transport Stream is provided with data control mechanisms which ensure the audiovisual content of the data being transmitted by the Transport Stream is synchronised when presented on an end user's display device. The Transport Stream also contains configuration data, such as Program Specific Information (PSI), Program Association Table (PAT), Program Map Table (PMT), Conditional Access Table (CAT), and Network Infoniiation Table (NIT The video and audio data content within a Transport Stream is typically compressed using a high performance coder-decoder (codec), for example H.264, which is a standard for video compression, and advanced audio coding (AAC), a standard for audio compression. The codec reduces the amount of data that needs to be transmitted to a display device, therefore optimising bandwidth whilst maintaining the same quality of service. Configuration data, such as encoder settings that the decoder needs in order to successfully uncompress the data associated with the codec, must also be provided to the display device. The H.264 standard, for example, encapsulates this information within Sequence Parameter Sets which apply to the decoding of coded video sequences and Picture Parameter Sets which apply to the decoding of one or more individual pictures within the coded video sequence.

The codec configuration data change relatively infrequently, for any given media stream, In view of this, the H264 standard recommends that when the network over which the media data are being transmitted is reliable, the bandwidth can be preserved by sending the codec configuration data at an appropriate frequency, out-of-band, e.g. separately from the media content data. However, no mechanism is available for out-of-band transmission of codec configuration data in general. or in other less favourable circumstances such as when the network is unreliable. In addition, whilst the transport stream configuration data for any given media stream changes relatively infrequently, no mechanism is available for out-of-band transmission of transport stream configuration data.

An object of the present invention is to obviate or mitigate at least one of the aforementioned problems.

According to a first aspect of the invention there is provided a method of transmitting multimedia data from a transmitter to a receiver over an air interface having a predetermined bandwidth, the method comprising: receiving a first data stream comprising multimedia data items and control data items; extracting the control data items from the first data stream to produce a multimedia data stream and a control data stream; transmitting the multimedia data stream to a receivers over a first channel; and transmitting the control data stream to the receiver over a second channel different to the first channel, wherein the first channel is an in-band channel of the air interface, and the second channel is an out-of-band channel.

The transmission of control data using a different channel from that used for multimedia data transmission enables optimisation of bandwidth use whilst maintaining quality of transmitted data.

Conveniently the method may further comprise receiving a multimedia data stream on a first channel; receiving a control data stream on a second channel different to the first channel; and combining the received multimedia data stream and the received control data stream, to produce an output stream, wherein the first channel is an in-band channel of the air interface, and the second channel is an out-of-band channel.

The receiving of control data on a second channel, different from the first channel for the receiving of multimedia data enables minimisation of reception of unnecessarily repeated control data thus optimising bandwidth usage for the transmission of multimedia data. The second channel may be a session announcement protocol channel, Conveniently the control data stream includes transport stream data items which may include data items relating to one or more of program specific information, program association table information, program map table information, conditional access table information and network information table information.

These transport stream data items will change infrequently therefore by inclusion in the control data stream will minimise transmission of unnecessarily duplicated data.

Conveniently, the control data stream includes codec configuration data items which may relate to one or more of encoder settings information, sequence parameter sets information and picture parameter sets information. The codec configuration data will change infrequently therefore by inclusion in the control data stream will minimise transmission of unnecessarily duplicated data.

According to a second aspect of the invention there is provided apparatus for transmitting multimedia data to a receiver over an air interface having a predetermined bandwidth, the apparatus comprising an input unit operable to receive a first data stream comprising multimedia data items and control data items; an extraction unit operable to extract control data items from a first data stream to produce a multimedia data stream and a control data stream; a transmitter operable to transmit a multimedia data stream to a receiver over a first channel, and to transmit a control data stream to that receiver over a second channel different to the first channel, wherein the first channel is an in-band channel of the air interface, and the second channel is an out-of-band channel.

Apparatus which enables transmission of control data using a different channel from that used for multimedia data transmission enables optimisation of bandwidth use whilst maintaining quality of transmitted data.

According to a third aspect of the invention there is provided apparatus for receiving multimedia data from a transmitter over an air interface having a predetermined bandwidth, the apparatus comprising a receiver operable to receive a multimedia data stream on a first channel, and to receive a control data stream on a second channel different to the first channel; and a combining unit operable to combine a received multimedia data stream and a received control data stream, to produce an output stream, wherein the first channel is an in-band channel of the air interface, and the second channel is an out-of-band channel.

The provision of apparatus which receives control data on a second channel, different from the first channel for the receiving of multimedia data enables minimisation of reception of unnecessarily repeated control data thus optimising bandwidth usage for the transmission of multimedia data. The second channel may be a session announcement protocol channel.

Conveniently, the control data stream includes transport stream data items which may include data items relating to one or more of program specific information, program association table information, program map table information, conditional access table information and network information table information.

The control data stream may include codec configuration data items which may relate to one or more of encoder settings information, sequence parameter sets information and picture parameter sets information.

These and other aspects of the present invention will be more clearly understood from the following description and, by way of example only, and with reference to the following figures, in which: Figure 1 is a schematic diagram of a network in which a data transmission mechanism according to the present invention may be implemented; Figure 2 is a schematic diagram of a server having a server data transmission mechanism according to a first embodiment of the present invention; Figure 3 is a schematic diagram of a client having a client data transmission mechanism according to a first embodiment of the present invention operable to receive data from the server of Figure 2; Figure 4 is a schematic diagram of server having a server data transmission mechanism according to a second embodiment of the present invention; and Figure 5 is a schematic diagram of a client having a client data transmission mechanism according to a second embodiment of the present invention operable to receive data from the server of Figure 4.

With reference to Figure 1 there is shown a schematic diagram of a network 2 comprising a server 10 provided with transmitters 22, 24 and a plurality of clients 30a-30n, each provided with receivers 26, 28.

With reference to Figure 2 there is shown in more detail server 10 which is provided with an in-band transmitter 22 and out-of band transmitter 24. The server 10 comprises encoders, in this case media encoders, 12a-12n, Transport Stream Multiplexers (TS Mux) 14a-14n which in this case are MPEG2 TS Mux, and server data transmission mechanism 16. Within the server data transmission mechanism there is provided extraction mechanism 18a-1 8n and data format mechanism 20.

The server 10, in this case, receives multiple input media streams 1 la-I in with each media stream ha-un provided to an audio encoder l2aa-h2na and a video encoder l2av-l2nv respectively. The data output from encoders l2aa,l2av-l2na,l2nv is input to corresponding MPEG2 TS Mux 14a-14n respectively. Each MPEG2 TS Mux 14a-14n combines the data from the multiple encoders l2aa,l2av -l2na,l2nv into corresponding multiplexed MREG2 TS data streams 1 3a-13n which are input to corresponding extraction mechanisms 1 8a-1 8n within server data transmission mechanism 16. Each extraction mechanism 18a-18n parses the generated transport stream 13a-13n and removes both the transport stream and codec configuration data which are provided as a data stream 15a-1 Sn to data format mechanism 20 wherein the transport stream and codec configuration data are packetized and suitably formatted for provision to out-of band transmitter 24 for transmission as an out-of-band configuration data stream 25. The multimedia data stream 1 7a-1 7n output from extraction mechanism 1 8a-1 8n is provided to in-band transmitter 22 for transmission in-band as an in-band media data stream 26. The network 2 over which server 10 transmits may be reliable or unreliable.

A receiving client 30 is shown in Figure 3, and may be any one of receiving clients 30a-30n.

Client 30 is provided with an in-band receiver 26 and an out-of-band receiver 28. The client 30 comprises client data transmission mechanism 32, Transport Stream Demultiplexer (TS Demux) 38 and an audio decoder 40a and a video decoder 40v. The client data transmission mechanism 32 comprises a data format mechanism 34 and an insertion mechanism 36. The client 30 receives both the in-band media data transport stream 17 and the out-of-band transport stream and codec configuration data 29 from in-band receiver 26 and out-of-band receiver 28 respectively. The out-of band configuration data 29 are provided to data format mechanism 34 where the out-of-band configuration data 29 are formatted into the original transport stream and codec data form 15. The insertion mechanism 36 receives the transport stream and codec configuration data 15 from the data format mechanism 34 and also receives media data transport stream 17 from the in-band receiver 26. The insertion mechanism 36 re-inserts the transport stream and codec configuration data 15 into transport stream 17. The output data of the client data transmission mechanism 32 are functionally identical data stream 13 to that going into the server data transmission mechanism 16. This consistency of data ensures that generic standards compliant codec can be used. The data stream 13 is then provided to TS Demux 38 which in this case is a MPEG2 TS Demux which demultiplexes the data stream before providing it to decoders 40a, 40v for decoding and provision to a display device (not shown).

As transport stream configuration data and codec configuration data 15 extracted by extraction mechanism 1 8a-1 8n change relatively infrequently for any given video and audio stream, bandwidth within the network can be preserved for transmission of the transport stream data 17 by sending the configuration data 15, at an appropriate frequency, out-of-band. The preserved bandwidth can then be optimised to maintain quality of service in the provision of the video and audio stream.

With reference to Figure 4, there is illustrated a second embodiment of a server 110 provided with a transmitter 123 suitable for transmission over an unreliable network. The server 110 comprises encoders, in this case media encoders, 112a-112n, Transport Stream Multiplexers (TS Mux) 1 14a-1 14n which in this case are MPEG2 TS Mux, and server data transmission mechanism 116. Within the server data transmission mechanism there is provided extraction mechanism 1 18a-1 18n and an announcement generator mechanism 119. The announcement generator mechanism 119 is in this case a Session Announcement Protocol announcement generator mechanism.

The server 110, in this case, receives multiple input media streams lila-il in with each media stream lila-i un provided to an audio encoder 1 l2aa-ll2na and a video encoder 1 l2av-1 l2nv respectively. The data output from encoders 1 i2aa, 1 l2av-1 i2na, 1 i2nv is input to a corresponding MPEG2 TS Mux 1 14a-l 14n respectively. Each MPEG2 TS Mux 1 14a-il4n combines the data from the multiple encoders 1 12aa, 1 l2av -1 i2na, 1 l2nv into corresponding multiplexed MREG2 TS data streams ii 3a-11 3n which are input into corresponding extraction mechanism ii 8a-ii 8n within server data transmission mechanism 116 Each extraction mechanism ii 8a-11 8n parses the multiplexed stream 11 3a-ii 3n and removes both the transport stream and codec configuration data which is provided as a data stream ii 5a-1 15n to announcement generator mechanism 119 wherein the transport stream and codec configuration data is packetized and suitably formatted and with identifiers for the available transport streams 1 17a-1 l7n to form an announcement message data stream 121 for provision to transmitter 123 for transmission an over an unreliable network. The multimedia data transport streams 11 7a-ii 7n output from extraction mechanisms 11 8a-ii 8n are also provided to transmitter 123 for transmission over an unreliable network. Announcement messages 121 are sent by transmitter 123 at predetermined bit rate allocated for sending announcement messages which is known as an announcement interval.

A receiving client 130 is shown in Figure 4, is operable to receive transmissions from server 110.

Client 130 is provided with a receiver 127 for receiving transmissions from server 110 transmitted over an unreliable network. The client 130 comprises client data transmission mechanism i32, Transport Stream Demultiplexer (TS Demux) 138, audio decoders 140a and video decoder 140v. The client data transmission mechanism 132 comprises an announcement receiver mechanism 133, a stream selector mechanism 135 and an insertion mechanism 136.

The announcement receiver mechanism 133 is, in this case, an SAP Announcement receiver mechanism.

In use, the client 130 receives the data transmitted over an unreliable network at receiver 127.

Receiver 127 listens for announcement messages 121. The configuration data 11 5a-11 Sn and identifiers for the available streams 117 a-li 7n are included in the announcement messages 121 which are received and forwarded to announcement receiver 133. Upon successfully receiving the announcement message 121, announcement receiver 133 extracts the configuration data 115 which is formatted appropriated and provided to the insertion mechanism 136. The announcement receiver 133 also extracts the identifiers for the available transport streams 11 7a- 1 17n and provides this data to stream selector mechanism 135. The stream selector mechanism selects the required transport stream and provides this to insertion mechanism 136. The insertion mechanism 136 re-inserts the transport stream and codec configuration data 115 into appropriate transport stream 117. The output data of the client data transmission mechanism 132 are functionally identical data stream 113 to that going into the server data transmission mechanism 116. This consistency of data ensures that generic standards compliant codec can be used. The data stream 113 is then provided to TS Demux 138 which in this case is a MPEG2 TS Demux which dernultiplexes the data stream before providing it to decoders 140a, 140v for decoding and provision to a display device (not shown).

As the client 130 must receive the announcement messages 121 in order to know what data streams 117a-1 17n are available and successful reception of an announcement message 121 means that the client 130 has also received the parameter information within the configuration data 1 15a-115n which provides the transmission mechanism with a pseudo-reliable characteristic of delivering the configuration data 1 17a-1 17n.

The inclusion of the configuration data 1 iSa-I 15n within an out-of-band stream broadcast service, such as in this example, Session Announcement Protocol (SAP) within a multicast environment simultaneously provides the client with available transport data 117a-1 17n and the corresponding transport stream and codec configuration data 11 5a-ii Sn required to deliver each transport data stream 1 17a-i 17n efficiently.

As transport stream configuration data and codec configuration data 1S extracted by extraction mechanism 1 8a-1 8n change relatively infrequently for any given video and audio stream, bandwidth within the network can be preserved for transmission of the transport stream data 17 by sending the configuration data 15, at an appropriate frequency, out-of-band. The preserved bandwidth can then be optimised to maintain quality of service in the provision of the video and audio stream.

An example of a situation in which the implementation of the transmission mechanism of the server-client system of Figure 4 and 5 is applicable is the multicast delivery of media streams to large numbers of receivers over an unreliable network, e.g. WiFi 8O2.llg. In such a situation, bandwidth available for transmission between the server and clients is very limited; hence reliably sending configuration data out-of-band is desirable.

For each transport data stream 117 a-i 17n, transmitted by the server 110 in Figure 4 above, the SAP announcement generatorll9 produces an Announcement message 121a-121n. The payload of each announcement message 12 la-12 in uses a Session Description Protocol to describe the parameters of the respective transport data stream ii7a-i i7n. An example format for the SAP announcement message 121, when using H264 and AAC to encode the media stream is: vO o=-<stream ID> <version> IN 1P4 <server IP Address> s=<Stream Name from UI> t=o 0 c=IN 1P4 <multicast address>/<ttl> m=data <port> UDP a=X-H264 <H264 parameters> a=X-AAC <AAC parameters> a=X-TS <TS Parameters> stream ID = a unique id number for the stream version = 0 and increments each time the stream session is updated port = the UDP port this multioast stream is sent on tti = muiticast time to live Where X-H264 -Contains the base64 encoded SPS and PPS strings, an example of this is: a=X-H264 profile-level-id=42E00D; sprop-parameter-sets=ZOLgDZWgUGIn/ 8AAQABEAAAPoAABhqGDAASTwBJWrgAC, aM44qA==; parameter-sets=ZOLqDZNqUGfn/ 8AAQABEAAAPoAAEhqGDAASTwBJNrqAC, aM4 4gA==; packetization-mode=l X-AAC -Contains the base64 encoded AAC strings, an example of this is a=X-AAC profile-level-id=l5; config=ll9O; streamtype=5; mode=AAC-hbr; SizeLength==13; IndexLength=3; IndexDeltaLength=3 and X-TS -Contains the base64 encoded PAT and PMT strings, example of this is: a=X-TS PAT=DZ1igUG; PMTPID=23; PMT=DAASTWBJW AT = base64 encoded Program Allocation Table (specifying a single Program Map Table present on PIP <PMTPID>) PMTPIP = the PID to send the Program Map Table on PMT base64 encoded Program Map Table

Claims

CLAIMS1. A method of transmitting multimedia data from a transmitter to a receiver over an air interface having a predetermined bandwidth, the method comprising: receiving a first data stream comprising multimedia data items and control data items; extracting the control data items from the first data stream to produce a multimedia data stream and a control data stream; transmitting the multimedia data stream to a receivers over a first channel; and transmitting the control data stream to the receiver over a second channel different to the first channel, wherein the first channel is an in-band channel of the air interface, and the second channel is an out-of-band channel.
2. A method of receiving multimedia data from a transmitter over an air interface having a predetermined bandwidth, the method comprising: receiving a multimedia data stream on a first channel; receiving a control data stream on a second channel different to the first channel; and combining the received multimedia data stream and the received control data stream, to produce an output stream, wherein the first channel is an in-band channel of the air interface, and the second channel is an out-of-band channel.
3. A method as claimed in claim 1 or 2, wherein the second channel is a session announcement protocol channel.
4. A method as claimed in claim 1, 2 or 3, wherein the control data stream includes transport stream data items.
5. A method as claimed in claim 4, wherein the transport steam data items include data items relating to one or more of program specific information, program association table information, program map table information, conditional access table information and network information table information.
6. A method as claimed in any one of the preceding claims, wherein the control data stream includes codec configuration data items.
7. A method as claimed in claim 6, wherein the codec configuration data items relate to one or more of encoder settings information, sequence parameter sets information and picture parameter sets information.
8. Apparatus for transmitting multimedia data to a receiver over an air interface having a predetermined banthvidth, the apparatus comprising: an input unit operable to receive a first data stream comprising multimedia data items and control data items: an extraction unit operable to extract control data items from a first data stream to produce a nmltimedia data stream and a control data stream; a transmitter operable to transmit a multimedia data stream to a receiver over a first channel, and to transmit a control data stream to that receiver over a second channel different to the first channel, wherein the first channel is an in-band channel of the air interface, and the second channel is an out-of-band channel.
Apparatus for receiving multimedia data from a transmitter over an air interface having a predetermined bandwidth, the apparatus comprising: a receiver operable to receive a multimedia data stream on a first channel, and to receive a control data stream on a second channel different to the first channel; and a combining unit operable to combine a received multimedia data stream and a received control data stream, to produce an output stream, wherein the first channel is an in-band channel of the air interface, and the second channel is an out-of-band channel.
10. Apparatus as claimed in claim 8 or 9, wherein the second channel is a session announcement protocol channel.
11. Apparatus as claimed in claim 8, 9 or 10, wherein the control data stream includes transport stream data items.
12. Apparatus as claimed in claim ii, wherein the transport steam data items include data items relating to one or more of program specific information, program association table information, program map table information, conditional access table information and network information table information.
13. Apparatus as claimed in any one of claims 8 to 12, wherein the control data stream includes codec configuration data items.
14. Apparatus as claimed in claim 13, wherein the codec configuration data items relate to one or more of encoder settings information, sequence parameter sets information and picture parameter sets information.