CN107005497A - Method and apparatus for signaling and operation of low-latency consumption of media data in MMT - Google Patents

Abstract

Methods and apparatus are provided for wireless communication between at least one base station and a user equipment, the user equipment comprising a transceiver and processing circuitry. The transceiver is configured to receive a packet including a control message having a header and a payload related to a segment of multimedia content. The header includes a message identifier indicating whether the control message is a low latency consumption message, a length of the controller message, and a version of the control message. The processing circuit is configured to determine whether the control message is a low latency consumption message based on the message identifier. The processing circuit is further configured to configure the packet based on the payload and the control message prior to receiving a header of the segment of the multimedia content in response to the control message being a low latency consumption message.

Description

Method and apparatus for signaling and operation of low-latency consumption of media data in MMT

technical field

This application relates generally to media data delivery in transmission systems, and more particularly to signaling of Moving Picture Experts Group (MPEG) Media Transport (MMT) Protocol (MMTP) decapsulation buffers and operation.

Background technique

MMT is a digital container standard or format that specifies techniques for the delivery of encoded media data for multimedia services over a heterogeneous IP network environment. The delivered encoded media data includes both: audio-visual media data that requires simultaneous decoding and presentation of specific units of data at specified times, i.e. timed data, and other types of data that are based on the context of a service or The user's interactions are decoded and presented at arbitrary times, ie, non-timed data.

MMT is designed on the assumption that encoded media will be delivered over a packet-based delivery network using Internet Protocol (IP) such as Real-time Transport Protocol (RTP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), etc. data. MMTs are also designed taking into account the characteristics of different delivery environments.

Contents of the invention

technical problem

This disclosure provides signaling and operation of MMTP decapsulation buffers.

Technical solutions

In a first embodiment, an apparatus for wireless communication between at least one base station and a user equipment comprising a transceiver and processing circuitry is provided. The transceiver is operable to communicate with the at least one base station by transmitting radio frequency signals to the at least one base station and by receiving radio frequency signals from the at least one base station. The transceiver is configured to receive a packet comprising a control message having a header and a payload related to a segment of the multimedia content. The header includes a message identifier indicating whether the control message is a low-latency consumption message, the length of the control message, and the version of the control message. The processing circuit is configured to determine whether the control message is a low-latency consumption message based on the message identifier. The processing circuit is further configured to configure the packet based on the payload and the control message prior to receiving the header of the segment of the multimedia content in response to the control message being a low-latency consumption message.

In a second embodiment, a method for wireless communication between at least one base station and user equipment is provided. The method includes receiving a packet including a control message having a header and a payload related to a segment of multimedia content. The header includes a message identifier indicating whether the control message is a low-latency consumption message, the length of the control message, and the version of the control message. The method also includes determining whether the control message is a low-latency consumption message based on the message identifier. The method also includes configuring the packet based on the payload and the control message prior to receiving the header of the segment of the multimedia content in response to the control message being a low-latency consumption message.

In a third embodiment, a system for wireless communication between at least one base station and user equipment comprising a transceiver and processing circuitry is provided. The transceiver is operable to communicate with the at least one base station by transmitting radio frequency signals to the at least one base station and by receiving radio frequency signals from the at least one base station. The transceiver is configured to receive a packet comprising a control message having a header and a payload related to a segment of the multimedia content. The header includes a message identifier indicating whether the control message is a low-latency consumption message, the length of the controller message, and the version of the control message. The processing circuit is configured to determine whether the control message is a low-latency consumption message based on the message identifier. The processing circuit is further configured to configure the packet based on the payload and the control message prior to receiving the header of the segment of the multimedia content in response to the control message being a low-latency consumption message.

Advantageous effect of the present invention

Other technical features will be readily apparent to those skilled in the art from the drawings, specification and claims.

Description of drawings

For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which:

FIG. 1 illustrates an example communication system in which various embodiments of the present disclosure may be implemented;

Figures 2 and 3 illustrate example devices in a communication system according to the present disclosure;

4 shows an example block diagram of MMTP input/output in an MMTP data transmission environment according to the present disclosure;

5 shows a block diagram of an example receiver buffer model for simulating receiver behavior at the receiver side and for estimating buffer delay and size requirements according to the present disclosure;

FIG. 6 illustrates a process for managing received data by a client device according to the present disclosure; and

FIG. 7 illustrates a process for indicating a presentation time by a server according to the present disclosure.

detailed description

Before proceeding to the detailed description that follows, it may be advantageous to set forth definitions for certain words and phrases used throughout this patent document. The term "couple" and its derivatives refer to any direct or indirect communication between two or more elements, whether or not those elements are in physical contact with each other. The terms "transmit", "receive" and "communicate" and their derivatives encompass both direct and indirect communication. The terms "include" and "comprises" and their derivatives mean including but not limited to. The term "or" is inclusive, meaning and/or. The phrase "associated with" and its derivatives mean including, included in, interconnected with, comprising, contained in, connected to or connected with, coupled to or coupled with, with ...can communicate with, cooperate with, juxtapose, or be close to, be bound to or be bound to, have, have the properties of, have a relationship with, or have a relationship with, etc. The term "controller" means any device, system or part thereof that controls at least one operation. Such controllers may be implemented in hardware or a combination of hardware and software and/or firmware. The functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. The phrase "at least one of" when used with a list of items means that different combinations of one or more of the listed items may be used and that only one of the listed items may be required. For example, "at least one of A, B, and C" includes any of the following combinations: A, B, C, "A and B," "A and C," "B and C," and "A and B and C".

In addition, various functions described below can be implemented or supported by one or more computer programs, each of which is formed of computer-readable program code and embodied in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, sets of instructions, procedures, functions, objects, classes, instances, related data or part thereof. The phrase "computer readable program code" includes any type of computer code, including source code, object code, and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as read-only memory (ROM), random-access memory (RAM), hard drive, compact disc (CD), digital video disc (DVD) or Any other type of memory. A "non-transitory" computer readable medium excludes wired, wireless, optical, or other communication links that transmit transient electrical or other signals. Non-transitory computer readable media include media in which data can be permanently stored as well as media in which data can be stored and later rewritten, such as rewritable optical discs or erasable memory devices.

Definitions for other certain words and phrases are provided throughout this patent document. Those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior, as well as future uses of such defined words and phrases.

1 through 7, discussed below, and the various embodiments used to describe the principles of the disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged wireless communication system.

MMT coding and media delivery is discussed in the following document and standard description: ISO/IEC JTC 1/SC29/WG11, High efficiency coding and media delivery in heterogeneous environments-Part 1: MPEG Media Transport (MMT), July 2012 (ISO/ IEC JTC 1/SC29/WG11, Efficient Coding and Media Delivery in Heterogeneous Environments - Part 1: MPEG Media Transport (MMT), July 2012), which is hereby incorporated as if fully set forth herein public. For the efficient and efficient delivery of encoded media data over heterogeneous IP network environments, MMT provides: logical models for building content composed of various components for mash-up applications; Information for delivering layered processing of encoded media data such as packetization and matching; packetization method and packet structure for delivery of information that is agnostic to the particular type of media or encoding method used over TCP or UDP Media content, including hybrid delivery; the format of signaling messages used to manage the presentation and delivery of media content; and the format of messages to be exchanged across layers to facilitate cross-layer communication.

MMT defines three functional areas, including encapsulation, delivery, and signaling. The encapsulation functional area defines the logical structure of the media content, the MMT package, and the format of the data units to be processed by the MMT compliant entity. The MMT package specifies the composition of media content and the relationship between media content, so as to provide information required for adaptive delivery. The format of the data unit is defined to encapsulate the encoded media to be stored or to be transported as the payload of the delivery protocol, and to be easily converted between storage and transport. The delivery functional area defines the application layer protocol and the format of the payload. Compared to traditional application layer protocols for the delivery of multimedia, the application layer protocol provides enhanced features for the delivery of MMT packets, including multiplexing. A payload format is defined to convey encoded media data that is agnostic to a particular media type or encoding method. The signaling functional area defines the format of messages used to manage the delivery and consumption of MMT packets. Messages for consumption management are used to signal the structure of the MMT packet, and messages for delivery management are used to signal the structure of the payload format and the configuration of the protocol.

MMT defines a new framework for the delivery of time-continuous multimedia such as audio, video, and other static content such as widgets, files, and the like. MMT specifies a protocol (ie, MMTP) for delivering MMT packets to receiving entities. MMTP signals the delivery time of MMTP packets as part of the protocol header. This time enables the receiving entity to perform dejittering by checking the transmission time and reception time of each incoming MMT packet.

Embodiments of the present disclosure recognize and take into account that the MMT specification enables low-latency delivery of ISOBMFF but does not provide appropriate information for low-latency consumption.

FIG. 1 illustrates an example communication system 100 in which various embodiments of the present disclosure may be implemented. The embodiment of communication system 100 shown in FIG. 1 is for illustration only. Other embodiments of the communication system 100 may be used without departing from the scope of this disclosure.

As shown in FIG. 1 , system 100 includes network 102 that facilitates communication between various components in system 100 . For example, network 102 may communicate Internet Protocol (IP) packets, frame relay frames, asynchronous transfer mode (ATM) cells, or other information between network addresses. Network 102 may also be a heterogeneous network including broadcast networks such as cable and satellite communication links. Network 102 may include one or more local area networks (LANs): metropolitan area networks (MANs); wide area networks (WANs); all or part of a global network such as the Internet; or any other communication system at one or more locations.

Network 102 facilitates communications between at least one server 104 and various client devices 106-115. Each server 104 includes any suitable computing or processing device that can provide computing services to one or more client devices. Each server 104 may include, for example, one or more processing devices, one or more memories to store instructions and data, and one or more network interfaces to facilitate communications over network 102 .

Each client device 106 - 115 represents any suitable computing or processing device that interacts with at least one server or other computing device(s) via network 102 . In this example, the client devices 106-115 include a desktop computer 106, a mobile phone or smart phone 108, a personal digital assistant (PDA) 110, a laptop computer 112, a tablet computer 114, and a set-top box and/or television 115 . However, any other or additional client devices may be used in communication system 100 .

In this example, some of the client devices 108 - 114 communicate with the network 102 indirectly. For example, client devices 108-110 communicate via one or more base stations 116, such as cellular base stations or eNodeBs. Additionally, client devices 112-115 communicate via one or more wireless access points 118, such as IEEE 802.11 wireless access points. Note that these are for illustration only, and that each client device may communicate with network 102 directly or indirectly via any suitable intermediate device(s) or network(s).

As described in more detail below, network 102 uses MMTP to facilitate communication of media data, such as images, video, and/or audio, for example, from server 104 to client devices 106-115. Assuming that the MMT is also designed in consideration of the characteristics of different delivery environments, the server 104 can broadcast or stream media data to the client devices 106 to 115 via the network using MMTP. Additionally, the server 104 may provide buffer removal mode signaling via messages together with or separately from the media data to instruct MMTP decapsulation buffers to operate and to manage MMTP decapsulation buffers.

Although FIG. 1 shows one example of a communication system 100, various changes may be made to FIG. 1 . For example, system 100 may include any number of each component in any suitable arrangement. In general, computing and communication systems have a wide variety of configurations, and Figure 1 does not limit the scope of the present disclosure to any particular configuration. Although FIG. 1 illustrates one operating environment in which the various features disclosed in this patent document may be used, the features may be used in any other suitable system.

2 and 3 illustrate example devices in a computing system according to the present disclosure. Specifically, FIG. 2 shows an example server 200 and FIG. 3 shows an example client device 300 . Server 200 may represent server 104 in FIG. 1 , and client device 300 may represent one or more of client devices 106 through 115 in FIG. 1 .

As shown in FIG. 2, the server 200 includes a bus system 205 that supports communication between at least one controller 210, at least one storage device 215, at least one communication unit 220, and at least one input/output (I/O) unit 225. Communication.

Controller 210 executes instructions that may be loaded into memory 230 . Controller 210 may include any suitable number and types of processors or other devices in any suitable arrangement. Example types of controllers 210 include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuits.

Memory 230 and persistent storage 235 are examples of storage devices 215, which represent any structure (one or multiple). Memory 230 may represent random access memory or any other suitable volatile or non-volatile storage device(s). Persistent storage 235 may include one or more components or devices that support long-term storage of data, such as read-only memory, hard drives, flash memory, or optical disks.

Communications unit 220 supports communications with other systems or devices. For example, communications unit 220 may include a network interface card or a wireless transceiver that facilitates communications over network 102 . Communications unit 220 may support communications over any suitable physical or wireless communications link(s).

The I/O unit 225 allows input and output of data. For example, I/O unit 225 may provide connections for user input through a keyboard, mouse, keypad, touch screen, or other suitable input device. I/O unit 225 may also send output to a display, printer, or other suitable output device.

Note that while FIG. 2 is described as representing server 104 of FIG. 1 , the same or similar structures may be used in one or more of client devices 106-115. For example, a laptop or desktop computer may have the same or similar structure as that shown in FIG. 2 .

As described in more detail below, the server 200 sends media data and/or buffer removal mode signaling via messages together with or separately from the media data to instruct the MMTP decapsulation buffer to operate and to manage the MMTP decapsulation buffer . In one example, the server 200 may be a broadcasting entity for broadcasting media data via an IP network.

As shown in FIG. 3 , client device 300 includes antenna 305 , transceiver 310 , transmit (TX) processing circuitry 315 , microphone 320 , and receive (RX) processing circuitry 325 . Client device 300 also includes speaker 330 , controller 340 , input/output (I/O) interface (IF) 345 , keypad 350 , display 355 , and memory 360 . The memory 360 includes an operating system (OS) 361 and one or more applications 363 .

Transceiver 310 receives from antenna 305 an incoming RF signal transmitted by another component in the system. Transceiver 310 downconverts the incoming RF signal to generate an intermediate frequency (IF) or baseband signal. The IF or baseband signal is sent to RX processing circuitry 325, which filters, decodes, and/or digitizes the baseband or IF signal to generate a processed baseband signal. RX processing circuitry 325 transmits the processed baseband signal to speaker 330 (such as for voice data) or to controller 340 for further processing (such as for web browsing data).

TX processing circuitry 315 receives analog or digital voice data from microphone 320 , or other outgoing baseband data (such as web data, email, or interactive video game data) from controller 340 . TX processing circuitry 315 encodes, multiplexes, and/or digitizes outgoing baseband data to generate processed baseband or IF signals. Transceiver 310 receives outgoing processed baseband or IF signals from TX processing circuitry 315 and upconverts the baseband or IF signals to RF signals for transmission via antenna 305 .

Controller 340 may include one or more processors or other processing devices, and runs a basic operating system 361 stored in memory 360 in order to control the overall operation of client device 300 . For example, controller 340 may control reception of forward channel signals and transmission of reverse channel signals by transceiver 310, RX processing circuit 325, and TX processing circuit 315 according to well-known principles. In some embodiments, controller 340 includes at least one microprocessor or microcontroller.

Controller 340 is also capable of executing other processes and programs that reside in memory 360 . Controller 340 may move data into or out of memory 360 as required by the running process. In some embodiments, the controller 340 is configured to run an application 363 based on the operating system 361 or in response to a signal received from an external device or a carrier. Controller 340 is also coupled to I/O interface 345, which provides client device 300 with the ability to connect to other devices, such as laptop computers and handheld computers. I/O interface 345 is the communication path between these accessories and controller 340 .

Controller 340 is also coupled to keypad 350 and display 355 . An operator of client device 300 may use keypad 350 to enter data into client device 300 . Display 355 may be a liquid crystal display or other display capable of rendering text such as from a website and/or at least limited pictures.

Memory 360 is coupled to controller 340 . A portion of memory 360 may include random access memory (RAM), and another portion of memory 360 may include flash memory or other read-only memory (ROM).

As described in more detail below, client device 300 receives a low latency consumption (LDC) message. For example, client device 300 may receive and process media data according to LDC. In one example, client device 300 may be a mobile device that receives broadcast media data via an IP network.

Although FIGS. 2 and 3 illustrate examples of devices in a computing system, various changes may be made to FIGS. 2 and 3 . For example, various components in FIGS. 2 and 3 may be combined, further subdivided, or omitted, and additional components may be added according to specific needs. As a specific example, controller 340 may be divided into multiple processors, such as one or more central processing units (CPUs) and one or more graphics processing units (GPUs). Additionally, while FIG. 3 shows client device 300 configured as a mobile phone or smartphone, the client device may be configured to operate as other types of mobile or stationary devices, including, for example, but not limited to, set-top boxes, televisions, and media streaming device. Additionally, as with computing and communication networks, client devices and servers can have a wide variety of configurations, and FIGS. 2 and 3 do not limit the present disclosure to any particular client device or server.

FIG. 4 shows an example block diagram of MMTP input/output in an MMTP data transfer environment 400 according to the present disclosure. In this illustrative example, a sending entity 405—for example, a server such as server 200 in FIG. 2—to a receiving entity 410—for example, a client such as client device 300 in FIG. End device - send media data. The media data 415 is processed at the sending entity 405 according to MMTP. For example, the sending entity 405 may perform encapsulation, encoding, delivery, and signaling for media data as an MMT Processing Unit (MPU) or an MMT Fragmentation Unit (MFU) (eg, segmentation of an MPU). The processed media data is then sent (eg, as packets) to the receiving entity 410 for processing (eg, decapsulation, decoding, etc.) according to MMTP. Then, the media data processed at the receiving entity 410 is passed up to upper layer programming (e.g., application layer programs such as media layer) as MPU and/or MFU for delivery of media data after completion of visual and/or auditory Presented to the user on the display device.

5 shows a block diagram of an example receiver buffer model 500 for simulating receiver behavior at the receiver side and for estimating buffer delay and size requirements according to the present disclosure. In various embodiments of the present disclosure, a sending entity 405, such as a media delivery server (or other MMT-aware node), calculates, determines and/or identifies fixed end-to-end delay. For example, sending entity 405 may utilize schema 500 to determine the effect of media data processing performed on a packet stream in a receiver of receiving entity 410 with respect to reception constraints. For example, the sending entity 405 can use the model to determine the required buffering delay and the required buffer size and communicate this information to the entity receiving the media data.

In this illustrative example, FEC decoding buffer 505 is a model for estimating delay and/or buffer size requirements associated with FEC decoding. FEC decoding is typical for many applications where low layer delivery may not be sufficient for recovering from channel errors or when network congestion may cause packet drops or excessive delays. To perform FEC decoding, the receiving entity 410 uses a buffer where incoming packets are stored until sufficient source ("S") and repair data ("P" parity data) are available to perform FEC decoding.

In this illustrative embodiment, sending entity 405 uses a model of FEC decoding buffer 505 to determine actions to be taken by receiving entity 410 with respect to FEC decoding for estimating delays associated with FEC decoding. In other words, the sending entity 405 uses the model of the FEC decoding buffer 505 to predict the actions taken by the receiving entity 410 to estimate the FEC decoding delay. This modeling of the FEC decoding buffer 505 by the sending entity 405 starts with the FEC decoding buffer 505 being assumed to be initially empty. Next, for each incoming packet i with transmission timestamp ts, if buffer_occupancy+packet_size<max_buffer_size, the receiving entity 410 buffers the packet i using the FEC decoding buffer 505 . Otherwise, the receiving entity 410 discards packet i because it does not conform to the buffer model. The receiving entity 410 then determines whether FEC is applied to packet i. If FEC is applied to packet i, the receiving entity 410 determines the source block j to which packet i belongs, determines the insertion time t of the first packet of source block j, at time t+FEC_buffer_time all packets of source block j (if required , then after FEC correction) moves to the de-jitter filter, and discards the repair packet. The sending entity 405 uses FEC_buffer_time as the buffer time required for FEC decoding from the reception of the first packet of the source block until the FEC decoding is attempted. This time is usually calculated based on the FEC block size.

De-jitter buffer 510 is a model used by a sending entity to estimate delay and/or buffer size requirements associated with de-jittering of a packet, ie, removal of delay jitter of a packet. The de-jitter buffer essentially ensures that the MMP packets experience a fixed transfer delay, assumed to be the maximum transfer delay, from the source to the output of the MMTP protocol stack. The receiving entity 410 may discard data units that experience a transfer delay greater than the maximum transfer delay because they are very late.

This modeling of the de-jitter buffer 510 by the sending entity 405 starts with the de-jitter buffer being assumed to be initially empty. Receiving entity 410 then inserts the MMTP packet into de-jitter buffer 510 as the packet arrives. The receiving entity 410 then removes the MMTP packet at time ts+Δ, where ts is the delivery timestamp of the MMTP packet and Δ is the fixed end-to-end delay signaled for the media data. After dejitter is applied, all MMTP packets that arrive correctly (or are recovered by FEC/retransmission) will experience the same end-to-end delay.

MMTP Decapsulation Buffer 515 is a model used by the sending entity to estimate the delay and/or buffer size requirements associated with MMTP processing before passing the output to upper layers. The output of the MMTP processor can be an MFU payload (in low-latency operation), a complete movie segment, or a complete MPU. Depending on its size, an MPU can be partitioned into smaller groups, or can be aggregated into larger groups. Then, as part of the MMTP processing, decapsulation (removal of MMTP packet and payload headers) and any required de-segmentation/de-aggregation are performed on the packet. This process may require some buffering delay - known as decapsulation delay - to perform assembly while the MPU is split into multiple MMTP packets. However, in this illustrative embodiment, the decapsulation delay may not be considered part of the fixed end-to-end delay, and the MPU may be guaranteed to be used by the encoding media layer by segmenting the MPU as a whole into multiple MMTP packets. Availability of consumed MPU independent of decapsulation latency. Although used as a model by the sending entity 405 , each of the buffers 505 , 510 and 515 may be implemented in a memory of the receiving entity, such as the memory 360 of the client device 300 , for example.

In various embodiments of the present disclosure, the MMTP decapsulation buffer 515 may operate as follows. MMTP decapsulation buffer 515 , when initially empty, receives MMTP packets after dejittering has been performed by dejitter buffer 510 . For MMTP packets carrying aggregated payloads, the receiving entity 410 strips the packet and payload headers and extracts each individual data unit. For MMTP packets carrying fragmented payloads, the packets are held in a buffer until all corresponding fragments are correctly received, or until packets not belonging to the same fragmented data unit are received. As discussed in more detail below, depending on the client's mode of operation, if a complete MPU, a movie segment, or a single MFU is recovered, the sending entity 405 forwards the reconstructed data to upper layers such as the presentation layer for presentation to the user to display.

As discussed above, the LDC defines a presentation order for multiple samples before receiving metadata such as movie fragment headers. Embodiments of the present disclosure further provide a message for signaling information used and/or required to calculate the presentation time of each sample before starting to remove data from the MMTP decapsulation buffer 515 .

Accordingly, embodiments of the present disclosure provide Low Latency Consumption (LDC) messages that provide the information needed to decode and render media data by the client before receiving metadata such as movie segment headers. This message indicates that the duration of each sample is fixed, as signaled by default_sample_duration in the Track Extend Box, and that the encoding dependency structure is fixed across assets. When this message is used, the value of the decoding time of the first sample of the MPU is smaller than the presentation time of the first sample of the MPU by the sum of the maximum value of fixed_presentation_time_offset and sample_composition_time_offset_value, and the paired sample_composition_time_offset_sign is "1".

Table 1 below provides example syntax for the LDC message.

Table 1

[Table 1]

In this illustrative example, "message_id" indicates the identifier of the LDC message, and "Version (version)" indicates the version of the LDC message. For example, an MMT receiving entity (eg, client device 300) may use this field to check the version of the received LDC message. Also, "length" indicates the length of the LDC message counted from the first byte of the next field to the last byte of the LDC message in bytes. The value "0" may not be valid for this field. Next, "base_presentaion_time_offset" provides information in microseconds about the time difference between the decoding time and the presentation time. The rendering time per sample can be greater than the decoding time by this value. This may not include any difference between the decoding time and presentation time of the samples due to reordering of the decoded media data.

Additionally, "coding_dependency_structure_flag" provides an indication that the decoding order and the presentation order of samples differ from each other. If this flag is set to '0', the decoding order shall be the same as the presentation order of the samples. If this flag is set to "1", the decoding order SHOULD be different from the sample's presentation order, and the detailed composition time offset shall be provided in this message for the client to calculate the appropriate decoding time and sample's presentation time. Furthermore, "period_of_intra_coded_sample" provides the number of samples between two independently coded samples.

FIG. 6 illustrates a process 600 for managing received data by a client device according to the present disclosure. For example, the processing depicted in FIG. 6 may be performed by the receiving entity 410 in FIG. 4 . Processing may also be implemented by the client device 300 in FIG. 3 .

Processing begins when the client device receives a message from a buffer with information about a presentation time for each of a plurality of samples of data (operation 610). For example, in operation 610, the client device receives a message from a buffer at the client device including information regarding a presentation time for each of a plurality of samples of data. This message may be an LDC message, may be included with other MMT signaling messages, and/or may be included at the beginning of the streaming media content to the client device. In one example, the buffer is MMTP decapsulation buffer 515 in FIG. 5 . Around the time the message is received, eg, after the message is received, the client device may begin receiving media data transfers associated with the message. The message may also be received after initiating reception of the media data transfer by the client device.

The client device then calculates a presentation time for each of the plurality of samples (operation 620). For example, in operation 620, the LDC message may include a value of the presentation time offset, a sign of the presentation time offset, a base presentation time offset, a number of samples between independently encoded samples, an encoding dependency structure identification, and the like.

In an example embodiment, the encoding dependency structure flag indicates whether the encoding of the samples is in the same order as the presentation order. The presentation order is the order in which samples are presented for display. If the encoding order is the same as the presentation order, samples can be removed in the same order as they were decoded. A sample of data may represent a frame.

In another example embodiment, the presentation time may be calculated by adding a base presentation time offset to a positive or negative presentation time offset. Sign indicates whether the presentation time offset is positive or negative, and value indicates the value of the presentation time offset. Values may be expressed in microseconds, milliseconds, or another type of time period. For example, if the base presentation time offset is t+40, and the presentation time offset for a sample is t+10, then the presentation time for that sample is t+50.

Thereafter, the client device removes data from the buffer based on the presentation time and passes the reconstructed data to the upper layer (operation 630). For example, in operation 630, the client device may remove data based on presentation time (eg, presentation time offset plus base presentation time offset). The client device passes the reconstructed data to an upper layer, such as a presentation layer, for presentation to the user on a display.

FIG. 7 illustrates a process 700 for indicating a presentation order by a server according to the present disclosure. For example, the processing depicted in FIG. 7 may be performed by the sending entity 405 in FIG. 4 . Processing can also be implemented by the server 200 in FIG. 2 .

Processing begins with the server generating a message including information on the presentation order of the received data (operation 710). For example, in operation 710, the message may include a presentation time for each sample. For example, a server may include values and symbols associated with each sample of data. The server may also include in the message information indicating a base presentation time offset.

Thereafter, the server sends a message to the client device (operation 720). For example, in operation 720, the server sends a message to the client device to signal the operation and management of removing data from the client device's MMTP decapsulation buffer. In these examples, the server may be the same server or a different server than the server sending the media data to the client.

Although FIGS. 6 and 7 respectively show examples of processes for managing received data by a client device and instructing a presentation order by a server, various changes may be made to FIGS. 6 and 7 . For example, while shown as a series of steps, various steps in each figure may overlap, occur in parallel, occur in a different order or occur multiple times.

Although the present disclosure has been described with reference to the exemplary embodiments, various changes and modifications may be suggested to one skilled in the art. It is intended that the present disclosure embrace such changes and modifications provided they come within the scope of the appended claims.

No part of the description in the present application should be falsely taken as implying that any specific element, step or function is an essential element which must be included in the scope of the claims. The scope of patented subject matter is defined only by the claims.

Claims (15)

1. a kind of user equipment for being communicated with least one base station radio, including：

Transceiver, is configured as by transmitting radiofrequency signal and by from least one described base at least one described base station Stand reception radiofrequency signal come with least one described base station communication, and be configured as receive include with header control message The packet of the payload relevant with the fragment of content of multimedia,

Wherein, the header includes：

Indicate control message whether be low latency consume message message identifier,

The length of controller message, and

The version of control message；And

Process circuit, is configured as determining whether control message is low latency consumption message based on message identifier, and by It is low latency consumption message, before the header of the fragment of receiving multimedia content to be configured in response to control message, based on having Effect load and control message carry out configuration packet.

2. user equipment according to claim 1, wherein, the payload includes：

Dependence structure mark is encoded, indicates whether the decoding order of packet is identical with presentation order；

It is basic that time migration is presented, indicate the basic skew of the fragment for content of multimedia to be presented；

Period between the sampling of two intraframe codings；

The symbol of time migration is presented in sampling for each fragment；And

The value of time migration is presented in sampling for each fragment.

3. user equipment according to claim 2, wherein, the process circuit is configured to：

Decoding order and presentation order identical coding dependence structure mark in response to indicating packet, based on basic presentation Time migration carrys out configuration packet.

4. user equipment according to claim 2, wherein, the process circuit is configured to：

The coding dependence structure mark different from presentation order in response to indicating the decoding order of packet, based on for each The sampling of fragment is presented time migration, symbol and value and carrys out configuration packet.

5. user equipment according to claim 1, wherein, the process circuit is configured to：

It is not low latency consumption message, sequentially configuration packet in response to control message.

6. user equipment according to claim 1, wherein, the header further comprises：

Extend information, including the configuration information relevant with control message payload.

7. user equipment according to claim 2, wherein, time migration, symbol are presented based on the sampling for each fragment Number and according to the value of version come configuration packet.

8. a kind of method for being communicated with least one base station radio, including：

The packet including the control message with header and the payload relevant with the fragment of content of multimedia is received, wherein, The header includes：

Indicate control message whether be low latency consume message message identifier,

The length of controller message, and

The version of control message；

Whether it is low latency consumption message that control message is determined based on message identifier；And

It is low latency consumption message, before the header of the fragment of receiving multimedia content in response to control message, based on effective Load and control message carry out configuration packet.

9. method according to claim 8, wherein, the payload includes：

Dependence structure mark is encoded, indicates whether the decoding order of packet is identical with presentation order；

It is basic that time migration is presented, indicate the basic skew of the fragment for content of multimedia to be presented；

Period between the sampling of two intraframe codings；

The symbol of time migration is presented in sampling for each fragment；And

The value of time migration is presented in sampling for each fragment.

10. method according to claim 9, further comprises：

Decoding order and presentation order identical coding dependence structure mark in response to indicating packet, based on basic presentation Time migration carrys out configuration packet.

11. method according to claim 9, further comprises：

The coding dependence structure mark different from presentation order in response to indicating the decoding order of packet, based on for each The sampling of fragment is presented time migration, symbol and value and carrys out configuration packet.

12. method according to claim 8, further comprises：

It is not low latency consumption message, sequentially configuration packet in response to control message.

13. method according to claim 8, wherein, the header further comprises：

Extend information, including the configuration information relevant with control message payload.

14. method according to claim 9, wherein, based on the sampling for each fragment present time migration, symbol and According to the value of version come configuration packet.

15. a kind of system, the system includes：

For the user equipment with the radio communication of at least one base station, the user equipment includes：

Transceiver, is configured as by transmitting radiofrequency signal and by from least one described base at least one described base station Stand reception radiofrequency signal come with least one described base station communication, and be configured as receive include with header control message The packet of the payload relevant with the fragment of content of multimedia,

Wherein, the header includes：

Indicate control message whether be low latency consume message message identifier,

The length of controller message, and

The version of control message；And

Process circuit, is configured as determining whether control message is low latency consumption message based on message identifier, and by It is low latency consumption message, before the header of the fragment of receiving multimedia content to be configured in response to control message, based on having Effect load and control message carry out configuration packet.