HK1223206B - Method and apparatus for transmission control protocol (tcp) based video streaming - Google Patents

Description

Video stream transmission method and device based on Transmission Control Protocol (TCP)

Related applications

This application claims priority to U.S. Non-Provisional Patent Application No. 14/041,446, filed September 30, 2013, docket number P56333, the entire specification of which is incorporated herein in its entirety for all purposes.

Technical Field

The present application relates to video streaming based on Transmission Control Protocol (TCP).

Background Art

Hypertext Transfer Protocol (HTTP) streaming is widely used in the form of multimedia delivery of Internet video. Multimedia delivery based on HTTP provides reliability and ease of deployment owing to the underlying protocol (comprising Transmission Control Protocol (TCP)/Internet Protocol (IP)) of HTTP and HTTP being widely adopted. In addition, multimedia delivery based on HTTP enables simple and easy streaming service by avoiding Network Address Translation (NAT) and firewall traversal problem. Streaming based on HTTP also provides the ability of using standard HTTP server and high-speed cache to replace special streaming server, and has better scalability due to the minimum state information on the server side. In one example, HTTP streaming can occur between node (e.g., transmission station) and wireless device (e.g., mobile device). Alternatively, HTTP streaming can occur between node and wired device (e.g., desktop computer).

Wireless mobile communication technology uses various standards and protocols to send data between nodes and wireless devices. Some wireless devices communicate by using orthogonal frequency division multiple access (OFDMA) in downlink (DL) transmission and single carrier frequency division multiple access (SC-FDMA) in uplink (UL) transmission. Standards and protocols using orthogonal frequency division multiplexing (OFDM) for signal transmission include the third generation partnership project (3GPP) long term evolution (LTE) (e.g., release 8, 9, 10, 11 or 12), the Institute of Electrical and Electronics Engineers (IEEE) 802.16 standards (e.g., 802.16e, 802.16m) (commonly referred to by industry groups as WiMAX (Worldwide Interoperability for Microwave Access)), and the IEEE 802.11 standards (commonly referred to by industry groups as WiFi, such as 802.11-2012, 802.11ac, or 802.11ad).

In a 3GPP radio access network (RAN) LTE system, a node may be a combination of an Evolved Universal Terrestrial Radio Access Network (E-UTRAN) Node B (often also referred to as an evolved Node B, enhanced Node B, eNodeB, or eNB) and a Radio Network Controller (RNC), which communicates with wireless devices known as user equipment (UE). Downlink (DL) transmissions may be communications from a node (e.g., an eNodeB) to a wireless device (e.g., a UE), and uplink (UL) transmissions may be communications from a wireless device to a node.

Summary of the Invention

In one aspect, a Transmission Control Protocol (TCP) receiver is provided, the TCP receiver having a computer circuit for reducing delays in data stream transmission, the computer circuit being configured to: receive a plurality of TCP segments from a network element at a TCP receiver buffer; detect a missing TCP segment based on out-of-order TCP segments received in the plurality of TCP segments; determine at the TCP receiver that the missing TCP segment to be received before the out-of-order TCP segment can be discarded based on context information associated with the data stream transmission, wherein the context information includes network layer context information and application layer context information; send a false acknowledgement (ACK) message to the network element based on the context information, the false ACK message confirming that the missing TCP segment is received at the TCP receiver, wherein the false ACK message includes a request for a TCP segment to be transmitted to the TCP receiver that logically follows the out-of-order TCP segment; and provide the out-of-order TCP segment that does not include the missing TCP segment from the TCP receiver buffer to a display device.

In another aspect, a method for reducing delay in data stream transmission at a wireless device includes: detecting a missing data segment based on out-of-sequence data segments received from a network element in a wireless network, wherein the data segment is a Transmission Control Protocol (TCP)-based data segment; determining that the missing data segment, which is to be received before the out-of-sequence data segment, can be discarded based on context information associated with the data stream transmission, wherein the context information includes network layer context information and application layer context information; sending a false acknowledgement (ACK) message to the network element in the wireless network based on the context information, the false ACK message acknowledging that the missing data segment is received at the wireless device, wherein the false ACK message includes a request for a data segment to be transmitted to the wireless device that logically follows the out-of-sequence data segment; and providing the out-of-sequence data segments without the missing data segment for display at the wireless device.

In another aspect, a wireless device for data stream transmission includes: a receiving device, which is configured to receive multiple TCP segments from a transmission control protocol (TCP) receiver buffer from a network element; a detection device, which is configured to detect a missing TCP segment based on out-of-order TCP segments received in the multiple TCP segments; a segment discarding device, which is configured to: discard the missing TCP segment to be received before the out-of-order data segment based on context information associated with the data stream transmission, wherein the context information includes network layer context information and application layer context information; and a confirmation device, which is configured to: send a false confirmation message to the network element based on the context information, wherein the false confirmation message confirms that the missing TCP segment is received at the TCP receiver, wherein the false confirmation message includes a request for a TCP segment to be transmitted to the TCP receiver that logically follows the out-of-order TCP segment.

BRIEF DESCRIPTION OF THE DRAWINGS

Features and advantages of the present disclosure will be apparent from the following detailed description taken in conjunction with the accompanying drawings, which together illustrate features of the present disclosure by way of example; and wherein:

FIG1A illustrates a plurality of Transmission Control Protocol (TCP) segments being transferred to a TCP buffer, according to an example;

FIG1B illustrates a plurality of Transmission Control Protocol (TCP) segments transmitted from a TCP buffer, according to an example;

FIG1C illustrates a plurality of Transmission Control Protocol (TCP) segments transmitted from a TCP buffer without missing TCP segments, according to an example;

FIG. 1D illustrates a plurality of Transmission Control Protocol (TCP) segments transmitted from a TCP buffer without missing TCP segments, according to an example;

FIG2 illustrates a system for transmitting a Transmission Control Protocol (TCP) segment using context information, according to an example;

3 illustrates the functionality of computer circuitry of a Transmission Control Protocol (TCP) receiver operable to reduce delays in data stream transmission, according to an example;

4 illustrates a flow chart of a method at a wireless device for reducing delay in data stream transmission, according to an example;

FIG5 illustrates a block diagram of a wireless device (eg, user equipment) according to an example; and

FIG6 shows a diagram of a wireless device (eg, user equipment), according to an example.

Reference will now be made to the exemplary embodiments illustrated, and specific language will be used herein to describe the same. However, it will be understood that no limitation of the scope of the invention is intended herein.

DETAILED DESCRIPTION

Before the present invention is disclosed and described, it should be understood that the present invention is not limited to the specific structures, processing steps, or materials disclosed herein, but is extended to its equivalents that will be recognized by those of ordinary skill in the relevant art. It should also be understood that the terms employed herein are only used to describe the purpose of specific examples and are not intended to be restrictive. The same reference numerals in different diagrams represent the same elements. The numerals provided in the flow charts and the processing are provided to clearly illustrate steps and operations, and do not necessarily indicate a specific order or sequence.

Example Embodiments

The following provides a preliminary overview of the technology embodiments, and then specific technology embodiments are described in more detail later. This preliminary overview is intended to help readers understand the technology more quickly, but is not intended to identify key features or essential features of the technology, nor is it intended to limit the scope of the claimed subject matter.

Hypertext Transfer Protocol (HTTP) streaming is a form of multimedia delivery of Internet video (e.g., live video or video on demand) and audio content (referred to as multimedia content, media content, media services, etc.). In HTTP streaming, multimedia files can be divided into one or more segments and delivered to a client using the HTTP protocol. Multimedia content delivery (streaming) based on HTTP provides reliability and simple content delivery due to the extensive use of HTTP and HTTP's underlying protocols (including Transmission Control Protocol (TCP)/Internet Protocol (IP)). Delivery based on HTTP can enable simple and easy streaming services by avoiding network address translation (NAT) and firewall traversal problems. The transmission of streaming data based on HTTP can also provide the ability to use a standard HTTP server and cache to replace a specialized streaming server.

In addition, HTTP streaming can provide several benefits, such as reliable transmission and adaptability to network conditions to ensure fairness and avoid congestion. HTTP streaming can provide scalability due to minimal or reduced state information on the server side. However, due to congestion control and strict flow control, HTTP streaming can cause delays and fluctuations in transmission rate. Therefore, HTTP-based streaming systems include buffers to mitigate rate fluctuations, but as a result, users may experience high latency when the video is streamed.

HTTP Dynamic Adaptive Streaming (DASH) is an adaptive multimedia streaming technology in which multimedia files can be divided into one or more segments and delivered to the client using HTTP. DASH specifies the format of a media presentation description (MPD) metadata file that provides information about the structure and different versions of media content representations stored in the server, as well as the segment format. For example, the metadata file contains information about the initialization of the media player and the media segments (the media player can view the initialization segment to understand the container format and media timing information) to ensure that the segments are mapped to the media presentation timeline for switching and synchronous presentation with other representations. A DASH client can receive multimedia content by downloading segments via a series of HTTP request-response processes. DASH can provide the ability to dynamically switch between different bit rate representations of media content when the bandwidth available to a mobile device changes. Therefore, DASH can allow for rapid adaptation to changing network and wireless link conditions, user preferences, and device performance (such as display resolution, the type of computer processor used, memory resource availability, etc.). DASH is an example technology that can be used to address the shortcomings of streaming based on the Real Time Protocol (RTP) and RTSP and progressive download based on HTTP. DASH-based adaptive streaming is an alternative to RTSP-based adaptive streaming, which is standardized in various versions of the 3rd Generation Partnership Project (3GPP) Technical Specification (TS) 26.247 (including versions 10 and 11) and the Moving Picture Experts Group (MPEG) ISO/IEC DIS 23009-1.

HTTP's underlying protocol, TCP, is a "connection-oriented" data delivery service, enabling two TCP-configured devices to establish a TCP connection with each other to enable data communication between the two TCP devices. Generally, "data" can refer to a TCP segment or bytes of data. Furthermore, TCP is a full-duplex protocol. Therefore, each of the two TCP devices can support a pair of data streams flowing in opposite directions. Thus, a first TCP device can transmit (i.e., send or receive) a TCP segment with a second TCP device, and the second TCP device can transmit (i.e., send or receive) a TCP segment with the first TCP device.

TCP may assign a segment number to each TCP segment transmitted between a first TCP device and a second TCP device. The sending TCP device may expect a positive acknowledgment (ACK) from the receiving TCP device after the receiving TCP device receives the TCP segment. In other words, the receiving TCP device may transmit an ACK message to the sending TCP device after receiving the TCP segment. If the ACK is not received within a timeout interval, the TCP segment may be retransmitted. Therefore, if the sending TCP device does not receive an ACK message from the receiving TCP device within the timeout interval, the sending TCP device may retransmit the TCP segment to the receiving TCP device. If TCP segments arrive at the receiving TCP device out of order, the receiving TCP device may use the segment numbers to reorder the TCP segments and/or eliminate overlapping TCP segments.

The ACK message transmitted between the receiving TCP device and the sending TCP device may include the number of TCP segments that the receiving TCP device can receive from the sending TCP device in addition to the TCP segment received last time. In other words, the receiving TCP device may transmit the highest number of segments that can be received from the sending TCP device so that the received TCP segments do not cause overload and overflow in the receiving TCP device buffer. Generally, the TCP device may temporarily store the TCP segments received from the network element in a buffer before the TCP segments are transmitted to the display device. Alternatively, before the TCP segments are provided to the display device, the TCP segments may be transmitted from the TCP receiver buffer to another area or module within the user equipment (UE) for processing. In one example, TCP segments that arrive out of order may be rearranged within the TCP receiver buffer so that in-order TCP segments can be delivered to the display device. The number of TCP segments that can be stored in the TCP buffer may depend on the TCP buffer size.

Furthermore, the ACK message transmitted between the receiving TCP device and the sending TCP device may include a next expected segment number. The next expected segment number may be the next segment number that the receiving TCP device expects to receive from the sending TCP device. In other words, the next expected segment number may logically follow the last TCP segment successfully received at the receiving TCP device.

In one example, HTTP/TCP-based video streaming can be enhanced by utilizing characteristics of video data and/or wireless networks. As a result, HTTP/TCP-based video streaming can experience reduced latency and/or improved video quality. Specifically, cross-layer information from the application layer and the network layer can be incorporated to modify the functionality of the TCP receiver. Since the modification of the TCP receiver can be implemented on the client side, modifications to the network infrastructure may be unnecessary. Modifications to the TCP receiver can improve rebuffering, average picture quality, the number of rate switches, and the like. As a result, the user's quality of experience (QoE) can also be improved.

As will be discussed in greater detail below, a TCP receiver may receive multiple TCP segments from a network element (e.g., a TCP device). The TCP receiver may determine that a TCP segment is missing from the multiple TCP segments. In other words, the TCP receiver expected to receive the TCP segment, but the TCP segment was not transmitted to the TCP receiver (i.e., the TCP segment is missing).

Generally, a missing TCP segment may be retransmitted from a network element to a TCP receiver. When the TCP receiver successfully receives the missing TCP segment, the missing TCP segment may be referred to as a delayed TCP segment. The TCP receiver determines whether the delayed TCP segment is received within a predetermined time threshold. In one example, the predetermined time threshold may be dynamically configured based on network layer information and application layer information.

In addition, the TCP receiver can determine whether the delayed TCP segment has a lower priority (compared to other TCP segments transmitted to the TCP receiver). The TCP receiver can use application layer information and network layer information to determine that the delayed TCP segment has a lower priority. The network layer context information can include at least one of the following: an explicit loss indication including media access control (MAC) layer packet loss, loss due to congestion inferred via analysis of TCP receiver buffer contents, or explicit network congestion information. The application layer context information can include at least one of the following: buffer status, frame type, saliency of the video frame, type of video content, and other context such as device context information or user context information.

If the delayed TCP segment is determined to have a lower priority (based on application layer and network layer information) and the delayed TCP segment is not received at the TCP receiver within a predetermined time threshold, the delayed TCP segment may be discarded. In other words, the TCP receiver may deliver multiple TCP segments with the missing TCP segment to the display device. In addition, the TCP receiver may send a false acknowledgement (ACK) message to the network element, which falsely confirms that the TCP receiver received the previously missing TCP segment. In addition, the false ACK message may include the segment number of the next TCP segment that the receiving TCP device expects to receive from the sending TCP device (i.e., the TCP segment that logically follows the delayed TCP segment).

FIG1A illustrates multiple Transmission Control Protocol (TCP) segments 110 transmitted to a TCP receiver. Specifically, the TCP segments may be received in a TCP receiver buffer. In one example, the TCP receiver buffer may receive TCP segments from a network element (e.g., a sending TCP device). The multiple TCP segments transmitted to the TCP receiver buffer may include missing TCP segments. For example, segment 1 may be transmitted at time (TS_1), segment 2 may not be transmitted to the TCP receiver buffer, segment 3 may be transmitted at TS_3, segment 4 may be transmitted at TS_4, and segment 5 may be transmitted at TS_5. Consequently, the TCP receiver buffer may include a hole corresponding to the missing segment 2.

When the TCP receiver buffer receives segment 3 but not segment 2, the TCP receiver buffer detects that segment 3 is an out-of-order segment. In other words, segment 3 was received out-of-order at the TCP receiver buffer because the TCP receiver buffer should have received segment 2 before receiving segment 3. When the TCP receiver receives an out-of-order segment (e.g., segment 3), the TCP receiver may transmit an acknowledgment (ACK) message to the network element, confirming that the out-of-order segment was received at the TCP receiver. Furthermore, the ACK message may include a next expected segment number (i.e., the next TCP segment the TCP receiver expects to receive from the network element). In one example, the next expected segment number may refer to the missing TCP segment (e.g., segment 2) that was scheduled to be received before the out-of-order segment (e.g., segment 3). Therefore, when the TCP receiver receives segment 3 from the network element, the TCP receiver may transmit an ACK message to the network element with segment 2 as the next expected segment number.

The network device may receive an ACK message from the TCP receiver indicating the next expected segment number (e.g., segment 2). The network device may resend the next expected segment number (e.g., segment 2) to the TCP receiver. Because segment 2 is missing, segments 3, 4, and 5 may be temporarily stored in the TCP receiver buffer until segment 2 is successfully retransmitted to the TCP receiver buffer. The number of segments that can be buffered in the TCP receiver buffer at a given time may be limited by the TCP receiver buffer size.

In one example, the TCP receiver buffer may receive segment 2 at a time (TS_H) after segments 3, 4, and 5 have been successfully delivered to the TCP receiver buffer. The delivery of segment 2 at the TCP receiver buffer may fill the hole in the TCP receiver buffer caused by the missing segment 2. Thus, segment 2 may change from a missing segment to a delayed segment after being successfully delivered to the TCP receiver buffer. The buffered segments 2-5 may be transmitted, for example, to a display device. Furthermore, after receiving delayed segment 2, the TCP receiver may send an ACK message to the network element: (1) acknowledging that segment 2 was received at the TCP receiver buffer and (2) notifying the network element of the next expected segment (e.g., segment 6).

As previously described, delayed TCP segments may be received at TS_H at the TCP receiver buffer. The TCP receiver buffer waits until the delayed TCP segments are received before transmitting the delayed TCP segments (e.g., segment 2) and out-of-order segments (e.g., segments 3-5) to the display device. Therefore, segments 2-5 may be transmitted to the display device at TS_H (not shown in FIG. 1A ). In other words, segments 2-5 may be transmitted to the display device at a time corresponding to when segment 2 is received at the TCP receiver buffer (i.e., TS_H). The delay (D) in segment 2-5 transmitted to the display device may be equal to TS_H-TS_1. As a result, the user may experience a delay when streaming video.

In one example, a TCP receiver may determine whether a delayed TCP segment is received within a time threshold (Δ). The time threshold (Δ) may be implemented to reduce the delay of segments transmitted to a display device. In other words, the amount of acceptable delay (D) may be limited by the time threshold (Δ). In one example, the time threshold (Δ) may be determined using a feedback mechanism, application layer information, and/or network layer information. Delay may result from one or more delayed TCP segments being delivered to a TCP receiver buffer out of order (i.e., a delayed TCP segment is delivered late), which in turn affects the time at which delayed TCP segments and/or out-of-order segments (e.g., TCP segments to be delivered after the delayed TCP segment) are delivered to the display device.

As shown in Figures 1B-1D, there may be three scenarios describing the delay (D) of TCP segments being transmitted to the display device relative to a time threshold (Δ). In scenario 1 (as shown in Figure 1B), the delay (D) may be less than or equal to the time threshold (Δ). In scenario 2 (as shown in Figure 1C), the delay (D) may be greater than the time threshold (Δ). In scenario 3 (as shown in Figure 1D), the time threshold (Δ) may be equal to zero.

FIG1B illustrates the communication of multiple TCP segments 120 (e.g., segments 1-5) from a TCP receiver buffer to a display device. Segments 1-5 may be transmitted from a network element to a TCP receiver buffer. As shown in FIG1A , segment 1 may be transmitted at TS_1, segment 3 at TS_3, segment 4 at TS_4, segment 5 at TS_5, and segment 2 at TS_H. In other words, segment 2 may be a delayed segment.

As shown in Figure 1B, when segment 2 is received at the TCP receiver buffer within a time threshold (Δ) (i.e., D ≤ Δ), segments 1-5 can be transmitted to the display device in sequence. In other words, the delayed segment (e.g., segment 2) can also be transmitted to the display device. In this example, although segment 2 is received at TS_H, the delay (i.e., (TS_H-TS_1)) is acceptable because it is less than the time threshold Δ. Segment 1 can be transmitted from the TCP receiver buffer to the display device at TS_1, and segments 2-5 can be transmitted at TS_H. In addition, the TCP receiver can transmit an acknowledgment (ACK) message including the expected segment number (e.g., the expected segment number is 6) to the network element.

When a delayed segment is received at the TCP receiver buffer within a time threshold (Δ), the TCP receiver may determine the priority associated with the delayed segment. Generally, the priority of the delayed segment may be determined using application layer information and/or network layer information. In other words, in scenario 1, the TCP receiver may deliver the TCP segment to the display device (without using the priority of the delayed segment) when the delayed segment is transmitted to the TCP receiver within the time threshold (Δ).

FIG1C illustrates the communication of multiple TCP segments 130 (e.g., segment 1 and segments 3-5) from a TCP receiver buffer to a display device. Segments 1-5 may be transmitted from a network element to the TCP receiver buffer. As shown in FIG1A , segment 1 may be transmitted at TS_1, segment 3 at TS_3, segment 4 at TS_4, segment 5 at TS_5, and segment 2 at TS_H. In other words, segment 2 may be a delayed segment.

As shown in FIG1C , when Segment 2 is not received at the TCP receiver buffer within a time threshold (Δ) (i.e., D>Δ), multiple segments (excluding the delayed segment) may be transmitted to the display device. Therefore, the delayed segment (e.g., Segment 2) is not transmitted to the display device. In this example, when Segment 2 is not received within the time threshold (Δ), the TCP receiver determines to discard the delayed segment. Since Segment 2 was not received within the time threshold (Δ), the delay is unacceptable and Segment 2 is discarded.

Segment 1 may be transmitted from the TCP receiver buffer to the display device at TS_1, and segments 3-5 may be transmitted at TS_1+Δ, where Δ represents the period of time the TCP receiver waits for delivery of segment 2. In addition, the TCP receiver may transmit an acknowledgment (ACK) message to the network element including the expected segment number (e.g., the expected segment number is 6).

As described in more detail below, when a delayed segment is not received at the TCP receiver buffer within a time threshold (Δ), the TCP receiver buffer may determine a priority associated with the delayed segment. For example, the priority of the delayed segment may be determined using application layer information and/or network layer information. In one configuration, when the delayed segment is not received within the time threshold (Δ), the TCP receiver may discard the delayed segment (e.g., segment 2) based on the priority of the delayed segment.

FIG1D illustrates the communication of multiple TCP segments 140 (e.g., segment 1 and segments 3-5) from a TCP receiver buffer to a display device. Segments 1-5 may be transmitted from a network element to a TCP receiver buffer. As shown in FIG1A , segment 1 may be transmitted at TS_1, segment 3 at TS_3, segment 4 at TS_4, segment 5 at TS_5, and segment 2 at TS_H. In other words, segment 2 may be a delayed segment.

As shown in FIG1D , when segment 2 is received late at the TCP receiver buffer (i.e., D = 0), multiple segments (excluding the delayed segment) may be transmitted to the display device. In other words, the late TCP segment may correspond to a hole when the TCP receiver sends multiple TCP segments to the display device. Therefore, the delayed segment (e.g., segment 2) is not transmitted to the display device. In this example, because segment 2 was received late, the TCP receiver determines to discard the delayed segment.

Segment 1 may be transmitted from the TCP receiver buffer to the display device at TS_1, segment 2 at TS_3, segment 4 at TS_4, and segment 5 at TS_5. In addition, the TCP receiver may transmit an acknowledgment (ACK) message including the expected segment number (e.g., the expected segment number is 6) to the network element.

When the time threshold (Δ) is equal to zero, the TCP receiver may not determine the priority associated with the delayed segment. In other words, in scenario 3, the TCP receiver may discard the delayed TCP segment from the multiple TCP segments transmitted to the display device without using the priority of the delayed segment.

In general, the out-of-order delay experienced by TCP segments in a TCP receiver buffer can be: out-of-order delivery delay (D = 0) ≤ Δ ≤ in-order delivery delay (D = TS_H - TS_1). With fully out-of-order delivery, all late, out-of-order TCP segments can be considered holes. In other words, delayed segments may not be delivered to the display device. With in-order delivery, TCP segments can be delivered to the display device without holes. Therefore, the trade-off between TCP packet loss and latency can be derived based on application layer information and network layer information.

In one configuration, a TCP receiver may use the priority of delayed segments (i.e., segments that were previously missing but were delivered at a later time) to determine whether to discard the delayed segments when the delayed segments are not received at the TCP receiver within a time threshold (Δ). Alternatively, the TCP receiver may use the priority of delayed segments to determine whether to not discard the delayed segments even when the delayed segments are not received at the TCP receiver within the time threshold (Δ). Generally, application layer information and/or network layer information may indicate that a reduction in video quality is more important than the video latency resulting from the delayed segments. In other words, a reduction in video quality is preferred over waiting for the video to be loaded. Therefore, as opposed to waiting for the delayed segments to be delivered, the TCP receiver may completely discard the delayed segments in response to analysis of the application layer and network layer information.

FIG2 illustrates a system 200 for transmitting Transmission Control Protocol (TCP) segments using context information. A TCP receiver may receive TCP segments. A context-adaptive decision block may determine whether the received TCP segments are out of sequence. For example, if a TCP receiver receives segment 3 without receiving segment 2, the TCP receiver may detect that the received TCP segment (e.g., segment 3) is out of sequence. The TCP receiver may then receive a delayed TCP segment (e.g., segment 2) after receiving the out-of-sequence segment.

In one example, the context adaptation decision block can determine whether a delayed TCP segment includes an application context trigger. In other words, the application context information can enable the TCP receiver to determine whether the delayed TCP segment should be discarded so that multiple TCP segments can be delivered without the delayed TCP segment. Various application context information about the multimedia information can be identified, such as the playback buffer state, the frame type or other saliency information for the next video frame expected in the playback buffer, historical information about recent rate switches performed by the adaptive streaming player, and so on. The application context information can be obtained from modifications to the client implementation of DASH or other HTTP adaptive streaming player software on the client.

Application layer context information may include buffer status and history, frame type, saliency, content type, and other context such as device context and/or user context. For example, video data in the replay buffer and the TCP receiver's buffer may have unequal priorities for different parts of the data stream (e.g., video data may have unequal priorities depending on whether the video frame is an I/P frame or a B frame). As another example, when the application is approaching play buffer starvation and/or the video information expected from the TCP receiver at the end of the replay buffer is determined to be a B frame, the context adaptive decision block may be provided with this information to determine whether the B frame should be discarded.

Adaptive streaming clients can use the buffer status and history of rate adaptation decisions. In addition to the current buffer status, the application can also provide a history of video rate switches (e.g., adaptations) made in the recent past. The history of video rate switches can be useful because user QoE can also be affected by the frequency associated with rate switches.

The frame type can influence whether delayed TCP segments can be discarded. I- and P-frames have temporal dependencies, which have a greater impact on the decoding of subsequent frames, whereas B-frames do not have temporal dependencies and can therefore be discarded with less impact on image quality. As an example, the I/P/B frame structure ("GOP") allows a video player to determine the position of the next expected P-frame in the sequence relative to the next I-frame. Therefore, the impact of potentially discarding this P-frame can be estimated. Frame type information is explicitly accessible to the video player in the frame header.

In one example, saliency in terms of visual impact (e.g., the importance of a given frame) can be used by a video player. For example, if H.264 Scalable Video Coding (SVC) is used, enhancement layer data can have lower importance than base layer video data. Application layer context information regarding saliency can be explicitly available in the frame header or provided through other mechanisms.

Furthermore, video applications can modify the tradeoff between rebuffering and picture quality differently depending on whether the content is being viewed as live content or as video on demand (VOD). Furthermore, video applications can modify the tradeoff between rebuffering and picture quality depending on whether the content is related to sports, news, etc. Application context information about the content type can be provided in the content metadata.

In one example, the context information may include device context information. Screen size may play a role in user perception and expectations. Device battery level may be used as context input, as the tradeoff between quality and throughput may differ based on the device's remaining battery level.

As an additional example, the context information may include user context information. For example, mobile users may face different conditions with respect to packet loss, latency, and/or throughput compared to nomadic/stationary users. Furthermore, users may have different QoE expectations depending on whether the content is free or subscription-based.

In one example, the context adaptation decision block can determine whether the delayed TCP segment includes a network layer context trigger. In other words, the network layer context information can determine whether the delayed TCP segment should be dropped so that multiple TCP segments can be delivered without the delayed TCP segment. The network layer context information can be merged with TCP processing at the TCP receiver. The network layer context information can include explicit cross-layer information from the network interface (e.g., a MAC layer retransmission failure indication) or explicit congestion notifications from the network element.

Furthermore, network layer context information can be derived based on analysis of TCP receiver buffer contents (e.g., statistics of missing TCP segments awaiting retransmission). Losses related to network congestion can differ from losses due to radio link layer errors on the uplink/downlink. In one example, network layer context information can be derived based on modifications to network interface card (NIC) driver waits. Furthermore, the context adaptation decision block can analyze network context layer information (e.g., analyzing TCP receiver buffer segment holes and associated statistics, integrating feedback from lower layers regarding radio/congestion losses), and adjust a threshold (Δ) for pending segments that can be released.

Thus, network layer context information can enable a TCP receiver to determine whether delayed segments should be discarded. The network layer context information can include an indication of media access control (MAC) layer packet loss (e.g., retransmission timeout). This indication can come from a network interface card (NIC) or modem indicating missing IP packets. The indication of missing MAC layer packets can be a clear signal to the TCP receiver that the missing data is due to radio link errors rather than network congestion. In addition, network layer context information can be obtained from analysis of the TCP receiver buffer contents. Specifically, statistics associated with missing segments in the TCP receiver buffer can be examined to provide contextual information about whether the TCP receiver buffer is experiencing radio link (e.g., random) loss or congestion-related (e.g., large, bursty holes) loss.

In one example, the network layer context information may include an Explicit Congestion Notification (ECN) marked in the IP header, thereby providing the TCP receiver with network layer context information about TCP segment holes. In one example, the network layer context information may be used to drop delayed TCP segments and forward ACKs when wireless link errors (rather than network congestion) cause the video player to experience an impact on QoE. Large bursts of packet loss are likely caused by network congestion and are therefore not viable candidates for ACKing due to the potential increased impact on picture quality. Additionally, smaller bursts of packet loss may come from wireless link loss and may be viable candidates for dropping delayed TCP segments and sending ACKs to continue with TCP segments other than the holes corresponding to the packet losses.

If the context adaptation decision block determines that the received segments are in order (i.e., not out of order) and no application context trigger or network context trigger is present, conventional TCP operations may be performed. If the context adaptation decision block determines the context trigger and the decision threshold has been reached (i.e., the delayed TCP segments do not meet the delay threshold), the out of order segments may be delivered to the display device, thereby skipping the delayed TCP segments.

As previously described, delayed TCP segments that exceed a delay threshold (Δ) and that also indicate a reduced priority based on application or network layer context information may be discarded. Consequently, multiple TCP segments are delivered with holes corresponding to the delayed TCP segments. Furthermore, acknowledgments (ACKs) may be transmitted to advance the TCP segments beyond the holes. In other words, an ACK indicates that the TCP receiver expects to receive the TCP segment that logically follows the delayed TCP segment. Therefore, the TCP receiver can relax the requirements for reliable information delivery by allowing the selective issuance of false ACKs for defined TCP segments that are determined to have lower priority based on application and network layer information. Therefore, once the delay threshold (Δ) is reached, the context-adaptive decision block can release the pending TCP segments, and the TCP receiver can proceed with the ACK for the next TCP segment.

Another example provides functionality 300 of a computer circuit of a Transmission Control Protocol (TCP) receiver operable to reduce delay in data stream transmission, as shown in the flowchart of FIG3 . The functionality may be implemented as a method or as instructions executable on a machine, wherein the instructions are included on at least one computer-readable medium or a non-transitory machine-readable storage medium. The computer circuit may be configured to receive a plurality of TCP segments from a network element at a Transmission Control Protocol (TCP) receiver buffer (as in block 310 ). The computer circuit may be further configured to detect missing TCP segments based on out-of-order TCP segments received in the plurality of TCP segments (as in block 320 ). The computer circuit may be further configured to determine, based on context information associated with the data stream transmission, at the TCP receiver that missing TCP segments received before the out-of-order TCP segments should be discarded (as in block 330 ). Furthermore, the computer circuit may be configured to provide the out-of-order TCP segments from the TCP receiver buffer to a display device if there are no missing TCP segments (as in block 340 ).

In one example, the computer circuitry may be configured to discard a missing TCP segment based on context information when the missing TCP segment is not received within a predetermined time threshold. In one configuration, the context information includes network layer context information and application layer context information. In one example, the network layer context information may include at least one of: media access control (MAC) layer packet loss, TCP receiver buffer content analysis, or network congestion information. Additionally, the application layer context information may include at least one of: buffer status, video frame saliency, frame type, video content type, or other context such as device context information or user context information.

In one configuration, the computer circuitry may be further configured to, based on the context information, send a false acknowledgement (ACK) message to the network element that acknowledges that the missing TCP segment was received at the TCP receiver. In one example, the false ACK message includes a request for a TCP segment to be transmitted to the TCP receiver that logically follows the out-of-order TCP segment.

In one configuration, the computer circuitry may be further configured to send an acknowledgment message to the network element requesting that missing TCP segments be retransmitted to the TCP receiver, wherein the missing TCP segments cannot be discarded based on the context information. Furthermore, the computer circuitry may be further configured to discard the missing TCP segments when the context information indicates that a radio link error caused out-of-order TCP segments to be delivered out-of-order to the TCP receiver buffer. Furthermore, the computer circuitry may be further configured to determine that the missing TCP segments should not be discarded when the context information indicates that network congestion caused out-of-order TCP segments to be delivered out-of-order to the TCP receiver buffer.

In one configuration, the TCP receiver may operate in a user equipment (UE). The UE may include an antenna, a touch display screen, a speaker, a microphone, a graphics processor, an application processor, internal memory, or a non-volatile memory port.

Another example provides a method 400 for reducing latency in data stream transmission at a wireless device, as shown in the flowchart of FIG4 . The method may be executed as instructions on a machine, wherein the instructions are included on at least one computer-readable medium or a non-transitory machine-readable storage medium. The method includes detecting missing data segments based on out-of-order data segments received from a network element in a wireless network (as in block 410 ). The method may include determining, based on context information associated with the data stream transmission, that the missing data segments received before the out-of-order data segments are discarded (as in block 420 ). A next operation of the method may include sending a false acknowledgement (ACK) to the network element in the wireless network based on the context information, confirming that the missing data segments were received at the wireless device (as in block 430 ). Furthermore, the method may include providing the out-of-order data segments for display at the wireless device in the absence of the missing data segments (as in block 440 ).

In one example, the context information includes network layer context information and application layer context information.In addition, the data segment may be a data segment based on the Transmission Control Protocol (TCP).

In one configuration, the method may include discarding a missing data segment when not received within a time threshold. Furthermore, the method may include dynamically configuring the time threshold based on network layer context information and application layer context information. Additionally, the method may include identifying context information at the wireless device. In one configuration, the method may include identifying context information at a wireless element in a wireless network. Furthermore, the method may include receiving a plurality of data segments at a Transmission Control Protocol (TCP) receiver, wherein the TCP receiver is included in the wireless device. In one example, the wireless device may be selected from the group consisting of a user equipment (UE), a mobile station, a Bluetooth receiver, an 802.11 receiver, and combinations thereof.

FIG5 illustrates an example wireless device (e.g., user equipment) 500 configured for data stream transmission, according to another embodiment of the present invention. The wireless device includes a receiving module 502 configured to receive a plurality of Transmission Control Protocol (TCP) segments from a network element 520 at a TCP receiver buffer 512. A detecting module 504 may be configured to detect missing TCP segments based on out-of-order TCP segments received in the plurality of TCP segments. A segment discarding module 506 may be configured to discard missing TCP segments that were received before the out-of-order data segments based on context information associated with the data stream transmission.

In one configuration, the wireless device 500 may include a segment delivery module 508 that may be configured to, in the absence of missing TCP segments, provide out-of-order TCP segments from the TCP receiver buffer 512 to the display device 530. Additionally, the wireless device 500 may include an acknowledgement module 510 that may be configured to send a false acknowledgement message to the network element 520 based on the context information, the message acknowledging that the missing TCP segments were received at the TCP receiver (e.g., the receiving module 502).

In one configuration, the segment discard module 506 may be further configured to discard the missing TCP segment based on the context information and that the missing TCP segment was not received within a time threshold. In one example, the context information includes network layer context information and application layer context information.

Figure 6 provides an example diagram of a wireless device, such as a user equipment (UE), a mobile station (MS), a mobile wireless device, a mobile communication device, a tablet, a handset, or other type of wireless device. The wireless device may include one or more antennas configured to communicate with a node or transmission station, such as a base station (BS), an evolved Node B (eNB), a baseband unit (BBU), a remote radio head (RRH), a remote radio equipment (RRE), a relay station (RS), a radio equipment (RE), or other type of wireless wide area network (WWAN) access point. The wireless device may be configured to communicate using at least one wireless communication standard, including 3GPP LTE, WiMAX, High Speed Packet Access (HSPA), Bluetooth, and WiFi. The wireless device may communicate using a different antenna for each wireless communication standard, or a shared antenna for multiple wireless communication standards. The wireless device may communicate in a wireless local area network (WLAN), a wireless personal area network (WPAN), and/or a WWAN.

FIG6 also provides an illustration of a microphone and one or more speakers that can be used for audio input and output of the wireless device. The display screen can be a liquid crystal display (LCD) screen or other type of display screen, such as an organic light emitting diode (OLED) display. The display screen can be configured as a touch screen. The touch screen can use capacitive, resistive, or another type of touch screen technology. An application processor and a graphics processor can be coupled to internal memory to provide processing and display capabilities. A non-volatile storage port can also be used to provide data input/output options to the user. The non-volatile storage port can also be used to expand the storage capacity of the wireless device. A keyboard can be integrated with the wireless device or wirelessly connected to the wireless device to provide additional user input. A virtual keyboard can also be provided using the touch screen.

Various techniques or aspects or portions thereof may take the form of program code embodied in a tangible medium such as a floppy disk, CD-ROM, hard drive, non-transitory computer-readable storage medium, or any other machine-readable storage medium, wherein when the program code is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for implementing the various techniques. Circuitry may include hardware, firmware, program code, executable code, computer instructions, and/or software. A non-transitory computer-readable storage medium may be a computer-readable storage medium that does not include signals. In the case where the program code runs on a programmable computer, the computing device may include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. The volatile and non-volatile memory and/or storage elements may be RAM, EPROM, a flash drive, an optical drive, a magnetic hard drive, or other media for storing electronic data. Nodes and wireless devices may also include a transceiver module, a counter module, a processing module, and/or a clock module or timer module. One or more programs that may implement or utilize the various techniques described herein may use an application programming interface (API), reusable controls, and the like. Such programs can be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language can be a compiled or interpreted language, combined with a hardware implementation.

It should be understood that many of the functional units described in this specification have been labeled as modules in order to more specifically emphasize their implementation independence. For example, a module can be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module can also be implemented using programmable hardware devices such as field programmable gate arrays, programmable array logic, or programmable logic devices.

Modules can also be implemented using software executed by various types of processors. An identified module of executable code may, for example, include one or more physical or logical blocks of computer instructions, such as those organized into objects, procedures, or functions. However, the executables of an identified module need not be physically together, but may include different instructions stored in different locations that, when logically combined together, comprise the module and achieve the module's stated purpose.

In fact, the module of executable code can be a single instruction or multiple instructions, and can even be distributed on several different code segments, between different programs and on several storage devices. Simply, operational data can be identified and illustrated in this article in the module, and any appropriate form can be provided to embody and be organized in the data structure of any appropriate type. Operational data can be collected as a single data set, or can be distributed on different locations including different storage devices, and can exist at least in part as only electronic signals on a system or network. Modules can be passive or active, including agents that can operate to perform desired functions.

References throughout this specification to "example" mean that a particular feature, structure, or characteristic described in conjunction with the example is included in at least one embodiment of the present invention. Therefore, the phrase "in one example" appearing throughout this specification is not necessarily all referring to the same embodiment.

As used herein, for convenience, multiple items, structural elements, constituent elements and/or materials can be presented in a common list. However, these lists should be interpreted as if each member of the list is identified as a separate and unique member. Therefore, in the absence of contrary instructions, any one member of such a list should not be interpreted as the actual equivalent of the other members based solely on their presentation in a common group with any other presentation of the same list. In addition, various embodiments and examples of the present invention can be referred to together with the substitutes of their various components in this article. It will be understood that such embodiments, examples and substitutes are not interpreted as actual equivalents to each other, but are considered to be separate and autonomous representations of the present invention.

In addition, the described features, structures or characteristics can be combined in one or more embodiments by any appropriate means. In some specifications, many specific details (such as layout examples, distances, network examples, etc.) are provided to provide a complete understanding of the embodiments of the invention. However, those skilled in the art will recognize that the invention can be implemented without one or more specific details or by utilizing other methods, components, layouts, etc. In other examples, known structures, materials or operations are not shown or described in detail to avoid blurring the aspects of the invention.

Although the foregoing examples illustrate the principles of the present invention in one or more specific applications, it will be apparent to those skilled in the art that many modifications in form, usage, and implementation details may be made without exercising the inventive power and without departing from the principles and concepts of the invention. Therefore, it is not intended that the invention be limited except as set forth in the appended claims.

Claims (17)

1. A Transmission Control Protocol (TCP) receiver, the TCP receiver having computer circuitry for reducing latency in data stream transmission, said computer circuitry being configured to: At the TCP receiver buffer, multiple TCP segments are received from network elements; Missing TCP segments are detected based on out-of-order TCP segments received from the plurality of TCP segments; Based on context information associated with the data stream transmission, the TCP receiver determines that the missing TCP segment to be received before the out-of-order TCP segment can be discarded, wherein the context information includes network layer context information and application layer context information. Based on the context information, a false ACK message is sent to the network element, acknowledging that the missing TCP segment was received at the TCP receiver. The false ACK message includes a request for a TCP segment logically following the out-of-order TCP segment that is to be transmitted to the TCP receiver. The out-of-order TCP segment, which does not contain the missing TCP segment, is provided from the TCP receiver buffer to the display device. 2. The receiver of claim 1, wherein the computer circuitry is further configured to: discard the missing TCP segment based on the context information when the missing TCP segment is not received within a predetermined time threshold. 3. The receiver of claim 1, wherein the network layer context information includes at least one of the following: an explicit loss indication of MAC layer packet loss, a loss related to congestion identification through TCP receiver buffer content analysis, or explicit network congestion information. 4. The receiver as claimed in claim 1, wherein the application layer context information includes at least one of the following: buffer state, frame type, saliency of video frames, type of video content, device context information, and user context information. 5. The receiver of claim 1, wherein the computer circuitry is further configured to send an acknowledgment message to the network element requesting that the missing TCP segment be retransmitted to the TCP receiver, wherein the missing TCP segment cannot be discarded based on the context information. 6. The receiver of claim 1, wherein the computer circuitry is further configured to: discard the missing TCP segment when the context information indicates that a wireless link error causes the out-of-order TCP segment to be delivered out of order to the TCP receiver buffer. 7. The receiver of claim 1, wherein the computer circuitry is further configured to: determine that the missing TCP segment should not be discarded when the context information indicates that network congestion causes the out-of-order TCP segment to be delivered out of order to the TCP receiver buffer. 8. The receiver of claim 1, wherein the TCP receiver operates in a user equipment (UE). 9. A method for reducing latency in data stream transmission at a wireless device, the method comprising: Based on out-of-order data segments received from multiple data segments from network elements in a wireless network, missing data segments are detected, wherein the data segments are data segments based on the Transmission Control Protocol (TCP). Based on context information associated with the data stream transmission, it is determined that the missing data segment to be received before the out-of-order data segment can be discarded, wherein the context information includes network layer context information and application layer context information. Based on the context information, a false ACK message is sent to the network element in the wireless network. This false ACK message acknowledges that the missing data segment was received at the wireless device. The false ACK message includes a request for a data segment logically following the out-of-order data segment that is to be transmitted to the wireless device. The out-of-order data fragments, excluding the missing data fragments, are provided for display at the wireless device. 10. The method of claim 9, further comprising: discarding the missing data segment when the missing data segment is not received within a time threshold. 11. The method of claim 10, further comprising: dynamically configuring the time threshold based on network layer context information and application layer context information. 12. The method of claim 9, further comprising: identifying context information at the wireless device. 13. The method of claim 9, further comprising: identifying context information at the network element in the wireless network. 14. The method of claim 9, further comprising: receiving the plurality of data segments at a TCP receiver, wherein the TCP receiver is included in the wireless device. 15. A wireless device for transmitting data streams, the device comprising: A receiving device configured to receive multiple TCP segments from a network element at a Transmission Control Protocol (TCP) receiver buffer; A detection device configured to detect missing TCP segments based on out-of-order TCP segments received among the plurality of TCP segments; A fragment discarding device configured to discard, based on context information associated with the data stream transmission, the missing TCP fragments to be received before out-of-order data fragments, wherein the context information includes network layer context information and application layer context information; An acknowledgment device is configured to send a false acknowledgment message to the network element based on the context information, the false acknowledgment message confirming that the missing TCP segment has been received at the TCP receiver, wherein the false acknowledgment message includes a request for a TCP segment that logically follows the out-of-order TCP segment and is to be transmitted to the TCP receiver. 16. The wireless device of claim 15, further comprising a fragment delivery means configured to provide the out-of-order TCP fragment, which does not contain the missing TCP fragment, from the TCP receiver buffer to the display device. 17. The wireless device of claim 15, wherein the fragment discarding device is further configured to discard the missing TCP fragment based on the context information and the fact that the missing TCP fragment was not received within a time threshold.