CN108900906A - A kind of delay control method and device of live video stream - Google Patents

Abstract

The present invention discloses a delay control method and device for a live video stream, wherein the method includes: detecting the current play delay of the live video stream, and comparing the current play delay with a specified delay range, When the current playback delay is outside the specified delay range, set the current playback multiple speed; based on the current playback delay, the specified delay range and the playback multiple speed, determine the multiple speed playback duration ; Play the live video stream at the double speed of playback, and play the live video stream at a normal speed after the playing time at the double speed reaches the playing time at the double speed. The technical solution provided by this application can control the playback delay within a reasonable range without affecting the user's viewing experience.

Description

A delay control method and device for live video streaming

technical field

The invention relates to the technical field of video playback, in particular to a delay control method and device for live video streaming.

Background technique

With the rise of the live broadcast industry, the playback technology of live video streams is also constantly evolving. Due to the strong real-time nature of live video streaming, the impact of playback delay on live video streaming is particularly significant. If the playback delay is too short, the live video stream may freeze due to network fluctuations and other reasons; if the playback delay is too long, it will affect the real-time performance of the live video stream. Therefore, when playing a live video stream at present, it is necessary to control the playback delay within a reasonable range.

At present, the playback delay is usually maintained by discarding the live video stream data or pausing the live video stream. Specifically, if the current playback delay is too long, resulting in a mismatch between the video content watched by the user and the real-time video content, part of the live video stream data will be discarded, thereby reducing the current playback delay. If the current playback delay is too short, in order to prevent the video from freezing, the live video stream will be paused for a period of time to increase the playback delay and ensure the smoothness of the live video stream.

It can be seen from the above that the methods of controlling playback delay in the prior art will result in incomplete or incoherent video content, which usually affects the viewing experience of the user. Therefore, there is an urgent need for a playback delay control method that does not affect the user's viewing experience.

Contents of the invention

The purpose of this application is to provide a delay control method and device for live video streaming, which can control the playback delay within a reasonable range without affecting the user's viewing experience.

In order to achieve the above purpose, the present application provides a delay control method for live video streams on the one hand, the method comprising: detecting the current playback delay of the live video stream, and comparing the current playback delay with the specified delay range In contrast, when the current playback delay is outside the specified delay range, set the current playback speed; based on the current playback delay, the specified delay range and the playback speed, determine the speed Playing duration: playing the live video stream at the double speed of playing, and playing the live video stream at the normal speed after the playing time at the double speed reaches the playing time at the double speed.

In order to achieve the above purpose, another aspect of the present application also provides a delay control device for live video streams, the device includes: a playback speed setting unit, used to detect the current playback delay of the live video stream, and set the current The playback delay is compared with the specified delay range, and when the current playback delay is outside the specified delay range, the current playback double speed is set; the double speed playback duration determination unit is used to determine the playback time based on the current playback Delay, the specified delay range and the playback double speed determine the duration of double speed playback; the double speed playback unit is used to play the live video stream according to the playback double speed, and play the time according to the playback double speed After the double-speed playback duration is reached, the live video stream is played at a normal speed.

In order to achieve the above object, another aspect of the present application also provides a delay control device for live video streaming, the device includes a memory and a processor, the memory is used to store a computer program, and the computer program is controlled by the processor When executing, implement the above method.

It can be seen from the above that the technical solution provided by this application can detect the current playback delay of the live video stream, and when the current playback delay exceeds the specified delay range, the playback speed of the live video stream can be adjusted. Specifically, if the current playback delay is greater than the maximum delay in the specified delay range, you can set a playback double speed greater than the normal rate to speed up the playback of the live video stream, thereby reducing the playback delay to within the specified delay range. Conversely, if the current playback delay is less than the minimum delay in the specified delay range, you can set a playback speed that is lower than the normal rate to make the live video stream play slowly, so that the playback delay can be increased to within the specified delay range. When setting the playback double speed, you can also set the double speed playback duration. When the playback time at the double speed reaches the double speed playback time, it means that the playback delay is within the specified delay range, so that the playback double speed can be restored to the normal speed. And play the live video stream at normal rate. It can be seen that the technical solution provided by this application can dynamically adjust the playback rate of the live video stream according to the current playback delay, so as to catch up with the playback progress or delay the playback progress, and achieve the effect of controlling the playback delay. In practical applications, as long as the playback speed is within a reasonable range, even if the live video stream is played faster or delayed, the user will not feel uncomfortable, and by adjusting the playback speed, the data of the live video stream will not be lost. The integrity of the video content is guaranteed. Therefore, the technical solution provided by this application can control the playback delay within a reasonable range without affecting the viewing experience of the user.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

Fig. 1 is a schematic diagram of a delay control method for a live video stream in an embodiment of the present invention;

Fig. 2 is a schematic diagram of dividing a live video stream into video slices in an embodiment of the present invention;

Fig. 3 is a schematic flow diagram of determining the double-speed playback and the double-speed playback duration in an embodiment of the present invention;

4 is a functional block diagram of a delay control device for a live video stream in an embodiment of the present invention;

5 is a schematic structural diagram of a delay control device for a live video stream in an embodiment of the present invention;

Fig. 6 is a schematic structural diagram of a computer terminal in an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the implementation manner of the present invention will be further described in detail below in conjunction with the accompanying drawings.

Embodiment one

The present application provides a method for controlling the delay of a live video stream. The method can be applied to a client, and the client can be a terminal device used by a user. For example, the client may be a smart phone, a tablet computer, a desktop computer, a notebook computer, a smart wearable device (virtual reality glasses, a smart watch, etc.), a video-on-demand device, and the like. The client may also be software running on the above-mentioned terminal device. For example, the client may be a player running on a terminal device.

Please refer to FIG. 1 , the method for controlling the delay of a live video stream provided by the present application may include the following steps.

S1: Detect the current playback delay of the live video stream, and compare the current playback delay with the specified delay range. If the current playback delay is outside the specified delay range, set the current playback speed.

In this embodiment, the live video stream may be recorded by the anchor client and then pushed to the streaming server, and the streaming server may respond to the loading request of the user client and send the live video stream to the user client. In this way, the user client can play the live video stream.

In practical applications, the streaming media server may sequentially divide the live video stream pushed by the anchor client into multiple video slices according to a specified time interval. For example, please refer to FIG. 2. After receiving the pushed live video stream, the streaming server may divide the live video stream into video slices with a duration of 1 second. After the streaming media server divides the live video stream into video slices, it may also assign numbers to each video slice according to the order in which the video slices are divided. For example, in FIG. 2 , the number of the video slice obtained by the first division may be 1, the number of the video slice obtained by the second division may be 2, and so on. When the streaming server divides the live video stream into video slices, it may update the index list of the live video stream, and the index list may include the arrangement sequence and the corresponding number of each video slice. For example, the index list can be an m3u8 file. When the user client requests to load the live video stream, it can load the index list from the streaming media server, and request the streaming media server to download corresponding video slices according to the index list.

In this embodiment, when the user client is playing a live video stream, it may be affected by network fluctuations or human operations, which may cause the currently played target video slice to be far away from the latest video slice that the streaming media server can provide. For example, the target video slice currently played by the user client is the video slice numbered 5, and the latest video slice that the streaming media server can currently provide is the video slice numbered 11. In view of this, in this implementation manner, the current playback delay of the live video stream can be detected. Specifically, the currently playing target video slice may be determined, and the latest video slice that has been divided in the server may be determined. Then, the interval between the target video slice and the latest video slice may be used as the current playback delay of the live video stream. Specifically, when determining the interval between the target video slice and the latest video slice, the first number of the target video slice and the second number of the latest video slice can be read from the index list, respectively. number, and then the difference between the first number and the second number can be calculated, and the product of the difference and the specified time interval can be used as the difference between the target video slice and the latest video slice interval of time. For the above example, the number of the target video slice is 5, and the number of the latest video slice is 11. According to the time interval of 1 second, when the user client is currently playing the live video stream, it will generate about 6 seconds. Playback delay; if the video slice is divided according to the time interval of 2 seconds, then there will be a delay of about 12 seconds.

In this implementation manner, after the current playback delay of the live video stream is detected, the current playback delay may be compared with a specified delay range. The specified delay range can be a reasonable delay range preset, and the specified delay range can have a minimum delay and a maximum delay, and the playback delay between the minimum delay and the maximum delay is all It is a normal delay. On the contrary, if the current playback delay is less than the minimum delay, or the current playback delay is greater than the maximum delay, it indicates that the current playback delay is outside the specified delay range, which is required Adjusted playback delay.

Of course, in practical applications, the normal delay may not be a delay range, but a specific value. For example, the normal delay can be set to 6 seconds, and playback delays lower than 6 seconds or higher than 6 seconds are playback delays that need to be adjusted.

In this embodiment, when it is determined that the current playback delay needs to be adjusted, the playback speed of the live video stream can be dynamically adjusted. Under normal circumstances, the live video stream is played at a normal rate, which may be the initial playback rate preset by the player, for example, the normal rate may be 30 frames per second (Frames Per Second, FPS), the Normal speed can be used as 1x speed. Then, according to the current playback delay, different playback speeds can be set. Specifically, when the current playback delay is less than the minimum delay in the specified delay range, a playback multiple speed lower than the normal rate may be set, thereby delaying the playback of the live video stream. And in the case that the current playback delay is greater than the maximum delay in the specified delay range, a playback multiple speed greater than the normal rate can be set, thereby speeding up the playback of the live video stream. For example, the minimum delay in the specified delay range is 4 seconds, and the maximum delay is 8 seconds. If the currently detected playback delay is 2 seconds, then the playback speed can be set to 0.8 times, that is, according to the normal Play at 0.8 times the speed. On the contrary, if the currently detected playback delay is 12 seconds, then the playback speed can be set to 1.2 times, that is, the playback is performed at 1.2 times the normal speed.

S3: Based on the current playback delay, the specified delay range, and the playback multiple speed, determine a multiple-speed playback duration.

In this embodiment, after dynamically adjusting the playback speed according to the current playback delay, it is considered that if the live video stream is always played at this playback speed, it will lead to overcorrection. As the playback time continues to advance, the playback delay is at After returning to normal, it may be outside the specified delay range again. Therefore, in this embodiment, the double-speed playback duration can also be determined, and the double-speed playback duration can be used as the duration for playing the live video stream at the double-speed playback set above. Specifically, referring to FIG. 3 , the double-speed playback duration may be affected by the current playback delay, the specified delay range, and the playback multiple speed. When determining the double-speed playback duration, firstly, the delay difference to be processed can be determined according to the current playback delay and the specified delay range. Specifically, if the current playback delay is greater than the maximum delay in the specified delay range, the difference between the current playback delay and the maximum delay can be used as the delay difference to be processed , and if the current playback delay is less than the minimum delay in the specified delay range, the difference between the minimum delay and the current playback delay may be used as the delay difference to be processed. For example, if the minimum delay in the specified delay range is 4 seconds, the maximum delay is 8 seconds, and the currently detected playback delay is 2 seconds, then 4-2=2 seconds can be used as the delay to be processed difference. For another example, if the currently detected playback delay is 12 seconds, then 12−8=4 seconds may be used as the delay difference to be processed. The determined delay difference needs to be eliminated after playing at the playback double speed set above for a period of time, so that the playback delay returns to normal. Specifically, the multiplied speed difference between the multiplied playback speed and the normal speed may be determined, and the ratio between the delay difference to be processed and the multiplied speed difference may be used as the multiplied speed playback duration. For example, if the playback double speed is 1.2 times, then the difference between the double speed and the normal speed is 0.2, and the delay difference to be processed is 4 seconds, then the determined double speed playback duration can be 20 seconds. That is to say, after 20 seconds of continuous playback at 1.2x speed, the playback delay will be reduced to within the specified delay range.

S5: Play the live video stream at the double speed of playing, and play the live video stream at a normal speed after the playing time at the double speed reaches the playing time at the double speed.

In this embodiment, after the playback speed is set and the playback duration is determined at the double speed, the live video stream can be played at the playback speed. When the live video stream is played at the playback speed, the playback duration can be counted. When the playback time reaches the above-mentioned double speed playback duration, the playback speed can be restored to the normal speed, and the live video stream can continue to be played at the normal speed. Subsequently, the playback delay of the live video stream can be detected again according to the specified period, so as to determine whether the playback delay needs to be adjusted again.

Embodiment two

Please refer to Figure 4, the present application also provides a delay control device for live video streaming, the device includes:

The playback speed setting unit is used to detect the current playback delay of the live video stream, and compare the current playback delay with the specified delay range, and when the current playback delay is outside the specified delay range In the case, set the current playback speed;

A double-speed playback duration determination unit, configured to determine the double-speed playback duration based on the current playback delay, the specified delay range, and the playback multiple speed;

The double-speed playback unit is configured to play the live video stream at the double-speed playback, and play the live video stream at a normal speed after the playback time at the double-speed playback reaches the double-speed playback duration.

In one embodiment, the live video stream is divided into multiple video slices according to specified time intervals in the server; correspondingly, the playback speed setting unit includes:

A video slice determining module, configured to determine the currently playing target video slice, and determine the latest video slice that has been divided in the server;

The playback delay determination module is configured to use the interval between the target video slice and the latest video slice as the current playback delay of the live video stream.

In one embodiment, the video slices are numbered according to the order in which they are divided; correspondingly, the playback delay determination module includes:

A serial number reading module, configured to respectively read the first serial number of the target video slice and the second serial number of the latest video slice;

An interval duration determination module, configured to calculate the difference between the first number and the second number, and use the product of the difference and the specified time interval as the target video slice and the latest video The interval between slices.

In one embodiment, the double-speed playback duration determination unit includes:

A delay difference calculation module, configured to determine the current delay difference to be processed according to the current playback delay and the specified delay range;

The ratio calculation module is used to determine the multiplied speed difference between the playback multiplied speed and the normal speed, and use the ratio between the delay difference to be processed and the multiplied speed difference as the multiplied speed playback duration.

In one embodiment, the delay difference calculation module is used to calculate the current playback delay and the maximum delay in the case that the current playback delay is greater than the maximum delay in the specified delay range. The delay difference is used as the delay difference to be processed; and when the current playback delay is less than the minimum delay in the specified delay range, the minimum delay and the current The difference in playback delay is used as the delay difference to be processed.

Referring to FIG. 5, the present application also provides a delay control device for live video streaming, the device includes a memory and a processor, the memory is used to store a computer program, and when the computer program is executed by the processor, The methods described above can be implemented.

Please refer to FIG. 6 , in this application, the technical solutions in the above embodiments can be applied to the computer terminal 10 shown in FIG. 6 . The computer terminal 10 may include one or more (only one is shown in the figure) processors 102 (the processors 102 may include but not limited to processing devices such as microprocessor MCU or programmable logic device FPGA, etc.), a A memory 104, and a transmission module 106 for communication functions. Those of ordinary skill in the art can understand that the structure shown in FIG. 6 is only a schematic diagram, which does not limit the structure of the above-mentioned electronic device. For example, computer terminal 10 may also include more or fewer components than shown in FIG. 6 , or have a different configuration than that shown in FIG. 6 .

The memory 104 can be used to store software programs and modules of application software, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104 . The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory that is remotely located relative to the processor 102 , and these remote memories may be connected to the computer terminal 10 through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used to receive or transmit data via a network. The specific example of the above-mentioned network may include a wireless network provided by the communication provider of the computer terminal 10 . In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

It can be seen from the above that the technical solution provided by this application can detect the current playback delay of the live video stream, and when the current playback delay exceeds the specified delay range, the playback speed of the live video stream can be adjusted. Specifically, if the current playback delay is greater than the maximum delay in the specified delay range, you can set a playback double speed greater than the normal rate to speed up the playback of the live video stream, thereby reducing the playback delay to within the specified delay range. Conversely, if the current playback delay is less than the minimum delay in the specified delay range, you can set a playback speed that is lower than the normal rate to make the live video stream play slowly, so that the playback delay can be increased to within the specified delay range. When setting the playback double speed, you can also set the double speed playback duration. When the playback time at the double speed reaches the double speed playback time, it means that the playback delay is within the specified delay range, so that the playback double speed can be restored to the normal speed. And play the live video stream at normal rate. It can be seen that the technical solution provided by this application can dynamically adjust the playback rate of the live video stream according to the current playback delay, so as to catch up with the playback progress or delay the playback progress, and achieve the effect of controlling the playback delay. In practical applications, as long as the playback speed is within a reasonable range, even if the live video stream is played faster or delayed, the user will not feel uncomfortable, and by adjusting the playback speed, the data of the live video stream will not be lost. The integrity of the video content is guaranteed. Therefore, the technical solution provided by this application can control the playback delay within a reasonable range without affecting the viewing experience of the user.

Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general-purpose hardware platform, and of course can also be implemented by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic discs, optical discs, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (13)

1. a kind of delay control method of live video stream, which is characterized in that the method includes：

The currently playing delay of live video stream is detected, and the currently playing delay is compared with specified reference time delay, In the case where the currently playing delay is except the specified reference time delay, current broadcasting speed is set；

Based on the currently playing delay, the specified reference time delay and the broadcasting speed, speed playing duration is determined；

The live video stream is played out according to the broadcasting speed, and according to it is described broadcasting speed play when be up to To after the speed playing duration, the live video stream is played according to normal speed.

2. the method according to claim 1, wherein the live video stream is in the server according to specified time Interval is divided into multiple video segments；Correspondingly, the currently playing delay for detecting live video stream includes：

It determines the target video slice being currently played, and determines the newest video segment for completing to divide in the server；

Using interval duration the working as the live video stream between the target video slice and the newest video segment Preceding broadcast time-delay.

3. according to the method described in claim 2, it is characterized in that, the video segment is compiled according to divided sequence Number；Correspondingly, the interval duration between the target video slice and the newest video segment determines in the following way：

The first number of the target video slice and the second number of the newest video segment are read respectively；

Calculate the difference between first number and second number, and by the difference and the specified time interval Product is as the interval duration between the target video slice and the newest video segment.

4. the method according to claim 1, wherein including minimum delay and maximum in the specified reference time delay Delay；Correspondingly, the currently playing delay, which is in except the specified reference time delay, includes：

The currently playing delay is less than the minimum delay or the currently playing delay is greater than the maximum delay.

5. method according to claim 1 or 4, which is characterized in that current broadcasting speed, which is arranged, includes：

In the case where the currently playing delay is less than the minimum delay in the specified reference time delay, setting is less than positive constant speed The broadcasting speed of rate；

It is described it is currently playing delay be greater than the specified reference time delay in maximum delay in the case where, setting be greater than it is described just The broadcasting speed of constant speed rate.

6. the method according to claim 1, wherein based on the currently playing delay, the specified delay model It encloses and the broadcasting speed, determines that speed playing duration includes：

According to the currently playing delay and the specified reference time delay, currently pending time delayed difference value is determined；

Determine the speed difference between the broadcasting speed and normal speed, and by the time delayed difference value to be processed and described times Ratio between speed difference value is as the speed playing duration.

7. according to the method described in claim 6, it is characterized in that, determining that currently pending time delayed difference value includes：

If the currently playing delay is greater than the maximum delay in the specified reference time delay, by the currently playing delay and institute The difference of maximum delay is stated as the time delayed difference value to be processed；

If the currently playing delay is less than the minimum delay in the specified reference time delay, the minimum delay is worked as with described The difference of preceding broadcast time-delay is as the time delayed difference value to be processed.

8. a kind of overrun control of live video stream, which is characterized in that described device includes：

Play speed setting unit, for detecting the currently playing delay of live video stream, and will the currently playing delay and Specified reference time delay compares, in the case where the currently playing delay is except the specified reference time delay, setting Current broadcasting speed；

Speed playing duration determination unit, for based on the currently playing delay, the specified reference time delay and described broadcasting Speed is put, determines speed playing duration；

Speed broadcast unit for playing out the live video stream according to the broadcasting speed, and is broadcast according to described Put speed broadcasting duration reach the speed playing duration after, play the live video stream according to normal speed.

9. device according to claim 8, which is characterized in that the live video stream is in the server according to specified time Interval is divided into multiple video segments；Correspondingly, the broadcasting speed setting unit includes：

Video segment determining module for determining the target video slice being currently played, and determines complete in the server At the newest video segment of division；

Broadcast time-delay determining module, for making the interval duration between the target video slice and the newest video segment For the currently playing delay of the live video stream.

10. device according to claim 9, which is characterized in that the video segment is compiled according to divided sequence Number；Correspondingly, the broadcast time-delay determining module includes：

Number read module, for read respectively the target video slice the first number and the newest video segment the Two numbers；

It is spaced duration determining module, for calculating the difference between first number and second number, and by the difference When value and the product at the specified time interval are as interval between the target video slice and the newest video segment It is long.

11. device according to claim 8, which is characterized in that the speed playing duration determination unit includes：

Time delayed difference value computing module, for determining currently wait locate according to the currently playing delay and the specified reference time delay The time delayed difference value of reason；

Ratio calculation module, for determining the speed difference between the broadcasting speed and normal speed, and will be described to be processed Time delayed difference value and the speed difference between ratio as the speed playing duration.

12. device according to claim 11, which is characterized in that the time delayed difference value computing module is used for described current In the case that broadcast time-delay is greater than the maximum delay in the specified reference time delay, by the currently playing delay and the maximum The difference of delay is as the time delayed difference value to be processed；And it is less than the specified reference time delay in the currently playing delay In minimum delay in the case where, the difference of the minimum delay and the currently playing delay is prolonged as described to be processed Time difference value.

13. a kind of overrun control of live video stream, which is characterized in that described device includes memory and processor, institute Memory is stated for storing computer program, when the computer program is executed by the processor, realize such as claim 1 to Any method in 7.