CN108574882A - A video transmission method and device - Google Patents

Abstract

The embodiment of the present application discloses a kind of video transmission method and device, is related to transmission technique field, to save transfer resource and cache resources.The video includes main coding stream and auxiliary encoding stream, and auxiliary encoding stream is used to improve the code check of chief editor's code stream, and chief editor's code stream includes at least two mainstream fragments, and auxiliary encoding stream includes at least two secondary flow fragments, and each mainstream fragment is corresponding with a secondary flow fragment.This method includes：The first mainstream fragment request is sent to server, wherein for the first mainstream fragment request for asking the first mainstream fragment, the first mainstream fragment is a mainstream fragment at least two mainstream fragments；Receive the first mainstream fragment that server is sent；If current real-time network situation meets preset condition, the first secondary flow fragment request is sent to server, wherein the first secondary flow fragment request is for asking the corresponding first secondary flow fragment of the first mainstream fragment；Receive the first secondary flow fragment that server is sent.

Description

A video transmission method and device

technical field

The present application relates to the technical field of transmission, and in particular to a video transmission method and device.

Background technique

HLS (HTTP live streaming) is a streaming media transmission protocol based on hypertext transfer protocol (HTTP), which can realize live broadcast and on-demand streaming media. In the HLS on-demand scenario, the server encodes the video content to obtain multiple video files with different bit rates, and then divides each of the multiple video files into multiple segments according to time. When the terminal needs to play the video, it requests the required fragments according to the playing time and current network conditions, and caches, decodes and plays the requested fragments.

In HLS technology, when the segment requested by the terminal is switched from a low bit rate to a high bit rate, the cached but unplayed low bit rate segment will lose its function, resulting in a waste of transmission resources and cache resources.

Contents of the invention

Embodiments of the present application provide a video transmission method and device to save transmission resources and cache resources.

In the first aspect, a video transmission method is provided, the video includes a main coded stream and an auxiliary coded stream, the auxiliary coded stream is used to increase the code rate of the main coded stream, the main coded stream includes at least two main stream slices, and the auxiliary coded stream includes at least There are two auxiliary stream slices, and each main stream slice corresponds to an auxiliary stream slice. The method includes: the terminal sends a first mainstream fragmentation request to the server, wherein the first mainstream fragmentation request is used to request the first mainstream fragmentation, and the first mainstream fragmentation is a mainstream fragmentation of at least two mainstream fragments ;Receive the first mainstream segment sent by the server; if the current real-time network conditions meet the preset conditions, then send the first auxiliary stream segment request to the server, wherein the first auxiliary stream segment request is used to request the first mainstream segment The corresponding first auxiliary stream segment; receiving the first auxiliary stream segment sent by the server. In this technical solution, the terminal requests mainstream fragmentation, and requests auxiliary stream fragmentation as needed; wherein, the auxiliary stream fragmentation is used to increase the code rate of the mainstream fragmentation. In the embodiment of the present application, the main coded stream can be played alone, or the main coded stream and the auxiliary coded stream can be superimposed and reconstructed to be played, so the cached main coded stream will not lose its effect due to buffering the auxiliary coded stream. Compared with the prior art, transmission resources and cache resources can be saved.

Optionally, the terminal communicates with the server through the HTTP/2 protocol. The server may be any server supporting the HTTP/2 protocol, such as a WEB server.

In a possible implementation manner, the method may further include: the terminal superimposes and reconstructs the first main stream segment and the first auxiliary stream segment, and plays the code stream generated after reconstruction. In this way, video playback quality can be improved.

In a possible implementation manner, receiving the first mainstream fragment sent by the server at the terminal may include: receiving the first mainstream fragment sent by the server on a link, where the link may be a link between the terminal and the server any of the links. In this case, receiving the first auxiliary stream segment sent by the server includes: receiving the first auxiliary stream segment sent by the server on the link. In this way, both the terminal and the server only need to maintain one link, so the implementation is simple.

In a possible implementation manner, the method may further include: the terminal sends a second mainstream fragmentation request to the server, where the second mainstream fragmentation request is used to request a second mainstream fragmentation, and the second mainstream fragmentation is at least One of the two mainstream fragments except the first mainstream fragment; receive the second mainstream fragment sent by the server; if the current real-time network conditions meet the preset conditions, send the second secondary stream fragment to the server slice request, wherein, the second auxiliary stream fragment request is used to request the second auxiliary stream fragment corresponding to the second main stream fragment; if the second auxiliary stream fragment request is sent to the server and the second auxiliary stream fragment is received Before the segment, if it is detected that the current real-time network condition does not meet the preset condition, an interrupt request is sent to the server, wherein the interrupt request is used to request to interrupt the transmission process of the second auxiliary stream segment. In this way, the acquisition of the next mainstream segment of the second mainstream segment by the terminal can be reduced to a certain extent, thereby effectively reducing the occurrence of stuttering during the video playback process of the terminal. It can be understood that this possible implementation manner may be implemented independently of any technical solution provided above, and thus, the second mainstream sharding is arbitrary.

In a second aspect, a video transmission method is provided, the video includes a main coded stream and an auxiliary coded stream, the main coded stream includes at least two main stream slices, the auxiliary coded stream includes at least two auxiliary stream slices, each main stream slice Corresponding to an auxiliary stream segment; the method may include: the server receives a first mainstream segment request sent by a terminal, wherein the first mainstream segment request is used to request a first mainstream segment, and the first mainstream segment is at least two One of the main stream fragments; sending the first mainstream fragment to the terminal; receiving the first auxiliary stream fragmentation request sent by the terminal, wherein the first auxiliary stream fragmentation request is used to request the corresponding first mainstream fragmentation The first auxiliary stream segment; the first auxiliary stream segment request is sent by the terminal when the current network condition meets the preset condition; the first auxiliary stream segment is sent to the terminal.

In a possible implementation manner, sending the first mainstream fragment to the terminal may include: sending the first mainstream fragment to the terminal on a link. In this case, sending the first auxiliary stream segment to the terminal may include: sending the first auxiliary stream segment to the terminal on the link.

In a possible implementation manner, the priority of the mainstream segment is higher than the priority of the secondary stream segment corresponding to the mainstream segment. In this case, sending the first auxiliary stream segment to the terminal may include: if it is determined that the first main stream segment has been sent to the terminal, sending the first auxiliary stream segment to the terminal. In this way, it can be guaranteed that the mainstream segment can be transmitted preferentially, thereby ensuring the playback continuity of the mainstream segment.

In a possible implementation manner, the method may further include: the server receives a second mainstream fragmentation request sent by the terminal, where the second mainstream fragmentation request is used to request a second mainstream fragmentation, and the second mainstream fragmentation is One of the at least two mainstream fragments except the first mainstream fragment; sending the second mainstream fragment to the terminal; receiving the second secondary stream fragmentation request sent by the terminal, wherein the second secondary stream fragment The segment request is used to request the second auxiliary stream segment corresponding to the second main stream segment; if the interrupt request sent by the terminal is received before the second auxiliary stream segment request is sent to the terminal, the second auxiliary stream segment is interrupted. transfer process. It can be understood that this possible implementation manner may not depend on any technical solution provided above, but may be executed independently. In this way, the second mainstream sharding is arbitrary.

For the beneficial effects of the technical solutions provided by the second aspect or any possible implementation manner of the second aspect, reference may be made to the above-mentioned first aspect or possible implementation manners of the first aspect, and details are not repeated here.

In a third aspect, a terminal is provided. The video includes a main coded stream and an auxiliary coded stream, and the auxiliary coded stream is used to increase the code rate of the main coded stream. The main coded stream includes at least two main stream slices, and the auxiliary coded stream includes at least two auxiliary coded streams. Stream shards, each mainstream shard corresponds to a secondary stream shard. The terminal may include: a sending unit and a receiving unit. Wherein, the sending unit is configured to send the first mainstream fragmentation request to the server, wherein the first mainstream fragmentation request is used to request the first mainstream fragmentation, and the first mainstream fragmentation is one mainstream among at least two mainstream fragments Fragmentation. The receiving unit is configured to receive the first main stream segment sent by the server. The sending unit is further configured to, if the current real-time network condition satisfies the preset condition, send a first auxiliary stream fragmentation request to the server, wherein the first auxiliary stream fragmentation request is used to request the first auxiliary stream fragmentation corresponding to the first main stream fragmentation. Flow fragmentation. The receiving unit is further configured to receive the first auxiliary stream segment sent by the server.

In a possible implementation manner, the terminal may further include: a reconstruction unit and a playback unit. Wherein, the reconstruction unit is configured to superimpose and reconstruct the first main stream segment and the first auxiliary stream segment. The playback unit is used to play the code stream generated after reconstruction.

In a possible implementation manner, the receiving unit is specifically configured to receive, on a link, the first main stream segment sent by the server; and receive, on the link, the first auxiliary stream segment sent by the server.

In a possible implementation manner, the sending unit may also be configured to send a second mainstream fragmentation request to the server, where the second mainstream fragmentation request is used to request a second mainstream fragmentation, and the second mainstream fragmentation is at least One of the two mainstream slices other than the first mainstream slice. The receiving unit may also be configured to receive the second main stream fragment sent by the server. The sending unit may also be configured to, if the current real-time network condition satisfies a preset condition, send a second auxiliary stream fragmentation request to the server, wherein the second auxiliary stream fragmentation request is used to request the second auxiliary stream fragmentation corresponding to the second main stream fragmentation. Content fragmentation. The sending unit may also be used to send an interruption request to the server if it detects that the current real-time network condition does not meet the preset condition after sending the second auxiliary stream segment request to the server and before receiving the second auxiliary stream segment, Wherein, the interrupt request is used to request to interrupt the transmission process of the second auxiliary stream segment.

In a fourth aspect, a server is provided. The video includes a main encoding stream and an auxiliary encoding stream. The main encoding stream includes at least two main stream fragments, and the auxiliary encoding stream includes at least two auxiliary stream fragments. Each main stream fragment is connected to an auxiliary stream. Stream sharding correspondence. The server may include: a receiving unit and a sending unit. Wherein, the receiving unit is configured to receive the first mainstream fragmentation request sent by the terminal, wherein the first mainstream fragmentation request is used to request the first mainstream fragmentation, and the first mainstream fragmentation is one of at least two mainstream fragments Mainstream sharding. A sending unit, configured to send the first main stream segment to the terminal. The receiving unit is also used to receive the first auxiliary stream fragmentation request sent by the terminal, wherein the first auxiliary stream fragmentation request is used to request the first auxiliary stream fragment corresponding to the first main stream fragment; the first auxiliary stream fragmentation The request is sent by the terminal when the current network condition meets the preset condition. The sending unit is further configured to send the first auxiliary stream segment to the terminal.

In a possible implementation manner, the sending unit is specifically configured to: send the first main stream segment to the terminal on a link; send the first auxiliary stream segment to the terminal on the link.

In a possible implementation manner, the priority of the mainstream segment is higher than the priority of the secondary stream segment corresponding to the mainstream segment; in this case, the sending unit is specifically configured to: if it is determined that the first mainstream segment has been sent to the terminal fragments, send the first auxiliary stream fragments to the terminal.

In a possible implementation manner, the receiving unit is further configured to receive the second mainstream fragmentation request sent by the terminal, where the second mainstream fragmentation request is used to request the second mainstream fragmentation, and the second mainstream fragmentation is at least One of the two mainstream slices other than the first mainstream slice. The sending unit is further configured to send the second main stream segment to the terminal. The receiving unit is further configured to receive a second auxiliary stream fragmentation request sent by the terminal, where the second auxiliary stream fragmentation request is used to request a second auxiliary stream fragment corresponding to the second main stream fragment. The receiving unit is also used to receive an interrupt request sent by the terminal. The server may further include: an interrupt unit, configured to interrupt the transmission process of the second auxiliary stream segment if the receiving unit receives an interrupt request sent by the terminal before the sending unit sends the second auxiliary stream segment request to the terminal.

Optionally, the interrupt requests in the first aspect to the fourth aspect above may include: RST_STREM frame in HTTP/2. In addition, the interrupt request may also be a newly designed frame, or may reuse other frames provided in the prior art, which is not limited in this embodiment of the present application.

In a fifth aspect, a terminal is provided, and the terminal has a function of implementing the terminal behavior in the foregoing method embodiments. This function may be implemented by hardware, or may be implemented by executing corresponding software on the hardware. The hardware or software includes one or more modules corresponding to the above functions.

In a possible implementation, the terminal may include: a processor, a memory, a communication bus, and a communication interface; wherein, the memory is used to store computer-executed instructions, the processor and the memory are connected through the communication bus, and when the terminal is running, the processing The device executes the computer-executed instructions stored in the memory, so that the terminal executes the video transmission method provided by the first aspect or any implementation manner of the first aspect.

A sixth aspect provides a computer-readable storage medium for storing computer software instructions used by the above-mentioned terminal, and when it is run on a computer, the computer can execute the video transmission method in any one of the above-mentioned first aspects.

In a seventh aspect, there is provided a computer program product containing instructions, which, when run on a computer, enable the computer to execute the video transmission method in any one of the above first aspects.

In addition, for the technical effects brought about by any one of the design methods in the third aspect, the fifth aspect and the sixth aspect, please refer to the technical effects brought about by different design methods in the first aspect, which will not be repeated here.

In an eighth aspect, a server is provided, and the server has the function of implementing the server behavior in the above method embodiment. This function may be implemented by hardware, or may be implemented by executing corresponding software on the hardware. The hardware or software includes one or more modules corresponding to the above functions.

In a possible implementation manner, the server may include: a processor, a memory, a communication bus, and a communication interface. Wherein, the memory is used to store computer-executable instructions, and the processor and the memory are connected through a communication bus. When the server is running, the processor executes the computer-executable instructions stored in the memory, so that the server executes any one of the above-mentioned second aspect or the second aspect. The video transmission method provided by the implementation.

In a ninth aspect, a computer-readable storage medium is provided for storing computer software instructions used by the above-mentioned server, and when running on a computer, the computer can execute the video transmission method in any one of the above-mentioned second aspects.

In a tenth aspect, there is provided a computer program product containing instructions, which, when run on a computer, enable the computer to execute the video transmission method in any one of the above-mentioned second aspects.

In addition, for the technical effects brought by any one of the design methods in the fourth aspect, the ninth aspect and the tenth aspect, please refer to the technical effects brought about by different design methods in the first aspect, which will not be repeated here.

Description of drawings

FIG. 1 is a schematic diagram of a system architecture to which the technical solution provided by the embodiment of the present application is applicable;

FIG. 2 is a schematic structural diagram of an encoding server provided in an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a transmission server provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a terminal provided in an embodiment of the present application;

FIG. 5 is an interactive schematic diagram of a video transmission method provided by an embodiment of the present application;

FIG. 6 is an interactive schematic diagram of another video transmission method provided by the embodiment of the present application;

FIG. 7 is a schematic flowchart of a video transmission method provided by an embodiment of the present application;

FIG. 8 is a schematic flow chart of playing a video provided in an embodiment of the present application;

FIG. 9 is an interactive schematic diagram of another video transmission method provided by the embodiment of the present application;

FIG. 10 is a schematic structural diagram of another terminal provided by an embodiment of the present application;

FIG. 11 is a schematic structural diagram of another server provided by the embodiment of the present application.

Detailed ways

In HLS technology, when the segment requested by the terminal is switched from a low bit rate to a high bit rate, the cached but unplayed low bit rate segment will lose its function, resulting in a waste of transmission resources and cache resources. For example, assume that the terminal has requested and cached low-bit-rate segments 1-10, and is currently playing segment 5; In this way, the cached but unplayed low-bit-rate fragment 6 will be useless, thereby causing a waste of transmission resources and cache resources.

Based on this, the embodiment of the present application provides a video transmission method and device, the basic principle of which is: the terminal requests the mainstream fragmentation, and requests the auxiliary stream fragmentation as needed; wherein, the auxiliary stream fragmentation is used to improve the efficiency of the mainstream fragmentation. code rate. In the prior art, fragments of the same content with different code rates exist independently, are transmitted independently, and one of them is played. In this application, the role of the auxiliary coded stream is to increase the code rate of the main coded stream, that is, the main coded stream can be played alone, or the main coded stream and the auxiliary coded stream can be superimposed and reconstructed to be played, so the cached main coded stream The stream will not be disabled by caching the sub-encoded stream. Compared with the prior art, transmission resources and cache resources can be saved.

The primary coded stream and the secondary coded stream involved in the embodiment of the present application may be coded streams obtained by the server after coding according to reconstructive video coding (reconstructive video coding, RVC) technology and the like. Wherein, in the RVC technology, the encoder can generate two streams at the same time, one of which is the main encoding stream and the other is the auxiliary encoding stream. The main encoded stream is a video stream with a lower resolution, which can be directly decoded and played by the player. The role of the auxiliary coded stream is to assist the player to improve the resolution of the main coded stream. For example, the bit rate of the main encoding stream is 720p. By performing super-resolution operations, superimposing the auxiliary encoding stream and then reconstructing, the bit rate can be increased to 1080p, thereby improving the playback quality of the video. Certainly, the embodiment of the present application is not limited to the encoding technique of the RVC technique, and may also be other encoding techniques.

The technical solutions provided by the embodiments of the present application can be applied to the system architecture shown in FIG. 1 , and the system packages shown in FIG. 1 are: an encoding server 11 , a transmission server 12 and a terminal 13 . The encoding server 11 is used to encode video content according to a certain encoding technology, and transmit the encoding result to the transmission server 12 . The transmission server 12 is used to save the encoding result, and send the encoded segment to the terminal 13 when receiving the segment request sent by the terminal 13 . The terminal 13 is configured to send a fragmentation request to the transmission server 12 according to the user's instruction or its own trigger, and decode and play the coded fragmentation received from the transmission server 12 .

It should be noted that FIG. 1 is only an exemplary diagram, and the embodiment of the present application does not limit the number of encoding servers 11, the number of transmission servers 12, and the number of at least one terminal 13. In practical applications, Network deployment can be performed with a number different from that shown in FIG. 1 as required.

Optionally, one or more links may be established between the transmission server 12 and the terminal 13, and the fragmentation request and the fragmentation are transmitted on these links. Wherein, the link here may be an HTTP/2 link, and in this case, the transmission server 12 may be a server supporting HTTP/2. Of course, the specific implementation is not limited to this.

In order to realize the technical solution provided by the embodiment of the present application, as shown in FIG. 2, the encoding server 11 in FIG. Encapsulation and fragmentation module 1105 and auxiliary stream encapsulation and fragmentation module 1106 .

Wherein, the down-sampling module 1101 is used for receiving the original video input stream, and reducing the code rate of the input original video input stream, and outputting the video stream obtained after the code rate reduction to the mainstream encoder 1102 . For example, receive a 4K YUV format video input stream, and reduce the bit rate of the 4K YUV format video input stream to generate a 2K YUV format video stream, and then output the 2K YUV format video stream to the mainstream encoder 1102. Among them, YUV is a coding format of color data.

The mainstream encoder 1102 is used to receive the video stream input by the down-sampling module 1101 , and encode the video stream, such as H.264 encoding, to generate a main encoded stream, and then output the main encoded stream to the mainstream encapsulation and fragmentation module 1105 .

The auxiliary stream information generating module 1103 is used for comparing the video stream output by the down-sampling module 1101 with the main encoded stream output by the mainstream encoder 1102 to generate an auxiliary encoded stream. Wherein, the secondary coded stream includes reconstructed content for improving video quality, such as information for correcting some pixels in the main coded stream.

The mainstream encapsulation and fragmentation module 1105 is used to receive the main encoded stream output by the mainstream encoder 1102, and under the control of the fragmentation control module 1104, fragment and encapsulate the main encoded stream according to time points to obtain at least two mainstream fragments , such as TS (transport stream) fragments or mp4 fragments, etc., and then output the at least two mainstream fragments.

The auxiliary stream encapsulation and slicing module 1106 is used to receive the auxiliary encoded stream output by the auxiliary stream generating module 1103, and under the control of the fragmentation control module 1104, fragment and encapsulate the auxiliary encoded stream according to time to obtain at least two auxiliary stream segments slice, and then output the at least two auxiliary stream slices. Among them, the format of the auxiliary stream segment can be a custom format, each auxiliary stream segment corresponds to a main stream segment, and the time point of each auxiliary stream segment is the same as that of the corresponding main stream segment, and each auxiliary stream Stream fragmentation is used to increase the code rate of its corresponding mainstream fragmentation.

The fragmentation control module 1104 is used to control the execution process of the main stream encapsulation fragmentation module 1105 and the auxiliary stream encapsulation fragmentation module 1106, and generate a warehouse receipt describing fragmentation (including mainstream fragmentation and auxiliary stream fragmentation) file, and then output the warehouse receipt file. Wherein, the warehouse receipt file may include the time point of each fragment, the corresponding relationship between each mainstream fragment and each mainstream fragment, and so on.

In addition, the encoding server 11 may also include a communication interface for communicating with external devices (such as the transmission server 2). For example, it is used to receive the main stream segment output by the mainstream encapsulation segment module 1105, the auxiliary stream segment output by the auxiliary stream encapsulation segment module 1106, and the warehouse receipt file output by the segment control module 1104; and send these information to A WEB server, wherein the WEB server may be a WEB server supporting the HTTP/2 protocol, and the server may be the transmission server 12 above.

In terms of hardware implementation, the communication interface 1107 can be a transceiver, down-sampling module 1101, mainstream encoder 1102, auxiliary stream information generation module 1103, fragmentation control module 1104, mainstream encapsulation fragmentation module 1105 and auxiliary stream encapsulation fragmentation module 1106 All can be embedded in or independent of the processor of the encoding server 11 in the form of hardware, and can also be stored in the memory of the encoding server 11 in the form of software, so that the processor can call and execute the corresponding operations of the above modules. In addition, besides the transceiver, the processor and the memory, the encoding server 11 may also include a communication bus for interconnecting the transceiver, the processor and the memory.

In order to implement the technical solution provided by the embodiment of the present application, as shown in FIG. 3 , the transmission server 12 in FIG. 1 may include: a communication interface 1201 , a processor 1202 , a memory 1203 and a communication bus 1204 . Wherein, the communication interface 1201 , the processor 1202 and the memory 1203 are connected to each other through a communication bus 1204 .

The communication interface 1201 is a unit for exchanging service data streams between the transmission server 12 and external network elements (such as the encoding server 11 and the terminal 13 ). Specifically, the communication interface 1201 may be used to receive the main stream segments, the auxiliary stream segments and the warehouse receipt file sent by the encoding server 11 . Or, receive the main stream fragmentation request, auxiliary stream encoding request or interrupt request sent by the terminal 13, or reply the requested main stream fragmentation, auxiliary stream fragmentation or warehouse receipt file to the terminal 13. The memory 1203 can be used to store data and/or codes, and the processor 1302 can implement various functions of the transmission server 12 by running or executing program codes stored in the memory 1203 and calling data stored in the memory 1203 .

It can be understood that the transmission server 12 shown in FIG. 3 does not constitute a limitation on the transmission server 12. In actual implementation, the transmission server 12 may also include more components than shown in the figure, or combine certain components, or use different Part placement. For example, the transmission server 12 may also include a power supply (power), a fan (fan), a clock (clock, CLK) and the like.

In order to realize the technical solution provided by the embodiment of the present application, as shown in FIG. Wherein, the communication interface 1301 , the processor 1302 , the memory 1303 , the player 1305 and the display screen 1306 are connected to each other through a communication bus 1304 .

The communication interface 1303 is a unit for exchanging service data flows between the terminal 13 and an external network element (such as the transmission server 12). Specifically, the communication interface 1303 may be used to send a main stream segment request, a secondary stream encoding request, or an interruption request to the transmission server 12, or receive a mainstream segment, secondary stream segment, or warehouse receipt file sent by the transport server. The memory 1303 can be used to store data and/or codes, and the processor 1302 can implement various functions of the terminal 13 by running or executing program codes stored in the memory 1303 and calling data stored in the memory 1303 .

It can be understood that the terminal 13 shown in FIG. 4 does not constitute a limitation on the terminal 13. In actual implementation, the terminal 13 may include more components than shown in the figure, or combine certain components, or arrange different components. For example, the terminal 13 may also include a power supply, a fan, a clock, and the like.

In addition, both the communication interface 1201 and the communication interface 1301 may include a receiving unit and a sending unit, which may specifically be a transceiver or a transmission interface.

The processor 1202 and the processor 1302 may include multiple central processing units (central processing unit, CPU) or multiple network processors (network processing unit, CPU), and may be specific integrated circuits (application specific integrated circuit, ASIC), Or one or more integrated circuits configured to implement the embodiments of the present application, for example: one or more microprocessors (digital signal processor, DSP), or, one or more field programmable gate arrays (field programmable gate array, FPGA), or a multi-core system-on-chip (system on chip, SoC).

Both the memory 1203 and the memory 1303 can be volatile memory (volatile memory), such as: random-access memory (random-access memory, RAM); they can also be non-volatile memory (non-volatile memory), such as: read-only memory (read-only memory, ROM), flash memory (flash memory), hard disk (hard disk drive, HDD), solid-state drive (solid-state drive, SSD), or a combination of the above-mentioned types of memory. Both the communication bus 1204 and the communication bus 1304 can be divided into address bus, data bus, control bus, etc., and can be Ethernet bus, industry standard architecture (industry standard architecture, ISA) bus, peripheral component interconnection (peripheral component, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, and the like. For ease of representation, only one thick line is used to represent each communication bus in FIG. 3 and FIG. 4 , but it does not mean that there is only one bus or one type of bus.

It should be noted that, the encoding server 11 and the transmission server 12 are described above as independent devices. During specific implementation, these two devices can also be integrated together. In addition, any of the two devices A device may also be integrated with other network-side devices, which is not limited in this embodiment of the present application.

For the convenience of description, the following embodiment shows and describes in detail the video transmission method provided by the present invention in the form of steps. Execute on any computer other than 13. In addition, although the logical order of the video transmission method provided by the embodiment of the present application is shown in the method flow chart, in some cases, the steps shown or described may be performed in a different order than here.

It should be noted that the "server" described in the embodiments provided below may specifically be the transmission server 12 described above. "And/or" in this article is just a relationship describing the relationship between related objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship. "Plurality" herein means two or more.

As shown in FIG. 5 , it is a schematic diagram of interaction of a video transmission method provided by the embodiment of the present application. The method shown in FIG. 5 may include the following steps S101-S104:

S101: The terminal acquires the warehouse receipt file of the video to be played from the server.

The video to be played may be any video requested by the terminal from the server. The video may include a main coded stream and an auxiliary coded stream. The auxiliary coded stream is used to increase the code rate of the main coded stream. The main coded stream includes at least two main stream slices, and the auxiliary encoded stream includes at least two auxiliary stream slices. A slice corresponds to a secondary stream slice. Wherein, the generation process of the main coded stream and the auxiliary coded stream and the fragmentation process can refer to the above, and will not be repeated here.

The warehouse receipt file of the video to be played may include, but is not limited to: information about each of the at least two main stream segments, and information about each of the at least two auxiliary stream segments. For example, the time point of each segment (including the mainstream segment and the auxiliary stream segment), the corresponding relationship between the mainstream segment and the auxiliary stream segment, and so on. The time point of each mainstream segment is the same as the time point of its corresponding auxiliary stream segment.

When the user needs to play a video, he can send instruction information to the terminal, such as through touch screen operation, text input, voice input, etc.; after the terminal detects the instruction information, if it determines that the terminal has established a connection with the server connection, request the warehouse receipt file of the video to be played from the server; if it is determined that the terminal has not established a connection with the server, establish a connection with the server first, and then request the warehouse receipt file of the time-frequency to be played from the server . For the process of establishing a connection between the terminal and the server, and the specific implementation process of the terminal requesting the warehouse receipt file from the server, reference may be made to the prior art, which will not be repeated here.

S102: The terminal parses the warehouse receipt file, and obtains the time point of each segment of the video to be played, the corresponding relationship between each mainstream segment and auxiliary stream segment, etc.

S103: The terminal acquires the mainstream segment of the video to be played from the server, and when the current real-time network condition meets the preset condition, acquires the auxiliary stream segment corresponding to the mainstream segment; and caches each acquired segment.

The terminal can determine which mainstream segment in the time-frequency to be played is requested from the server according to the detected indication information sent by the user, and then, after receiving the mainstream segment returned by the server, the terminal can request the server for the mainstream segment. The next mainstream segment of the video, and so on, until the terminal requests the server for the last segment of the video to be played. For example, assuming that the video to be played includes 10 mainstream segments, then the mainstream segment corresponding to the playback start moment indicated by the indication information sent by the user may be the first mainstream segment or the fifth mainstream segment. film etc. If the mainstream segment corresponding to the indicated playback start time is the fifth mainstream segment, the terminal may request the server for the fifth mainstream segment, and after receiving the fifth mainstream segment replied by the server, Request the server for the 6th mainstream fragment, and so on until the terminal requests the 10th mainstream fragment.

The current network status is not static, and may change in real time as factors such as environment and load change. The terminal can detect the current network status periodically, and when the detection period is short, it can be considered as real-time detection of the current network status. Wherein, detecting the current network status may include but not limited to detecting at least one of the following parameters: network delay, network packet loss rate and network fluctuation system. For the specific detection process, reference may be made to the prior art, which will not be repeated here.

It should be noted that, in the technical solutions provided by the embodiments of the present application, the current real-time network status refers to the detected network status when the terminal executes the action of detecting the network status. That is to say, the meaning of the "current real-time network status" involved in the different steps of the embodiments of the present application needs to be determined according to the actual implementation or the actual context, and the same word "current real-time network status" in different steps cannot be considered "Status" indicates the network status measured at the same time. Similarly, the "current playing moment" refers to the moment when the terminal is playing.

The current real-time network condition satisfies the preset condition, which can be understood as that the quality of the current network condition is relatively good, that is, the communication quality is relatively good. In this case, the terminal can request the auxiliary stream segment from the server to superimpose and reconstruct the auxiliary stream segment with the main stream segment, so as to increase the code rate of the mainstream segment corresponding to the auxiliary stream segment, thereby improving the viewing experience of the user. Video experience. The current real-time network condition satisfies the preset condition, which may include at least one of the following: the current network delay is less than or equal to the first threshold, the current network packet loss rate is less than or equal to the second threshold, and the current network fluctuation coefficient is less than or equal to the third threshold. The embodiment of the present application does not limit specific values and how to take the first threshold, the second threshold, and the third threshold.

In this embodiment of the present application, the terminal can independently cache the main stream slices and auxiliary stream slices, for example, open up two specific storage spaces in the terminal, one part of the storage space is used to store the main stream slices, and the other part of the storage space is used for Store content fragments. Alternatively, the terminal may cache the mainstream segment and the auxiliary stream segment integratedly, for example, open a specific storage space in the terminal, and store the mainstream segment and the auxiliary stream segment in the storage space.

The following describes the process for the terminal to obtain a mainstream segment and its corresponding auxiliary stream segment. As shown in Figure 6, the method includes the following steps S11-S16:

S11: The terminal sends a first mainstream fragmentation request to the server, where the first mainstream fragmentation request is used to request the first mainstream fragmentation.

The first mainstream segment request may include an identifier of the first mainstream segment, for example, a time point of the first mainstream segment, or an index of the first mainstream segment. The first mainstream segment may be any one of the at least two mainstream segments, specifically, it may be from the mainstream segment corresponding to the playback start moment indicated by the indication information to the last segment included in the video to be played. Any mainstream shard in a mainstream shard.

S12: The server receives the first mainstream fragmentation request sent by the terminal, and sends the first mainstream fragmentation to the terminal according to the first mainstream fragmentation request.

S13: The terminal receives the first mainstream segment sent by the server, and caches the first mainstream segment.

S14: If the current real-time network condition satisfies the preset condition, the terminal sends a first auxiliary stream segment request to the server, wherein the first auxiliary stream segment request is used to request the first auxiliary stream segment corresponding to the first main stream segment .

The first auxiliary stream segment request may include an identifier of the first auxiliary stream segment, for example, a time point of the first auxiliary stream segment, or an index of the first auxiliary stream segment.

S15: The server receives the first auxiliary stream segment request sent by the terminal, and sends the first auxiliary stream segment to the terminal.

S16: The terminal receives the first auxiliary stream segment sent by the server, and caches the first auxiliary stream segment.

S11-S13 is the process of obtaining the first mainstream segment, and S14-S16 is the process of obtaining the first auxiliary stream segment; the embodiment of the present application does not limit the execution order of S11-S13 and S14-S16, for example, you can first Execute S11-S13 and then execute S14-S16, or execute S14-S16 first and then execute S11-S13, or execute S14-S16 during the execution of S11-S13. That is to say, the process of the terminal acquiring a mainstream segment and the process of acquiring the auxiliary stream segment corresponding to the mainstream segment may not be sequenced.

The following describes the process in which the terminal obtains multiple mainstream segments and the auxiliary stream segment corresponding to each mainstream segment.

For the terminal, it generally distinguishes the order of requesting the main stream fragments according to the time point; and, it can distinguish the order of requesting the auxiliary stream fragments according to the time point, or it can not distinguish the order of requesting the auxiliary stream fragments according to the time point. The sequence of the slices. There is no limitation on the request order of the main stream slice and the auxiliary stream slice (ie, the main stream slice and the auxiliary stream slice corresponding to the main stream slice) at the same time point.

For the server, it generally also sends each main stream segment to the terminal according to the time point, and sends each auxiliary stream segment to the terminal according to the time point. For the sending order of the main stream slice and the auxiliary stream slice at the same time point (that is, the main stream slice and the auxiliary stream slice corresponding to the main stream slice), in the embodiment of this application, it can be determined by setting the priority . Optionally, the server and the terminal may agree in advance, or the server and the terminal may agree on the priorities of the main stream segment and the auxiliary stream segment corresponding to the main stream segment through signaling. For example, use HTTP/2's PRIORITY frame to set priorities. For example, when the terminal sends the request for the main stream segment and/or the auxiliary stream segment to the server, it carries the requested priority of the main stream segment and/or the auxiliary stream segment. Optionally, considering that the mainstream segment can be played alone, and the auxiliary stream segment needs to be played together with the mainstream segment, you can set the priority of the mainstream segment to be higher than that of the auxiliary stream segment corresponding to the mainstream segment level, so as to ensure that the mainstream segment can be transmitted first, and then ensure the continuity of the playback of the mainstream segment; that is, the server will send the auxiliary stream segment corresponding to the mainstream segment to the terminal after sending a mainstream segment piece.

Based on this, the process for the terminal to obtain multiple mainstream slices and auxiliary stream slices corresponding to each mainstream slice, as shown in FIG. 7 , may specifically include the following steps S21-S24:

S21: The terminal acquires the i-th mainstream fragment from the server, and caches the i-th mainstream fragment.

Among them, the i-th mainstream segment refers to the i-th mainstream segment of the video to be played, i is an integer greater than or equal to 1, the initial value of i is 1, and the maximum value of i is the mainstream segment contained in the video to be played quantity.

S22: The terminal judges whether to request the auxiliary stream segment corresponding to the i-th mainstream segment.

If yes, execute step S23; if not, execute step S24.

Specifically, when the terminal detects that the current real-time network conditions meet the preset conditions and does not cache the i-th auxiliary stream segment, it determines that it needs to request the i-th auxiliary stream segment; when detecting that the current real-time network status does not meet the Under preset conditions, or when the i-th auxiliary stream segment has been cached, it is determined that there is no need to request the i-th auxiliary stream segment.

S23: The terminal acquires the auxiliary stream segment corresponding to the i-th mainstream segment, and caches the auxiliary stream segment.

After step S23 is executed, step S24 is executed.

S24: The terminal judges whether the i-th mainstream segment is the last mainstream segment to be played.

If so, end. If not, execute i=i+1, and return to step S21.

It can be understood that after the terminal acquires the last mainstream segment and before acquiring the auxiliary stream segment corresponding to the mainstream segment, it may not judge whether the current real-time network status satisfies the preset condition. In addition, after executing the above S21-S24, the terminal has acquired each main stream segment to be played, and auxiliary stream segments corresponding to part or all of the main stream segments. Optionally, the method may further include: the terminal acquiring an unbuffered auxiliary stream segment corresponding to an unplayed main stream segment. For example, assuming that the mainstream segments to be played are sorted according to the time points and are the mainstream segments 1-5, and the mainstream segments 1-5 are in one-to-one correspondence with the auxiliary stream segments 1-5, then the mainstream segments obtained by the terminal and The order of content fragmentation can be: Main stream slice 1, Second stream slice 1, Main stream slice 2, Main stream slice 3, Main stream slice 4, Second stream slice 4, Main stream slice 5, Second stream slice 5. Further, if the video content corresponding to the main stream segment 1 is being played at the current playback moment when the auxiliary stream segment 5 is obtained, the terminal can also obtain the following auxiliary stream segments in sequence: auxiliary stream segment 2 and auxiliary stream segment 3.

It should be noted that, in specific implementation, after obtaining each mainstream segment, if the terminal detects that the current real-time network conditions meet the preset conditions, it will obtain the uncached first auxiliary stream segment corresponding to the unplayed mainstream segment , until the last mainstream segment is obtained, and then, the uncached auxiliary stream segment corresponding to the unplayed mainstream segment is obtained. Alternatively, the terminal may acquire unbuffered auxiliary stream segments corresponding to one or more unplayed main stream segments after acquiring multiple mainstream segments, which is not limited in this embodiment of the present application.

S104: The terminal plays the video according to the cached segments.

After the terminal requests each fragment (including mainstream fragments and secondary stream fragments), it first caches the fragment, and after caching a certain number of mainstream The cached mainstream segment plays the video to be played to ensure the continuity of playback. It should be noted that since the main stream segment and the auxiliary stream segment obtained by the terminal from the server are code streams obtained after encoding by the server, the terminal needs to decode the main stream segment and the auxiliary stream segment before playing. In addition, the embodiment of the present application does not limit the specific value of the certain number and how to take the value.

Understandably, for a video to be played that contains multiple mainstream segments, the terminal can play the cached video while obtaining the video, but for the same mainstream segment, the terminal first obtains the mainstream segment and then plays The main stream shard that has been cached.

Based on this, the process of playing video by the terminal, as shown in Figure 8, may specifically include the following steps S31-S35:

S31: The terminal determines the j-th mainstream segment according to the time point of the mainstream segment. Wherein, the jth mainstream segment indicates the mainstream segment that the terminal will play recently, for example, if the terminal is currently playing mainstream segment 1, then the mainstream segment to be played by the terminal most recently is mainstream segment 2.

Among them, the jth mainstream segment refers to the jth mainstream segment of the video to be played, j is an integer greater than or equal to 1, the initial value of j is 1, and the maximum value of j is the mainstream segment contained in the video to be played quantity.

S32: The terminal judges whether the auxiliary stream segment corresponding to the jth mainstream segment is cached.

If yes, execute step S33; if not, execute step S34.

S33: The terminal plays the jth mainstream segment. That is, the video content corresponding to the jth mainstream segment is played.

After step S33 is executed, step S35 is executed.

S34: The terminal superimposes and reconstructs the j-th mainstream segment and the auxiliary stream segment to obtain a composite segment, and plays the composite segment.

After step S34 is executed, step S35 is executed.

It can be understood that the video content corresponding to the video content played in step S33 is the same as the video content corresponding to the synthesized segment played in step S34, but the code rates of the two are different, so the playback quality is different.

Fragmentation and reconstruction of mainstream and superimposed auxiliary streams are key steps in reconstructing video coding. When the encoding server encodes video content and generates mainstream fragments, it will perform bit rate reduction processing. For example, the bit rate of 4K video stream is reduced to generate 2K video stream. Therefore, the resolution of the image frame obtained by the terminal after decoding the mainstream slice is relatively low. Generally, after the terminal decodes the mainstream slice, it will perform super-resolution processing on the decoded image, and increase the bit rate to 4K again; however, during the upgrading process, the time-frequency quality of the original 4K video stream will decrease, so the auxiliary encoding stream can be used for correction, such as supplementing lost details, correcting errors introduced in the encoding process, etc.

S35: The terminal judges whether the jth mainstream segment is the last mainstream segment to be played.

If so, end. If not, execute i=i+1, and return to step S31.

For example, suppose the video to be played includes 10 mainstream segments, and each mainstream segment corresponds to a secondary stream segment, the mainstream segment indicated at the current playback time is the fifth mainstream segment, and the sixth to tenth mainstream fragments, and the 6th to 7th auxiliary stream fragments, then a possible situation is: sequentially play synthetic fragment 6, synthetic fragment 7, and the 8th to 10th mainstream fragments. Among them, synthetic segment 6 is the composite segment obtained by superimposing and reconstructing the sixth mainstream segment and the sixth auxiliary stream segment, and synthetic segment 7 is the seventh mainstream segment and the seventh auxiliary stream segment. Composite slices obtained after the slices are superimposed and reconstructed.

In the video transmission method provided by the embodiment of the present application, the terminal requests the main stream fragmentation, and requests the auxiliary stream fragmentation as needed; wherein, the auxiliary stream fragmentation is used to increase the code rate of the main stream fragmentation. In the embodiment of the present application, the main coded stream can be played alone, or the main coded stream and the auxiliary coded stream can be superimposed and reconstructed to be played, so the cached main coded stream will not lose its effect due to buffering the auxiliary coded stream. Compared with the prior art, transmission resources and cache resources can be saved.

It can be understood that one or more links can be established between the terminal and the server. In the embodiment of this application, the main stream fragment and the secondary stream fragment can be transmitted on two independent links, or they can be transmitted on the same link Transport on the road. If transmission is performed on the same link, both the terminal and the server only need to maintain this link, so the implementation is simple. Exemplarily, the link here may be an HTTP/2 link. That is to say, the terminal uses the same HTTP/2 link when sending the main stream segment request and the auxiliary stream segment request, that is, the main stream segment and the auxiliary stream segment use different HTTP/2 frames, and the two streams are multiplexed into the same HTTP/2 link.

The above steps S11 to S16 provide a process of obtaining the first main stream segment and the first auxiliary stream segment. During this process, an interrupt request can also be set to interrupt the transmission process of the auxiliary stream segment. As shown in FIG. 9 , the following steps S41 to S47 may be specifically included. Wherein, in order to distinguish it from the first mainstream segment, the following uses the second mainstream segment as an example for illustration. It can be understood that the second mainstream segment is arbitrary, that is, the terminal can interrupt the transmission process of any auxiliary stream segment in the time-frequency to be played.

S41: The terminal sends a second mainstream fragmentation request to the server, wherein the second mainstream fragmentation request is used to request a second mainstream fragmentation, and the second mainstream fragmentation is at least two mainstream fragments except the first mainstream fragmentation A mainstream shard other than .

S42: The server receives the second mainstream fragmentation request sent by the terminal, and sends the second mainstream fragmentation to the server according to the second mainstream fragmentation request.

S43: The terminal receives the second mainstream segment sent by the server, and caches the second mainstream segment.

S44: If the current real-time network condition satisfies the preset condition, the terminal sends a second auxiliary stream fragmentation request to the server, wherein the second auxiliary stream fragmentation request is used to request the second auxiliary stream fragmentation corresponding to the second mainstream fragmentation .

Wherein, for specific implementation manners of S41-S44, reference may be made to S11-S14 above.

S45: The server receives the second auxiliary stream fragmentation request sent by the terminal.

S46: If the terminal detects that the current real-time network condition does not meet the preset condition after sending the second auxiliary stream segment request to the server and before receiving the second auxiliary stream segment, send an interrupt request to the server, wherein the interrupt request Used to request to interrupt the transmission process of the second auxiliary stream segment.

Since the terminal can detect the current network status in real time, if the terminal detects that the current real-time network status does not meet the preset condition after sending the second auxiliary stream segment request to the server and before receiving the second auxiliary stream segment, it means At this time, the network condition is poor. If the server sends the second substream segment to the terminal, it may take a long time, which may cause the terminal to freeze during the video playback process. Based on this, the embodiment of the present application provides the terminal to send A technical solution for an interrupt request. Wherein, the terminal request may be a RST_STREM frame in HTTP/2, or a new frame, or other frames, which is not limited in this embodiment of the present application.

It can be understood that if the terminal detects that the current real-time network condition still satisfies the preset condition after sending the second auxiliary stream segment request to the server and before receiving the second auxiliary stream segment, it will not send an interruption request to the server, that is, The server will not receive an interrupt request. In this case, the server sends a request for the second auxiliary stream segment to the terminal, and the terminal receives the second auxiliary stream segment and caches the second auxiliary stream segment. For the specific implementation process, refer to the above.

S47: The server receives the interrupt request sent by the terminal, and in a case where it is determined that the second auxiliary stream segment has not been sent to the terminal, interrupts the transmission process of the second auxiliary stream segment according to the interrupt request.

Specifically, after receiving the interruption request, the server may not respond to the second auxiliary stream segment request message, that is, not send the second auxiliary stream segment to the terminal. If the server has already sent the second auxiliary stream segment to the terminal when receiving the interruption request sent by the terminal, the server may discard the interruption request.

It can be understood that after executing S41-S47, the terminal may not receive the second auxiliary stream segment sent by the server, but the terminal has obtained the second main stream segment, so the terminal can continue to request the second main stream segment from the server The next mainstream shard.

In the technical solution provided by this embodiment, after the terminal has sent the auxiliary stream fragmentation request to the server but before receiving the requested auxiliary stream fragmentation, if the terminal detects that the current real-time network conditions do not meet the preset conditions, it can trigger the interrupt. The transmission process of the requested auxiliary stream fragment. In this way, it can reduce to a certain extent that the terminal acquires the next mainstream segment of the mainstream segment corresponding to the secondary stream segment, thereby effectively reducing the occurrence of stuttering during the video playback process of the terminal.

The foregoing mainly introduces the solution provided by the embodiment of the present application from the perspective of interaction between various network elements. It can be understood that each network element, such as a terminal or a server. In order to realize the above functions, it includes corresponding hardware structures and/or software modules for performing various functions. Those skilled in the art should easily realize that the present application can be implemented in the form of hardware or a combination of hardware and computer software in combination with the units and algorithm steps of each example described in the embodiments disclosed herein. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

The embodiment of the present application may divide the functional modules of the terminal or server according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

For example, in the case of dividing each functional module corresponding to each function, FIG. 10 shows a possible structural diagram of the terminal 100 involved in the foregoing embodiment. The terminal 100 includes a sending unit 1401 and a receiving unit 1402 . Optionally, it may further include: a reconstruction unit 1403 and a playback unit 1404 . Wherein, the sending unit 1401 may be configured to perform the actions performed by the terminal in S11 and S14 in FIG. 6 , the actions performed by the terminal in S41 and S46 in FIG. 9 , and/or other processes for supporting the technology described herein. The receiving unit may be used to perform the actions performed by the terminal in S12 and S15 in FIG. 6 , the actions performed by the terminal in S42 in FIG. 9 , and/or other processes for supporting the technology described herein. The reconstruction unit 1403 may be used to execute the actions reconstructed in S34 in FIG. 8 , and/or to support other processes of the technology described herein. The playing unit 1404 may be used to execute the action played in S34 in FIG. 8 , and/or to support other processes of the technology described herein.

Wherein, all relevant content of each step involved in the above-mentioned method embodiment can be referred to the function description of the corresponding function module, and will not be repeated here.

In the embodiment of the present application, the terminal is presented in the form of dividing each function module corresponding to each function, or the policy entity is presented in the form of dividing each function module in an integrated manner. The "module" here may refer to an application-specific integrated circuit (Application-Specific Integrated Circuit, ASIC), a processor and memory that execute one or more software or firmware programs, an integrated logic circuit, and/or other devices that can provide the above functions . In a simple embodiment, those skilled in the art can imagine that the terminal can be in the form shown in FIG. 4 . For example, the sending unit 1401 provided above may be a transmitter, the receiving unit 1402 may be a receiver, and the transmitter and the receiver may form a transceiver together, which may specifically be the communication interface 1301 in FIG. 4 . In addition, the reconstruction unit 1403 can be embedded in or independent of the processor of the terminal in the form of hardware (such as the processor 1302 in Figure 4), and can also be stored in the memory of the terminal in the form of software (such as the memory 1303 in Figure 4). )middle. The playing unit 1404 may be the player 1305 and/or the display screen 1306 in FIG. 4 .

Since the terminal provided by the embodiment of the present application can be used to execute the above video transmission method, the technical effect it can obtain can refer to the above method embodiment, and the embodiment of the present application will not repeat it here.

For example, in the case of dividing each functional module corresponding to each function, FIG. 11 shows a possible structural diagram of the server 110 involved in the above embodiment. The server 110 may include a receiving unit 1501 and a sending unit 1502 . Optionally, an interrupt unit 1503 may also be included. Wherein, the receiving unit 1501 may be used to perform the actions performed by the server in S11 and S14 in FIG. 6 , the actions performed by the server in S41 and S44 in FIG. 9 , and/or other processes for supporting the technology described herein. The sending unit 1502 may be configured to perform the actions performed by the server in S12 and S15 in FIG. 6 , the actions performed by the server in S45 and S46 in FIG. 9 , and/or other processes for supporting the technologies described herein. The interrupt unit 1503 may be used to execute the action of S47 in FIG. 9 , and/or to support other processes of the technology described herein.

Wherein, all relevant content of each step involved in the above-mentioned method embodiment can be referred to the function description of the corresponding function module, and will not be repeated here.

In the embodiment of the present application, the server is presented in the form of dividing each functional module corresponding to each function, or the server is presented in the form of dividing each functional module in an integrated manner. The "module" here may refer to a specific ASIC circuit, a processor and memory executing one or more software or firmware programs, an integrated logic circuit, and/or other devices that can provide the above functions. In a simple embodiment, those skilled in the art can imagine that the terminal can be in the form shown in FIG. 3 . For example, the sending unit 1502 provided above may be a transmitter, the receiving unit 1501 may be a receiver, and the transmitter and the receiver may form a transceiver together, which may specifically be the communication interface 1201 in FIG. 3 . In addition, the interrupt unit 1503 may be embedded in or independent of the processor of the terminal in the form of hardware (such as the processor 1202 in Figure 3), and may also be stored in the memory of the terminal in the form of software (such as the memory 1203 in Figure 3). middle.

Since the terminal provided by the embodiment of the present application can be used to execute the above video transmission method, the technical effect it can obtain can refer to the above method embodiment, and the embodiment of the present application will not repeat it here.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or may be a data storage device including one or more servers, data centers, etc. that can be integrated with the medium. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), etc.

Although the present application has been described in conjunction with various embodiments here, however, in the process of implementing the claimed application, those skilled in the art can understand and Other variations of the disclosed embodiments are implemented. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that these measures cannot be combined to advantage.

Although the application has been described in conjunction with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the specification and drawings are merely illustrative of the application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of this application. Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims (22)

1. A video transmission method, characterized in that, the video includes a primary coded stream and a secondary coded stream, the secondary coded stream is used to improve the code rate of the primary coded stream, and the primary coded stream includes at least two main streams Fragmentation, the auxiliary encoded stream includes at least two auxiliary stream fragments, and each main stream fragment corresponds to an auxiliary stream fragment; the method includes: Sending a first mainstream fragmentation request to the server, wherein the first mainstream fragmentation request is used to request a first mainstream fragmentation, and the first mainstream fragmentation is a mainstream fragmentation of the at least two mainstream fragments piece; receiving the first mainstream segment sent by the server; If the current real-time network condition satisfies the preset condition, send a first auxiliary stream fragmentation request to the server, where the first auxiliary stream fragmentation request is used to request the first auxiliary stream fragmentation corresponding to the first main stream fragmentation. stream fragmentation; Receive the first auxiliary stream segment sent by the server. 2. The method of claim 1, wherein, The receiving the first mainstream segment sent by the server includes: receiving the first mainstream segment sent by the server on a link; The receiving the first auxiliary stream segment sent by the server includes: The first auxiliary stream segment sent by the server is received on the link. 3. The method according to claim 1 or 2, characterized in that the method further comprises: Sending a second mainstream fragmentation request to the server, wherein the second mainstream fragmentation request is used to request a second mainstream fragmentation, and the second mainstream fragmentation is a subset of the at least two mainstream fragments except A mainstream segment other than the first mainstream segment; receiving the second mainstream segment sent by the server; If the current real-time network condition satisfies the preset condition, then send a second secondary stream fragmentation request to the server, where the second secondary stream fragmentation request is used to request the second secondary stream fragmentation corresponding to the second mainstream stream Second auxiliary stream fragmentation; If it is detected that the current real-time network condition does not satisfy the preset condition after sending the second auxiliary stream segment request to the server and before receiving the second auxiliary stream segment, then send a request to the server An interrupt request, wherein the interrupt request is used to request to interrupt the transmission process of the second auxiliary stream segment. 4. The method according to claim 3, wherein the interrupt request comprises: a RST_STREM frame in HTTP/2. 5. The method according to any one of claims 1 to 3, wherein the method further comprises: Superimpose and reconstruct the first main stream segment and the first auxiliary stream segment, and play the reconstructed code stream. 6. A video transmission method, characterized in that, the video includes a main coded stream and an auxiliary coded stream, the main coded stream includes at least two main stream slices, and the auxiliary coded stream includes at least two auxiliary stream slices , each mainstream segment corresponds to a secondary stream segment; the method includes: receiving a first mainstream fragmentation request sent by a terminal, wherein the first mainstream fragmentation request is used to request a first mainstream fragmentation, and the first mainstream fragmentation is a mainstream among the at least two mainstream fragments Fragmentation; sending the first mainstream fragment to the terminal; receiving a first auxiliary stream fragmentation request sent by the terminal, wherein the first auxiliary stream fragmentation request is used to request a first auxiliary stream fragment corresponding to the first main stream fragment; the first auxiliary stream The fragmentation request is sent by the terminal when the current network condition meets the preset condition; sending the first auxiliary stream segment to the terminal. 7. The method of claim 6, wherein, The sending the first mainstream segment to the terminal includes: sending the first mainstream fragment to the terminal on a link; The sending the first auxiliary stream fragment to the terminal includes: sending the first auxiliary stream segment to the terminal on the link. 8. The method according to claim 6 or 7, wherein the priority of the mainstream segment is higher than the priority of the auxiliary stream segment corresponding to the mainstream segment; Presentation fragmentation, including: If it is determined that the first main stream segment has been sent to the terminal, send the first auxiliary stream segment to the terminal. 9. The method according to any one of claims 6 to 8, wherein the method further comprises: receiving a second mainstream fragmentation request sent by the terminal, where the second mainstream fragmentation request is used to request a second mainstream fragmentation, and the second mainstream fragmentation is one of the at least two mainstream fragments A mainstream segment other than the first mainstream segment; sending the second mainstream fragment to the terminal; receiving a second auxiliary stream fragmentation request sent by the terminal, where the second auxiliary stream fragmentation request is used to request a second auxiliary stream fragment corresponding to the second main stream fragment; If the interruption request sent by the terminal is received before the second auxiliary stream segment request is sent to the terminal, the transmission process of the second auxiliary stream segment is interrupted. 10. The method according to claim 9, wherein the interrupt request comprises: a RST_STREM frame in HTTP/2. 11. A terminal, wherein the video includes a main coded stream and an auxiliary coded stream, the auxiliary coded stream is used to increase the code rate of the main coded stream, and the main coded stream includes at least two main stream slices, so The auxiliary coded stream includes at least two auxiliary stream fragments, and each main stream fragment corresponds to an auxiliary stream fragment; the terminal includes: a sending unit and a receiving unit; The sending unit is configured to send a first mainstream fragmentation request to a server, wherein the first mainstream fragmentation request is used to request a first mainstream fragmentation, and the first mainstream fragmentation is the at least two mainstream fragments A mainstream shard in the shard; The receiving unit is configured to receive the first mainstream segment sent by the server; The sending unit is further configured to, if the current real-time network condition satisfies a preset condition, send a first auxiliary stream fragmentation request to the server, wherein the first auxiliary stream fragmentation request is used to request the first The first auxiliary stream segment corresponding to the mainstream segment; The receiving unit is further configured to receive the first auxiliary stream segment sent by the server. 12. The terminal according to claim 11, characterized in that, The receiving unit is specifically configured to receive, on a link, the first main stream segment sent by the server; and receive, on the link, the first auxiliary stream segment sent by the server. 13. The terminal according to claim 11 or 12, characterized in that, The sending unit is further configured to send a second mainstream fragmentation request to the server, where the second mainstream fragmentation request is used to request a second mainstream fragmentation, and the second mainstream fragmentation is the at least One of the two mainstream slices except the first mainstream slice; The receiving unit is further configured to receive the second mainstream segment sent by the server; The sending unit is further configured to, if the current real-time network condition satisfies the preset condition, send a second auxiliary stream fragmentation request to the server, wherein the second auxiliary stream fragmentation request is used to request the A second auxiliary stream segment corresponding to the second mainstream segment; The sending unit is further configured to detect that the current real-time network condition does not satisfy the preset condition after sending the second auxiliary stream segment request to the server and before receiving the second auxiliary stream segment conditions, an interruption request is sent to the server, where the interruption request is used to request interruption of the transmission process of the second auxiliary stream segment. 14. The terminal according to claim 13, wherein the interrupt request comprises: a RST_STREM frame in HTTP/2. 15. The terminal according to any one of claims 11 to 14, wherein the terminal further comprises: a reconstruction unit, configured to superimpose and reconstruct the first main stream segment and the first auxiliary stream segment; The playback unit is used to play the code stream generated after reconstruction. 16. A server, wherein the video includes a main encoded stream and an auxiliary encoded stream, the main encoded stream includes at least two main stream slices, the auxiliary encoded stream includes at least two auxiliary stream slices, each main stream The slice corresponds to a secondary stream slice; the server includes: a receiving unit and a sending unit; The receiving unit is configured to receive a first mainstream fragmentation request sent by a terminal, wherein the first mainstream fragmentation request is used to request a first mainstream fragmentation, and the first mainstream fragmentation is the at least two A mainstream shard in the mainstream shard; The sending unit is configured to send the first mainstream segment to the terminal; The receiving unit is further configured to receive a first auxiliary stream segment request sent by the terminal, wherein the first auxiliary stream segment request is used to request a first auxiliary stream segment corresponding to the first main stream segment segment; the first auxiliary stream segment request is sent by the terminal when the current network condition meets a preset condition; The sending unit is further configured to send the first auxiliary stream segment to the terminal. 17. The server according to claim 16, wherein: The sending unit is specifically configured to: send the first main stream segment to the terminal on a link; send the first auxiliary stream segment to the terminal on the link. 18. The server according to claim 16 or 17, wherein the priority of the mainstream segment is higher than the priority of the auxiliary stream segment corresponding to the mainstream segment; The sending unit is specifically configured to: if it is determined that the first main stream segment has been sent to the terminal, send the first auxiliary stream segment to the terminal. 19. The server according to any one of claims 16 to 18, characterized in that, The receiving unit is further configured to receive a second mainstream fragmentation request sent by the terminal, where the second mainstream fragmentation request is used to request a second mainstream fragmentation, and the second mainstream fragmentation is the One of the at least two mainstream slices except the first mainstream slice; The sending unit is further configured to send the second mainstream segment to the terminal; The receiving unit is further configured to receive a second auxiliary stream segment request sent by the terminal, wherein the second auxiliary stream segment request is used to request a second auxiliary stream segment corresponding to the second main stream segment piece; The receiving unit is further configured to receive an interrupt request sent by the terminal; The server also includes: an interrupt unit, configured to interrupt the second auxiliary stream if the receiving unit receives the interrupt request sent by the terminal before the sending unit sends the second auxiliary stream fragmentation request to the terminal; The transmission process of stream fragmentation. 20. The server according to claim 19, wherein the interrupt request comprises: a RST_STREM frame in HTTP/2. 21. A terminal, comprising: a processor, a memory, a communication bus, and a communication interface; The memory is used to store computer-executable instructions, the processor is connected to the memory through the communication bus, and when the terminal is running, the processor executes the computer-executable instructions stored in the memory, so that The terminal executes the video transmission method according to any one of claims 1-5. 22. A server, comprising: a processor, a memory, a communication bus, and a communication interface; The memory is used to store computer-executable instructions, the processor is connected to the memory through the communication bus, and when the server is running, the processor executes the computer-executable instructions stored in the memory, so that The server executes the video transmission method according to any one of claims 6-10.