CN108259815B - Video key frame forwarding method, device and live video system - Google Patents

Abstract

The present invention relates to a video key frame forwarding method, device and live video system, wherein the method includes: obtaining the arrival time required for the arrival of the next key frame of a video source terminal that forwards the video, wherein the key frame is the video source terminal The key frame sent periodically; if the arrival time exceeds the time threshold that allows timeout, send a key frame request to the video source terminal and wait for the key frame to arrive; when the key frame arrives, receive the arrived key frame and forward the received key frame to the video requesting terminal. The above video key frame forwarding method sends a key frame request to the video source terminal when the arrival time required to obtain the next key frame exceeds the time threshold of the allowable timeout, so as to avoid the key frame waiting time being too long, thereby effectively improving the key frame. Frame forwarding efficiency.

Description

Video key frame forwarding method, device and live video system

technical field

The present invention relates to the technical field of live video, in particular to a video key frame forwarding method, device, live video system, computer equipment and computer storage medium.

Background technique

When performing live video, such as a video conference, the terminal that plays the video usually needs to call the video data of the corresponding video source terminal through the server, and the video data generally includes a key frame (I frame) with complete information, a forward prediction frame (P frame) and bidirectionally predicted frame (B frame). However, when the terminal that plays the video acquires the live video of a video source terminal for the first time, it can display the video screen normally and start the live broadcast only after receiving the key frame data.

The traditional key frame processing method is generally that the video source terminal periodically sends key frames, and the server forwards the key frames to the terminal playing the video to achieve the acquisition of the key frames. By increasing the forwarding frequency of the key frames, the screen of the terminal playing the video can be guaranteed. However, compared with the predicted frame, the key frame occupies much more resources. Increasing the forwarding frequency will greatly increase the resource occupation, resulting in a lower forwarding efficiency of the video key frame.

SUMMARY OF THE INVENTION

Based on this, it is necessary to provide a video key frame forwarding method, device, video live broadcast system, computer equipment and computer storage medium that can reduce the resource occupancy rate of key frame forwarding and effectively improve the forwarding efficiency of key frames.

A video key frame forwarding method, comprising the following steps:

Obtain the arrival time required for the arrival of the next key frame of the video source terminal that forwards the video, wherein the key frame is a key frame periodically sent by the video source terminal;

If the arrival time exceeds the time threshold that allows timeout, send a key frame request to the video source terminal, and wait for the key frame to arrive;

When the key frame arrives, the arrived key frame is received and the received key frame is forwarded to the video requesting terminal.

The above video key frame forwarding method, by monitoring the arrival time of the next key frame of the video source terminal, when the arrival time required to obtain the next key frame exceeds the time threshold of the allowable timeout, it will actively send a key frame request to the video The source terminal can effectively avoid too long waiting time for key frames, thereby effectively improving the forwarding efficiency of key frames.

In one embodiment, after the step of obtaining the arrival time required for the arrival of the next key frame of the video source terminal of the video to be forwarded, the step further includes: if the arrival time does not exceed the time threshold allowed for overtime, waiting for the key frame frame arrives. Before requesting a key frame, the server will first compare the arrival time of the next key frame and the server's allowable key frame timeout time, so as to choose whether to continue waiting for the next key frame or actively request the key frame, which can effectively reduce the actual waiting time before the key frame arrives. time to improve the forwarding efficiency of key frames.

In one embodiment, the video key frame forwarding method of the embodiment of the present invention further comprises the steps of: when waiting for the arrival of the key frame, recording the waiting time before the arrival of the key frame; if the waiting time exceeds the time threshold of the allowable timeout, then Select one of the alternative video source terminals to replace the current video source terminal for key frame forwarding.

In the technical solution of the above embodiment, when the waiting time for the arrival of the key frame of the current video source terminal exceeds the time threshold, other alternative video source terminals are selected to replace the current video source terminal to perform the key frame forwarding operation, which can effectively avoid the current video source terminal. The source terminal line failure and other factors cause the waiting time to be too long. The video request terminal needs to wait for a long time for the forwarding of the key frame of the video source terminal, and cannot complete the key frame forwarding in time, which can improve the key frame forwarding efficiency.

In one embodiment, before the required arrival time of the next key frame of the video source terminal for obtaining the forwarded video, the method further includes: receiving a video forwarding request from the video requesting terminal, and obtaining the video forwarding request according to the video forwarding request. The information of the video source terminal that matches the video forwarding request, and according to the information of the video source terminal, the forwarding priority of each video source terminal is sorted, and a forwarding priority list is generated; according to the priority order of the forwarding priority list, select The video source terminal with the highest priority is used as the video source terminal to forward the video to monitor the arrival time of key frames.

According to the technical solution of the above embodiment, according to the video forwarding request received by the video requesting terminal, the forwarding priority of the video source terminals accessed by the server is sorted, and the video source terminal with the highest priority is first selected in order to forward the video. The frame arrival time is monitored, and go to step S201 to perform the key frame forwarding operation. When multiple matching video source terminals are connected, the optimal video source terminal can be selected according to the needs of the video requesting terminal to perform key frame forwarding, which can effectively improve the Quality and efficiency of video keyframe forwarding.

In one embodiment, before the required arrival time of the next key frame of the video source terminal for obtaining the forwarded video, the method further includes the steps of: obtaining the number of video forwarding paths of the video requesting terminal; the selection priority is the highest The step of performing key frame forwarding by the video source terminal as the video source terminal that forwards the video includes: according to the priority order of the forwarding priority list, selecting the video source terminals corresponding to the number of the video forwarding paths with the highest priority respectively. As the video source terminal that forwards the video, it performs key frame forwarding.

In the technical solutions of the above embodiments, the video request terminal can simultaneously receive video data from multiple video source terminals, and accordingly, when selecting the video source terminal for key frame forwarding for each video source terminal according to the forwarding priority, it further selects the same video source terminal. The number of video source terminals corresponding to the number of forwarding channels is used as the video source terminal for forwarding video to perform key frame forwarding, which can realize the synchronous forwarding of key frames to multiple video source terminals.

In one embodiment, before the required arrival time of the next key frame of the video source terminal for obtaining the forwarded video, the method further includes: receiving a video forwarding request from the video requesting terminal, and obtaining the video forwarding request according to the video forwarding request. The information of the video source terminal matched by the video forwarding request, according to the information of the video source terminal, carry out forwarding priority sorting to each video source terminal, and generate a forwarding priority list; , then the step of selecting a replacement video source terminal from the alternative video source terminal to perform key frame forwarding includes: if the waiting time exceeds the time threshold of the allowable timeout, then according to the priority order of the forwarding priority list , select an alternative video source terminal in the forwarding priority list to replace the current video source terminal to perform key frame forwarding.

According to the technical solution of the above embodiment, according to the video forwarding request received by the video requesting terminal, the forwarding priority is sorted for the video source terminal accessed by the server, and the video source terminal is sequentially selected to perform key frame forwarding according to the order of the forwarding priority, When the key frame forwarding of the current video source terminal exceeds the time threshold, the alternative video source terminal is selected in turn according to the priority order of the forwarding priority list to wait for or request the key frame, so that the key frame can be carried out according to the priority order. Forwarding, improving the forwarding quality of key frames, avoiding the situation of waiting for the same client key frame all the time, and improving the forwarding efficiency of key frames.

In an embodiment, the step of performing forwarding priority sorting on each video source terminal according to the information of the video source terminal, the step of generating a forwarding priority list includes: according to the obtained audio energy size order of each of the video source terminals Perform priority sorting to generate a forwarding priority list of the video source terminal, and perform priority sorting according to the order of the audio energy of the video source terminal. The video data of the video source terminal currently being called can be preferentially forwarded, and the quality of video forwarding can be improved.

In another embodiment, the step of performing forwarding priority ordering on each video source terminal according to the information of the video source terminal, the step of generating a forwarding priority list includes: performing the forwarding priority according to the video source terminal priority information specified by the video requesting terminal. The priority is sorted, and a forwarding priority list of the video source terminal is generated. In the technical solution of this embodiment, the video requesting terminal can designate a specific video source terminal to perform video forwarding preferentially.

In yet another embodiment, the step of performing forwarding priority sorting on each video source terminal according to the information of the video source terminals, the step of generating a forwarding priority list includes: performing priority sorting according to the temporal order of video source terminals access , to generate a forwarding priority list of the video source terminal. The technical solution of this embodiment can sort the forwarding priority according to the order in which the video source terminal accesses the server. For example, the newly accessed video source terminal can be preferentially selected for video forwarding. Improve video forwarding efficiency.

In one embodiment, the step of obtaining the video forwarding paths of the video requesting terminal includes: acquiring the video forwarding paths that the video requesting terminal can receive according to the downlink bandwidth of the video requesting terminal. The solution can obtain the appropriate number of video forwarding channels according to the current downlink bandwidth capacity of the video requesting terminal. While realizing multi-threaded video forwarding and improving video forwarding efficiency, it ensures the stability of current video forwarding data transmission and avoids too many forwarding threads. Live video freezes caused by exceeding the bandwidth that the video requesting terminal can receive.

In another embodiment, the step of acquiring the number of video forwarding paths of the video requesting terminal includes: acquiring the number of video forwarding paths that can be received by the video requesting terminal according to the decoding capability of the video requesting terminal. The technical solution can obtain the appropriate number of video forwarding paths according to the decoding capability of the video requesting terminal, realize multi-threaded video forwarding, improve the video forwarding efficiency, and at the same time ensure the stability of the current video forwarding data transmission, and avoid too many forwarding threads exceeding the video. Live video freezes caused by requesting the terminal's decoding capability.

A video key frame forwarding device, comprising:

A key frame monitoring module, used to obtain the arrival time required for the arrival of the next key frame of the video source terminal that forwards the video, wherein the key frame is a key frame periodically sent by the video source terminal;

A key frame request module, configured to send a key frame request to the video source terminal if the arrival time exceeds the time threshold that allows timeout, and wait for the key frame to arrive;

A key frame forwarding module is configured to, when the key frame arrives, receive the arrived key frame and forward the received key frame to the video requesting terminal.

The above video key frame forwarding device, by monitoring the arrival time of the next key frame of the video source terminal, actively sends a key frame request to the video when the arrival time required to obtain the next key frame exceeds the time threshold allowed for timeout The source terminal can effectively avoid too long waiting time for key frames, thereby effectively improving the forwarding efficiency of key frames.

A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein, when the processor executes the program, any video as described in the above embodiments is implemented Keyframe forwarding method.

The above-mentioned computer equipment, through the computer program running on the processor, realizes that by monitoring the arrival time of the next key frame of the video source terminal, when the arrival time required to obtain the next key frame will exceed the time threshold of the allowable timeout, Then, the key frame request is actively sent to the video source terminal, which effectively avoids a long waiting time for the key frame, thereby effectively improving the forwarding efficiency of the key frame.

A computer storage medium on which a computer program is stored, wherein when the program is executed by a processor, any one of the video key frame forwarding methods in the foregoing embodiments is implemented.

The above-mentioned computer storage medium, through the computer program stored in it, realizes that by monitoring the arrival time of the next key frame of the video source terminal, when the arrival time required to obtain the next key frame exceeds the time threshold of the allowable timeout, it will actively send the The key frame request is sent to the video source terminal, which effectively avoids the long waiting time for the key frame, thereby effectively improving the forwarding efficiency of the key frame.

In addition, the present invention also provides a live video system, including: a video request terminal, a video source terminal, and a server device; wherein, the video request terminal and the video source terminal are respectively connected in communication with the server device; the video request terminal is used for Send a video forwarding request to the server device, and receive video data from the video source terminal forwarded by the server device; the video source terminal is used to periodically send key frames to the server device, receive the video data request from the server device, and Forwarding the corresponding video data to the server device; the server device is configured to execute the video key frame forwarding method of any one of the above embodiments, and is used to forward the received key frame to the video requesting terminal that requests to forward the video. , and is also used to forward the video data of the corresponding video source terminal to the video requesting terminal that requests to forward the video.

In the above-mentioned live video system, the video source terminal periodically sends key frames to the server, and the server receives the video forwarding request from the video requesting terminal, monitors the arrival time of the next key frame of the video source terminal, and obtains the arrival time required for the next key frame after obtaining the next key frame. When the allowable timeout time threshold is exceeded, it will actively send a key frame request to the video source terminal, which can effectively prevent the key frame from waiting too long, thereby effectively improving the key frame forwarding efficiency.

Description of drawings

1 is a schematic diagram of an application environment of a method for requesting a video key frame according to an embodiment;

2 is a schematic flowchart of a method for requesting a video key frame according to an embodiment;

3 is a schematic flowchart of a method for requesting a video key frame according to an embodiment;

4 is a schematic flowchart of a method for requesting a video key frame according to an embodiment;

5 is a schematic flowchart of a method for requesting a video key frame according to an embodiment;

6 is a schematic structural diagram of a video key frame request apparatus according to an embodiment;

FIG. 7 is a schematic structural diagram of a video key frame request system according to an embodiment.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

The terms "comprising" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example a process, method, system, product or device comprising a series of steps or (module) units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or alternatively It also includes other steps or units inherent to these processes, methods, products or devices.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

The video key frame forwarding method provided by the present invention can be applied to the application environment shown in FIG. 1 . The video request terminal 101 and the video source terminal 102 respectively communicate with the server 103 through the network. The video requesting terminal 101 sends a video forwarding request to the server 103, and the server 103 forwards the video data of the video source terminal 102 to the video requesting terminal 101 according to the received video forwarding request, wherein the video requesting terminal 101 is a terminal device with a video playback function, The video source terminal 102 is a terminal device that provides a video source. The video request terminal 101 and the video source terminal 102 can be, but are not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The video source terminal 102 It may include one or more, and the server 104 may be implemented by an independent server or a server cluster composed of multiple servers.

Those skilled in the art can understand that the structure shown in FIG. 1 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the server or device to which the solution of the present application is applied. A device may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

In one embodiment, as shown in FIG. 2 , a video key frame forwarding method is provided, and the method is applied to the server 103 in FIG. 1 as an example for description, including the following steps:

S201: Obtain the arrival time required for the arrival of the next key frame of the video source terminal that forwards the video, where the key frame is a key frame periodically sent by the video source terminal.

The video source terminal that forwards the video is selected by the current server and is used to monitor the arrival time of key frames and perform video forwarding. The video source terminal periodically sends key frames to the server according to a predetermined time period. .

In this step, the server monitors the arrival time of the key frame on the video source terminal currently selected for forwarding the video, and obtains the arrival time required for the arrival of the next key frame of the video source terminal.

S202, if the arrival time exceeds a time threshold that allows timeout, send a key frame request to the video source terminal, and wait for the key frame to arrive.

Wherein, the time threshold for allowing timeout is a preset acceptable time-consuming upper limit time value of key frame forwarding.

In this step, when it is determined that the arrival time will exceed the allowable timeout time threshold, the server sends a key frame request to the video source terminal, requesting the video source terminal to send the key frame to the server, and wait for the arrival of the key frame.

S203, when the key frame arrives, receive the arrived key frame and forward the received key frame to the video requesting terminal.

In this step, the server receives the received key frame and forwards the received key frame to the video requesting terminal to complete the forwarding of the video key frame.

The above video key frame forwarding method, by monitoring the arrival time of the next key frame of the video source terminal, when the arrival time required to obtain the next key frame exceeds the time threshold of the allowable timeout, it will actively send a key frame request to the video The source terminal can effectively avoid too long waiting time for key frames, thereby effectively improving the forwarding efficiency of key frames.

In one embodiment, after the step of obtaining the arrival time required for the arrival of the next key frame of the video source terminal of the video to be forwarded in step S201, the method further includes: if the arrival time does not exceed the time threshold for allowing timeout, Then wait for the keyframe to arrive. Before requesting a key frame, the server will first compare the arrival time of the next key frame and the server's allowable key frame timeout time, so as to choose whether to continue waiting for the next key frame or actively request the key frame, which can effectively reduce the actual waiting time before the key frame arrives. time to improve the forwarding efficiency of key frames.

Referring to FIG. 3, FIG. 3 is a schematic flowchart of a video key frame forwarding method according to an embodiment. The video key frame forwarding method in this embodiment includes steps:

S301, obtaining the arrival time required for the arrival of the next key frame of the video source terminal that forwards the video, wherein the key frame is a key frame periodically sent by the video source terminal;

S302, judging whether the arrival time exceeds the time threshold that allows timeout;

S303, if the arrival time does not exceed the time threshold that allows timeout, wait for the key frame to arrive;

S304, if the arrival time exceeds the time threshold that allows timeout, send a key frame request to the video source terminal, and wait for the key frame to arrive;

S308, when the key frame arrives, receive the arrived key frame and forward the received key frame to the video requesting terminal.

In one embodiment, as shown in FIG. 4 , after step S303 or step S304, it further includes steps:

S305, when waiting for the key frame to arrive, record the waiting time before the key frame arrives;

S306, judging whether the waiting time exceeds the time threshold that allows timeout;

S307, if the waiting time exceeds the time threshold for allowing timeout, select one of the alternative video source terminals to replace the current video source terminal to perform key frame forwarding.

The key frame forwarding described in step S306 may be the key frame forwarding operation performed according to steps S301 to S307.

The alternative video source terminals are the video source terminals currently accessing the server that match the video request sent by the video requesting terminal, and the remaining video source terminals except the video source terminal currently forwarding the video.

During the transmission of the key frame, the arrival of the key frame of the current video source terminal may be delayed due to reasons such as network failure. The technical solution of the above embodiment, the waiting time for the arrival of the key frame of the current video source terminal exceeds the time threshold. When the video source terminal is selected, another alternative video source terminal is selected to replace the current video source terminal for the key frame forwarding operation, which can effectively avoid the long waiting time caused by the current video source terminal line failure and other factors, and the video request terminal needs to wait for the video for a long time. For the forwarding of key frames of the source terminal, the problem that the forwarding of key frames cannot be completed in time can improve the forwarding efficiency of key frames.

In an embodiment, before the required arrival time of the next key frame of the video source terminal for acquiring the forwarded video in step S201, the method further includes: receiving a video forwarding request from the video requesting terminal, and according to the video forwarding request, Obtain the information of the video source terminal that matches the video forwarding request, perform forwarding priority sorting on each video source terminal according to the information of the video source terminal, and generate a forwarding priority list; according to the priority of the forwarding priority list In order, the video source terminal with the highest priority is selected as the video source terminal for forwarding the video to monitor the arrival time of key frames.

According to the technical solution of the above embodiment, according to the video forwarding request received by the video requesting terminal, the forwarding priority of the video source terminals accessed by the server is sorted, and the video source terminal with the highest priority is first selected in order to forward the video. The frame arrival time is monitored, and go to step S201 to perform the key frame forwarding operation. When multiple matching video source terminals are connected, the optimal video source terminal can be selected according to the needs of the video requesting terminal to perform key frame forwarding, which can effectively improve the Quality and efficiency of video keyframe forwarding.

Further, in one embodiment, before the required arrival time of the next key frame of the video source terminal for acquiring the forwarded video in step S201, the method further includes the step of: acquiring the video forwarding path number of the video requesting terminal; The step of selecting the video source terminal with the highest priority as the video source terminal for forwarding the video to perform key frame forwarding includes: according to the priority order of the forwarding priority list, selecting a corresponding number of the video forwarding paths with the highest priority. A number of video source terminals are respectively used as video source terminals for forwarding video to perform key frame forwarding.

In the technical solutions of the above embodiments, the video request terminal can simultaneously receive video data from multiple video source terminals, and accordingly, when selecting the video source terminal for key frame forwarding for each video source terminal according to the forwarding priority, it further selects the same video source terminal. The number of video source terminals corresponding to the number of forwarding channels is used as the video source terminal for forwarding video to perform key frame forwarding, which can realize the synchronous forwarding of key frames to multiple video source terminals.

In an embodiment, before the required arrival time of the next key frame of the video source terminal for acquiring the forwarded video in step S201, the method further includes: receiving a video forwarding request from the video requesting terminal, and according to the video forwarding request, Obtain the information of the video source terminal that matches the video forwarding request, perform forwarding priority sorting on each video source terminal according to the information of the video source terminal, and generate a forwarding priority list; if the waiting time exceeds the allowable timeout the time threshold value, then the step of selecting a replacement video source terminal to perform key frame forwarding from the alternative video source terminals includes: if the waiting time exceeds the time threshold that allows timeout, then according to the forwarding priority list Priority order, select an alternative video source terminal in the forwarding priority list to replace the current video source terminal to perform key frame forwarding.

According to the technical solution of the above embodiment, according to the video forwarding request received by the video requesting terminal, the forwarding priority is sorted for the video source terminal accessed by the server, and the video source terminal is sequentially selected to perform key frame forwarding according to the order of the forwarding priority, When the key frame forwarding of the current video source terminal exceeds the time threshold, the alternative video source terminal is selected in turn according to the priority order of the forwarding priority list to wait for or request the key frame, so that the key frame can be carried out according to the priority order. Forwarding, improving the forwarding quality of key frames, avoiding the situation of waiting for the same client key frame all the time, and improving the forwarding efficiency of key frames.

Referring to FIG. 5, FIG. 5 is a schematic flowchart of an embodiment, which includes steps:

S501, receiving a video forwarding request from a video requesting terminal;

S502, according to the video forwarding request, obtain the information of the video source terminal matching the video forwarding request, and perform forwarding priority sorting on each video source terminal according to the information of the video source terminal, and generate a forwarding priority list;

S503, acquiring the number of video forwarding paths of the video requesting terminal;

S504, according to the priority order of the forwarding priority list, select the video source terminals corresponding to the number of the video forwarding paths with the highest priority as the video source terminals for forwarding the video respectively to monitor the arrival time of key frames;

S505, obtaining the arrival time required for the arrival of the next key frame of the video source terminal that forwards the video, wherein the key frame is a key frame periodically sent by the video source terminal;

S506, judging whether the arrival time exceeds the time threshold that allows timeout;

S507, if the arrival time does not exceed the time threshold for allowing timeout, wait for the key frame to arrive;

S508, if the arrival time exceeds the time threshold that allows timeout, send a key frame request to the video source terminal, and wait for the key frame to arrive;

S509, when waiting for the key frame to arrive, record the waiting time before the key frame arrives;

S510, judging whether the waiting time exceeds the time threshold that allows timeout;

S511, if the waiting time exceeds the time threshold that allows timeout, select an alternative video source terminal in the forwarding priority list to replace the current video source terminal according to the priority order of the forwarding priority list The frame is forwarded, and the process returns to S505.

S512, when the key frame arrives, receive the arrived key frame and forward the received key frame to the video requesting terminal.

Wherein, for any video source terminal in step S504, it can jump to the steps of executing steps S505 to S512.

In the technical solution of the above embodiment, before requesting a key frame, the server will first compare the arrival time of the next key frame and the timeout time of the key frame allowed by the server, so as to choose whether to continue waiting for the next key frame or actively request the key frame. Get keyframes in the shortest time possible. In addition, since it is possible to select whether to request key frames according to the arrival time, the period for sending key frames by the video source terminal can be set to be longer, which can also ensure the forwarding efficiency of live video, and avoid the need to increase the frequency of sending key frames in order to improve the effect of live video. The resource consumption is large and the network is stuck, etc. During the transmission of key frames, possibly due to network and other reasons, if it fails to arrive within the expected time, the server will re-select a client and wait for or request its key frame, so as to avoid waiting for the same client key all the time. frame, which effectively improves the forwarding efficiency of key frames.

For the forwarding priority sorting of video source terminals, the priority sorting can be carried out according to different evaluation methods. For example, you can refer to the specified order of video requesting terminals, the audio energy size of each video source terminal, or the access order of each video source terminal, etc. Wait.

In an embodiment, the step of performing forwarding priority sorting on each video source terminal according to the information of the video source terminal, the step of generating a forwarding priority list includes: according to the obtained audio energy size order of each of the video source terminals Perform priority sorting to generate a forwarding priority list of the video source terminal, and perform priority sorting according to the order of the audio energy of the video source terminal. The video data of the video source terminal currently being called can be preferentially forwarded, and the quality of video forwarding can be improved.

In another embodiment, the step of performing forwarding priority sorting on each video source terminal according to the information of the video source terminal, the step of generating a forwarding priority list includes: performing a forwarding priority according to the video source terminal priority information specified by the video requesting terminal. The priority is sorted, and a forwarding priority list of the video source terminal is generated. In the technical solution of this embodiment, the video requesting terminal can designate a specific video source terminal to perform video forwarding preferentially.

In yet another embodiment, the step of performing forwarding priority sorting on each video source terminal according to the information of the video source terminals, the step of generating a forwarding priority list includes: performing priority sorting according to the temporal order of video source terminals access , to generate a forwarding priority list of the video source terminal. The technical solution of this embodiment can sort the forwarding priority according to the order in which the video source terminal accesses the server. For example, the newly accessed video source terminal can be preferentially selected for video forwarding. Improve video forwarding efficiency.

The number of video forwarding channels that can be received by the video requesting terminal can be obtained according to different evaluation methods. For example, the number of video forwarding channels of the current video requesting terminal can be calculated according to the downlink bandwidth or decoding capability of the video requesting terminal, or the video requesting terminal You can also specify the number of video forwarding channels and so on.

In one embodiment, the step of obtaining the video forwarding paths of the video requesting terminal includes: acquiring the video forwarding paths that the video requesting terminal can receive according to the downlink bandwidth of the video requesting terminal. The solution can obtain the appropriate number of video forwarding channels according to the current downlink bandwidth capacity of the video requesting terminal. While realizing multi-threaded video forwarding and improving video forwarding efficiency, it ensures the stability of current video forwarding data transmission and avoids too many forwarding threads. Live video freezes caused by exceeding the bandwidth that the video requesting terminal can receive.

In another embodiment, the step of acquiring the number of video forwarding paths of the video requesting terminal includes: acquiring the number of video forwarding paths that can be received by the video requesting terminal according to the decoding capability of the video requesting terminal. The technical solution can obtain the appropriate number of video forwarding paths according to the decoding capability of the video requesting terminal, realize multi-threaded video forwarding, improve the video forwarding efficiency, and at the same time ensure the stability of the current video forwarding data transmission, and avoid too many forwarding threads exceeding the video. Live video freezes caused by requesting the terminal's decoding capability.

In one embodiment, as shown in FIG. 6, a video key frame forwarding apparatus is provided, including:

The key frame monitoring module 601 is used to obtain the arrival time required for the arrival of the next key frame of the video source terminal that forwards the video, wherein the key frame is the key frame periodically sent by the video source terminal;

A key frame request module 602, configured to send a key frame request to the video source terminal if the arrival time exceeds the time threshold that allows timeout, and wait for the key frame to arrive;

The key frame forwarding module 603 is configured to, when the key frame arrives, receive the arrived key frame and forward the received key frame to the video requesting terminal.

The above video key frame forwarding device, by monitoring the arrival time of the next key frame of the video source terminal, actively sends a key frame request to the video when the arrival time required to obtain the next key frame exceeds the time threshold allowed for timeout The source terminal can effectively avoid too long waiting time for key frames, thereby effectively improving the forwarding efficiency of key frames.

In one embodiment, the video key frame forwarding apparatus of the embodiment of the present invention further includes: a key frame waiting module, configured to wait for the key frame to arrive if the arrival time does not exceed the time threshold allowed for timeout.

In one embodiment, the video key frame forwarding apparatus according to the embodiment of the present invention further includes:

The waiting time recording module is used to record the waiting time before the key frame arrives when waiting for the key frame to arrive;

A video source terminal replacement module, configured to select one alternative video source terminal to replace the current video source terminal to perform key frame forwarding if the waiting time exceeds the time threshold allowed for timeout.

In one embodiment, the video key frame forwarding apparatus according to the embodiment of the present invention further includes:

A priority sorting module is configured to receive a video forwarding request from a video requesting terminal, obtain information of a video source terminal matching the video forwarding request according to the video forwarding request, and perform a search for each video source terminal according to the information of the video source terminal. The terminal performs forwarding priority sorting and generates a forwarding priority list;

The video source terminal selection module is configured to select the video source terminal with the highest priority as the video source terminal for forwarding video according to the priority order of the forwarding priority list to monitor the arrival time of key frames.

In one embodiment, the video key frame forwarding apparatus according to the embodiment of the present invention further includes:

a number of forwarding paths acquisition module, used for acquiring the number of video forwarding paths of the video requesting terminal;

The video source terminal selection module is further configured to, according to the priority order of the forwarding priority list, select the video source terminals corresponding to the number of the video forwarding paths with the highest priority as the video source terminals for forwarding the video respectively to perform key operations. Frame forwarding.

In one embodiment, the video key frame forwarding apparatus according to the embodiment of the present invention further includes:

A priority sorting module is configured to receive a video forwarding request from a video requesting terminal, obtain information of a video source terminal matching the video forwarding request according to the video forwarding request, and perform a search for each video source terminal according to the information of the video source terminal. The terminal performs forwarding priority sorting and generates a forwarding priority list;

The video source terminal replacement module is further configured to select an alternative video source terminal in the forwarding priority list according to the priority order of the forwarding priority list if the waiting time exceeds the time threshold that allows timeout Replace the current video source terminal for key frame forwarding.

In an embodiment, the priority sorting module is further configured to perform priority sorting according to the acquired audio energy size order of each of the video source terminals, and generate a forwarding priority list of the video source terminals.

In another embodiment, the priority sorting module is further configured to perform priority sorting according to the video source terminal priority information specified by the video request terminal to generate a forwarding priority list of the video source terminal.

In yet another embodiment, the priority sorting module is further configured to perform priority sorting according to the temporal order of video source terminal access, and generate a forwarding priority list of the video source terminal.

In one embodiment, the module for obtaining the number of forwarding channels is further configured to obtain the number of video forwarding channels that can be received by the video requesting terminal according to the downlink bandwidth of the video requesting terminal.

In another embodiment, the module for obtaining the number of forwarding channels is further configured to obtain the number of video forwarding channels that can be received by the video requesting terminal according to the decoding capability of the video requesting terminal.

The video key frame forwarding system of the present invention has a one-to-one correspondence with the video key frame forwarding method of the present invention, and the technical features and beneficial effects described in the embodiments of the video key frame forwarding method are applicable to the embodiments of the video key frame forwarding system , hereby declare.

Based on the examples described above, in one embodiment there is also provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the During the program, any one of the video key frame forwarding methods in the foregoing embodiments is implemented.

The above-mentioned computer equipment, through the computer program running on the processor, realizes that by monitoring the arrival time of the next key frame of the video source terminal, when the arrival time required to obtain the next key frame will exceed the time threshold of the allowable timeout, Then, the key frame request is actively sent to the video source terminal, which effectively avoids a long waiting time for the key frame, thereby effectively improving the forwarding efficiency of the key frame.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage In the medium, as in this embodiment of the present invention, the program may be stored in a storage medium of a computer system and executed by at least one processor in the computer system to implement the processes including the above-mentioned embodiments of the sleep assistance methods. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

Accordingly, in an embodiment, a storage medium is also provided, on which a computer program is stored, wherein, when the program is executed by a processor, any one of the video key frame forwarding methods in the foregoing embodiments is implemented.

The above-mentioned computer storage medium, through the computer program stored in it, realizes that by monitoring the arrival time of the next key frame of the video source terminal, when the arrival time required to obtain the next key frame exceeds the time threshold of the allowable timeout, it will actively send the The key frame request is sent to the video source terminal, which effectively avoids the long waiting time for the key frame, thereby effectively improving the forwarding efficiency of the key frame.

Those skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by program instructions related to hardware, the aforementioned program may be stored in a computer-readable storage medium, and when the program is executed, execute Including the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a random access memory (RAM, Random Access Memory), a read-only memory (ROM, Read-Only Memory), a magnetic disk or an optical disk and other various A medium on which program code can be stored.

In addition, the present invention also provides a live video system, as shown in FIG. 7 , including: a video request terminal 701, a video source terminal 702, and a server device 703; wherein, the video source terminal 702 may have one or more, so The video request terminal 701 and the video source terminal 702 are respectively connected in communication with the server device 703; the video request terminal 701 is used to send a video forwarding request to the server device 703, and receive the video source terminal 702 forwarded by the server device 703. Video data; the video source terminal 702 is used to periodically send key frames to the server device 703, receive video data requests from the server device 703, and forward the corresponding video data to the server device 703; the server device 703 uses For executing the video key frame forwarding method of any one of the above embodiments, and for forwarding the received key frame to the video requesting terminal 701 requesting to forward the video, it is also used for forwarding the video data of the corresponding video source terminal 702. To the video request terminal 701 that requests to forward the video.

In the above-mentioned live video system, the video source terminal periodically sends key frames to the server, and the server receives the video forwarding request from the video requesting terminal, monitors the arrival time of the next key frame of the video source terminal, and obtains the arrival time required for the next key frame after obtaining the next key frame. When the allowable timeout time threshold is exceeded, it will actively send a key frame request to the video source terminal, which can effectively prevent the key frame from waiting too long, thereby effectively improving the key frame forwarding efficiency.

The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity, all possible combinations of the technical features in the above-described embodiments are not described. However, as long as there is no contradiction between the combinations of these technical features, All should be regarded as the scope described in this specification.

The above-mentioned embodiments only represent several embodiments of the present invention, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can also be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims (17)

1. a video key frame forwarding method, is characterized in that, is applied to server, comprises the steps: Obtain the arrival time required for the arrival of the next key frame of the video source terminal that forwards the video, wherein the key frame is a key frame periodically sent by the video source terminal; If the arrival time exceeds the time threshold that allows timeout, send a key frame request to the video source terminal, and wait for the key frame to arrive; When waiting for the key frame to arrive, record the waiting time before the key frame arrives; If the waiting time exceeds the time threshold that allows timeout, then select a replacement current video source terminal from the alternative video source terminals to perform key frame forwarding; When the key frame arrives, the arrived key frame is received and the received key frame is forwarded to the video requesting terminal. 2. The video key frame forwarding method according to claim 1, wherein after the step of obtaining the required arrival time for the arrival of the next key frame of the video source terminal of the video to be forwarded, further comprising: if the described If the arrival time does not exceed the time threshold allowed for timeout, wait for the key frame to arrive. 3. The video key frame forwarding method according to claim 1, wherein before the next key frame of the video source terminal that obtains the forwarded video arrives at the required arrival time, further comprising: receiving a video forwarding request from a video requesting terminal, obtaining information of a video source terminal matching the video forwarding request according to the video forwarding request, and performing forwarding priority sorting on each video source terminal according to the information of the video source terminal, Generate a forwarding priority list; According to the priority order of the forwarding priority list, the video source terminal with the highest priority is selected as the video source terminal for forwarding the video to monitor the arrival time of the key frame. 4. video key frame forwarding method according to claim 3, is characterized in that, also comprises the step: obtaining the video forwarding paths of the video requesting terminal; The step of selecting the video source terminal with the highest priority as the video source terminal for forwarding the video to monitor the arrival time of key frames includes: According to the priority order of the forwarding priority list, the video source terminals corresponding to the number of forwarding paths with the highest priority are selected as the video source terminals for forwarding the video respectively to monitor the arrival time of key frames. 5. video key frame forwarding method according to claim 1, is characterized in that, also comprises: receiving a video forwarding request from a video requesting terminal, obtaining information of a video source terminal matching the video forwarding request according to the video forwarding request, and performing forwarding priority sorting on each video source terminal according to the information of the video source terminal, Generate a forwarding priority list; Described if the described waiting time exceeds the time threshold that allows timeout, then the step of selecting a replacement current video source terminal from the alternative video source terminal to perform key frame forwarding includes: If the waiting time exceeds the time threshold for allowing timeout, select an alternative video source terminal in the forwarding priority list to replace the current video source terminal to perform key frame forwarding according to the priority order of the forwarding priority list . 6. The video key frame forwarding method according to claim 3 or 5, wherein the forwarding priority sorting is performed on each video source terminal according to the information of the video source terminal, and the step of generating a forwarding priority list comprises the following steps: : Perform priority sorting according to the obtained audio energy size order of each of the video source terminals, and generate a forwarding priority list of the video source terminals; or Perform priority sorting according to the priority information of the video source terminal specified by the video request terminal, and generate a forwarding priority list of the video source terminal; or The priority is sorted according to the time sequence of the access of the video source terminal, and the forwarding priority list of the video source terminal is generated. 7. The video key frame forwarding method according to claim 4, wherein the step of acquiring the number of video forwarding paths of the video requesting terminal comprises: Acquire the number of video forwarding paths that can be received by the video requesting terminal according to the downlink bandwidth of the video requesting terminal; or The number of video forwarding paths that can be received by the video requesting terminal is acquired according to the decoding capability of the video requesting terminal. 8. A video key frame forwarding device, characterized in that, applied to a server, comprising: A key frame monitoring module, used to obtain the arrival time required for the arrival of the next key frame of the video source terminal that forwards the video, wherein the key frame is a key frame periodically sent by the video source terminal; A key frame request module, configured to send a key frame request to the video source terminal if the arrival time exceeds the time threshold that allows timeout, and wait for the key frame to arrive; The waiting time recording module is used to record the waiting time before the key frame arrives when waiting for the key frame to arrive; A video source terminal replacement module, used for if the waiting time exceeds the time threshold of the allowable timeout, then select a replacement current video source terminal from the alternative video source terminals to perform key frame forwarding; A key frame forwarding module is configured to, when the key frame arrives, receive the arrived key frame and forward the received key frame to the video requesting terminal. 9. The video key frame forwarding device according to claim 8, wherein the video key frame forwarding device further comprises: The key frame waiting module is configured to wait for the key frame to arrive if the arrival time does not exceed the time threshold allowed for timeout. 10. The video key frame forwarding device according to claim 8, wherein the video key frame forwarding device further comprises: A priority sorting module is configured to receive a video forwarding request from a video requesting terminal, obtain information of a video source terminal matching the video forwarding request according to the video forwarding request, and perform a search for each video source terminal according to the information of the video source terminal. The terminal performs forwarding priority sorting and generates a forwarding priority list; The video source terminal selection module is configured to select the video source terminal with the highest priority as the video source terminal for forwarding video according to the priority order of the forwarding priority list to monitor the arrival time of key frames. 11. The video key frame forwarding device according to claim 10, wherein the video key frame forwarding device further comprises: a number of forwarding paths acquisition module, used for acquiring the number of video forwarding paths of the video requesting terminal; The video source terminal selection module is further configured to, according to the priority order of the forwarding priority list, select the video source terminals corresponding to the number of the forwarding paths with the highest priority as the video source terminals for forwarding the video respectively to perform key operations. Frame arrival time monitoring. 12. The video key frame forwarding device according to claim 8, wherein the video key frame forwarding device further comprises: A priority sorting module is configured to receive a video forwarding request from a video requesting terminal, obtain information of a video source terminal matching the video forwarding request according to the video forwarding request, and perform a search for each video source terminal according to the information of the video source terminal. The terminal performs forwarding priority sorting and generates a forwarding priority list; The video source terminal replacement module is further configured to, if the waiting time exceeds the time threshold that allows timeout, select an alternative video source in the forwarding priority list according to the priority order of the forwarding priority list The terminal replaces the current video source terminal to forward key frames. 13. The video key frame forwarding device according to claim 10 or 12, wherein the priority sorting module is further used for: Perform priority sorting according to the obtained audio energy size order of each of the video source terminals, and generate a forwarding priority list of the video source terminals; or Perform priority sorting according to the priority information of the video source terminal specified by the video request terminal, and generate a forwarding priority list of the video source terminal; or The priority is sorted according to the time sequence of the access of the video source terminal, and the forwarding priority list of the video source terminal is generated. 14. video key frame forwarding device according to claim 11, is characterized in that, described forwarding path number acquisition module is also used for: Acquire the number of video forwarding paths that can be received by the video requesting terminal according to the downlink bandwidth of the video requesting terminal; or The number of video forwarding paths that can be received by the video requesting terminal is acquired according to the decoding capability of the video requesting terminal. 15. A live video system, comprising: a video request terminal, a video source terminal, and a server device; wherein, the video request terminal and the video source terminal are respectively connected in communication with the server device; The video request terminal is configured to send a video forwarding request to the server device, and receive video data from the video source terminal forwarded by the server device; The video source terminal is configured to periodically send key frames to the server device, receive a video data request from the server device, and forward corresponding video data to the server device; The server device is configured to perform the video key frame forwarding method according to any one of claims 1 to 7, and is configured to forward the received key frame to the video requesting terminal requesting to forward the video, and also to forward the video key frame. The video data of the corresponding video source terminal is forwarded to the video requesting terminal that requests to forward the video. 16. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the computer program as claimed in the claims The video key frame forwarding method described in any one of 1 to 7. 17. A computer storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the video key frame forwarding method according to any one of claims 1 to 7 is implemented.

Images