CN116017046B

CN116017046B - Video processing method, device, equipment and storage medium

Info

Publication number: CN116017046B
Application number: CN202211631783.4A
Authority: CN
Inventors: 郭靖; 曹洪彬; 李一鸣; 张佳
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-12-16
Filing date: 2022-12-16
Publication date: 2025-06-24
Anticipated expiration: 2042-12-16
Also published as: CN116017046A

Abstract

The embodiment of the application discloses a video processing method, a device, equipment and a storage medium. The method comprises the steps of responding to the completion of decoding a first video frame in a video to be processed, obtaining a speed change parameter in a decoding queue corresponding to the video to be processed, wherein the speed change parameter is used for indicating whether the video frame writing speed of the decoding queue is greater than the video frame decoding speed, obtaining a rendering parameter of the video to be processed, wherein the rendering parameter is used for indicating the time interval between video frames, and if the speed change parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed and the time interval between video frames indicated by the rendering parameter meets the rendering frame loss condition, rendering frame loss processing is carried out on the first video frame. Therefore, on the premise of ensuring the continuity of the video to be processed, the video delay can be reduced by carrying out frame loss processing on the decoded video frames.

Description

Video processing method, device, equipment and storage medium

Technical Field

The present application relates to the field of computer technology, and in particular, to a video processing method, a video processing apparatus, a computer device, and a computer readable storage medium.

Background

With the progress of scientific research, video has been widely used in people's daily lives. In low-delay video scenes (such as live broadcast, video call, cloud game service and the like), the conditions of data packet loss, disorder, delayed arrival and the like are processed through buffering in a video communication engine, so that smooth output of video frames to a decoder is ensured. Research shows that in the video processing process, video delay is increased due to the influence of factors such as network fluctuation, decoder performance fluctuation and the like, so that video interaction experience is poor.

Disclosure of Invention

The embodiment of the application provides a video processing method, a device, equipment and a computer readable storage medium, which can reduce video time delay.

In one aspect, an embodiment of the present application provides a video processing method, including:

Responding to the completion of decoding a first video frame in the video to be processed, and acquiring a speed variation parameter in a decoding queue corresponding to the video to be processed, wherein the speed variation parameter is used for indicating whether the video frame writing speed of the decoding queue is greater than the video frame decoding speed;

acquiring rendering parameters of the video to be processed, wherein the rendering parameters are used for indicating the time interval between video frames;

If the speed change parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed and the time interval between the video frames indicated by the rendering parameter meets the rendering frame loss condition, rendering frame loss processing is carried out on the first video frame.

In one aspect, an embodiment of the present application provides a video processing apparatus, including:

The acquisition unit is used for responding to the completion of decoding of the first video frame in the video to be processed, acquiring a speed change parameter in a decoding queue corresponding to the video to be processed, wherein the speed change parameter is used for indicating whether the video frame writing speed of the decoding queue is greater than the video frame decoding speed;

The method comprises the steps of obtaining rendering parameters of a video to be processed, wherein the rendering parameters are used for indicating time intervals between video frames;

And the processing unit is used for performing rendering frame loss processing on the first video frame if the speed change parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed and the time interval between the video frames indicated by the rendering parameter meets the rendering frame loss condition.

In one embodiment, the speed variation parameter in the decoding queue corresponding to the video to be processed includes a frame number indicating a number of video frames written in the decoding queue with a time interval less than a first time interval threshold, and the processing unit is further configured to:

if the number of frames is greater than or equal to the preset number of frames, determining that the video frame writing speed of the decoding queue is greater than the video frame decoding speed.

In one embodiment, the processing unit is further configured to:

If the number of frames sent is smaller than the preset number of frames and the time interval between video frames indicated by the rendering parameters meets the rendering frame loss condition, obtaining a buffer parameter, wherein the buffer parameter is used for indicating whether the buffer resource corresponding to the decoding queue meets the decoding requirement or not;

If the buffer parameter indicates that the buffer resource corresponding to the decoding queue does not meet the decoding requirement, determining the rendering frame loss frequency based on the buffer parameter;

and according to the rendering frame loss frequency, performing rendering frame loss processing on the video frames decoded in the target time period.

In one embodiment, the buffering parameter comprises an input buffering time of a second video frame, the second video frame being a last enqueued video frame in the decoding queue, the processing unit further configured to:

If the time consumption of the input buffer of the second video frame is greater than the time consumption threshold, determining that the buffer resource corresponding to the decoding queue does not meet the decoding requirement.

In one embodiment, the buffering parameters include an input buffering time of a second video frame, the second video frame being a last enqueued video frame in the decoding queue, the second video frame being transferred from the input buffering to the decoding queue, and the processing unit is configured to obtain the buffering parameters, specifically configured to:

acquiring the time of the first request of the second video frame to join the input buffer and the time of the second video frame successfully transferred to the decoding queue;

And calculating the time consumption of the input buffer of the second video frame according to the time of the first request of the second video frame to be added into the input buffer and the time of the second video frame to be successfully transferred to the decoding queue.

In one embodiment, the buffering parameter includes an input buffering time of a second video frame, the second video frame being a last enqueued video frame in the decoding queue, and the processing unit is configured to determine a rendering frame loss frequency based on the buffering parameter, specifically configured to:

determining a time consuming interval of the time consuming of the input buffer of the second video frame, wherein different time consuming intervals correspond to different frame losing frequencies;

And determining the frame loss frequency corresponding to the time consuming interval to which the time consuming of the input buffer memory of the second video frame belongs as the rendering frame loss frequency.

In one embodiment, the processing unit is further configured to:

and if the buffer parameter indicates that the buffer resource corresponding to the decoding queue meets the decoding requirement, rendering the first video frame and outputting the first video frame.

In one embodiment, the processing unit is further configured to:

If the number of frames is less than or equal to the preset number of frames and the time interval between video frames indicated by the rendering parameters meets the rendering frame loss condition, acquiring decoding performance parameters, wherein the decoding performance parameters are used for indicating whether the decoding performance of a decoder meets the decoding requirement;

if the decoding performance parameter indicates that the decoding performance of the decoder does not meet the decoding requirement, determining the rendering frame loss frequency as a first frequency;

and according to the first frequency, rendering and frame loss processing is carried out on the video frames decoded in the first time period.

In one embodiment, the decoding performance parameter includes a decoding bin number at a first time and a decoding bin number threshold at the first time, the decoding bin number at the first time being indicative of a number of video frames in the decoding queue at the first time, the decoding bin number threshold at the first time being determined based on a decoding performance of the decoder at the first time, the processing unit further configured to:

If the decoding bin number at the first moment is greater than the decoding bin number threshold at the first moment, determining that the decoding performance of the decoder at the first moment does not meet the decoding requirement.

In one embodiment, the decoding performance parameters further include a decoding bin number at a second time and a decoding bin number threshold at a second time, the second time being after the first time, the processing unit further configured to:

if the decoding bin number at the second moment is larger than the decoding bin number threshold at the second moment, determining the rendering frame loss frequency as a second frequency, wherein the second frequency is larger than the first frequency;

And according to the second frequency, rendering and frame loss processing is carried out on the video frames decoded in a second time period, wherein the second time period is after the first time period.

In one embodiment, the processing unit is further configured to:

If the decoding bin number at the second moment is smaller than or equal to the decoding bin number threshold at the second moment, determining the rendering frame loss frequency as a third frequency, wherein the third frequency is smaller than the first frequency;

and according to the third frequency, rendering and frame loss processing is carried out on the video frames decoded in the second time period.

In one embodiment, the decoding queue contains M video frames, M is an integer greater than 1, and the processing unit is configured to obtain a speed variation parameter in the decoding queue corresponding to the video to be processed, and specifically configured to:

Acquiring the time of writing M video frames into a decoding queue;

Calculating a write time interval between adjacent video frames in the M video frames based on the time of writing the M video frames into the decoding queue;

And counting the number of frames according to the writing time interval and the first time interval threshold value between adjacent video frames in the M video frames.

In one embodiment, the processing unit is further configured to:

If the speed change parameter indicates that the video frame writing speed of the decoding queue is less than or equal to the video frame decoding speed, rendering the first video frame and outputting the first video frame, or

And if the time interval between the video frames indicated by the rendering parameters does not meet the rendering frame loss condition, rendering the first video frame and outputting the first video frame.

In one embodiment, the rendering parameters of the video to be processed include a rendering time interval, the rendering time interval is used for indicating a time interval between a time when the first video frame finishes decoding and a time when the third video frame renders, the third video frame is a video frame with the shortest time interval between a time when the first video frame finishes decoding in the video frames to be processed rendered, and the time interval between the video frames meets a rendering frame loss condition and includes a rendering time interval being less than or equal to a second time interval threshold.

Accordingly, the present application provides a computer device comprising:

A memory in which a computer program is stored;

And the processor is used for loading a computer program to realize the video processing method.

Accordingly, the present application provides a computer readable storage medium storing a computer program adapted to be loaded by a processor and to perform the above-described video processing method.

Accordingly, the present application provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the video processing method described above.

In the embodiment of the application, in response to completion of decoding a first video frame in a video to be processed, a speed change parameter in a decoding queue corresponding to the video to be processed is obtained, the speed change parameter is used for indicating whether the video frame writing speed of the decoding queue is greater than the video frame decoding speed, a rendering parameter of the video to be processed is obtained, the rendering parameter is used for indicating a time interval between video frames, and if the speed change parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed and the time interval between video frames indicated by the rendering parameter meets a rendering frame loss condition, the first video frame is subjected to rendering frame loss processing. Therefore, on the premise of ensuring the continuity of the video to be processed, the video delay can be reduced by carrying out frame loss processing on the decoded video frames.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1a is a scene structure diagram of a video processing system according to an embodiment of the present application;

Fig. 1b is a flowchart of a video processing scheme according to an embodiment of the present application;

fig. 2 is a flowchart of a video processing method according to an embodiment of the present application;

FIG. 3 is a flowchart of another video processing method according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

First, the related art terms related to the present application will be briefly described:

received frame rate-received frame rate refers to the number of video frames received per second by the decoder in frames per second (abbreviated fps).

Decoding bin the decoding bin refers to the phenomenon that the decoder starts to output decoded images after a certain number of video frames are transmitted, for example, in a low-delay video service scene requiring 1080p@60fps, the normal video delay is 1000/60=16.66 ms, and when the decoding bin number is 2 (i.e. the number of video frames to be decoded in the decoding queue of the decoder is 2), the video delay is increased to 2×16.66=33.32 ms.

The input buffer refers to a video frame buffer allocated to the decoder by the computer equipment, and video frames received by the computer equipment are transmitted to the decoder for decoding through the input buffer. Specifically, the computer device requests to add a video frame to the input buffer after receiving the video frame, and the video frame is transferred from the input buffer to the decoding queue of the decoder for decoding after being successfully added to the input buffer.

Rendering lost frames means that the computer equipment discards after finishing decoding of a video frame and does not render and present the decoded video frame.

Based on the above technology, the embodiment of the application provides a video processing scheme capable of reducing video time delay. Fig. 1a is a schematic diagram of a video processing system according to an embodiment of the present application. As shown in fig. 1a, the video processing system may comprise a terminal device 101 and a server 102. The video processing method provided by the embodiment of the present application may be performed by the terminal device 101. The terminal device 101 includes, but is not limited to, a smart phone (such as an Android mobile phone, an IOS mobile phone, etc.), a tablet computer, a portable personal computer, an intelligent home appliance, a vehicle-mounted terminal, a wearable device, an unmanned aerial vehicle, etc. with a video playing function, which is not limited in the embodiment of the present application. The server 102 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), and basic cloud computing services such as big data and artificial intelligence platforms.

It should be noted that, the terminal device 101 and the server 102 may be directly or indirectly connected through a wired communication or a wireless communication, and the present application is not limited herein. The number of terminal devices 101 and servers 102 is for example only and does not constitute a practical limitation of the application, and the video processing system may also comprise, for example, terminal devices 103 or servers 104.

Fig. 1b is a flowchart of a video processing scheme according to an embodiment of the present application. As shown in fig. 1b, the general flow of the video processing scheme is as follows:

(1) The terminal device 101 receives the video to be processed transmitted from the server 102. In one embodiment, the video frames in the video to be processed are sent by the server 102 to the terminal device 101 in real time, that is, the terminal device 101 may decode and render the received video frames while receiving the video frames sent by the server 102. The terminal device 101 updates relevant parameters in real time during the process of decoding the video frames, where the parameters include input buffer time consumption, frame sending number, decoding bin number, and rendering time interval, where the input buffer time consumption may be used to indicate whether the buffer resource corresponding to the decoding queue meets the decoding requirement, the frame sending number may be used to indicate whether the video frame writing speed of the decoding queue corresponding to the decoder is greater than the video frame decoding speed, the decoding bin number may be used to indicate whether the decoding performance of the decoder meets the decoding requirement, and the rendering time interval may be used to indicate whether the video to be processed after the frame loss processing is performed on the decoded video frames can ensure the playing continuity.

The process of decoding video frames and the acquisition of the various parameters are described in detail below:

In one implementation, after receiving a video frame sent by the server 102, the terminal device 101 requests to add the video frame to an input buffer (i.e. a buffer resource corresponding to a decoding queue), and after being successfully added to the input buffer, the video frame is transferred (written) from the input buffer to the decoding queue corresponding to the decoder, and when the video frame is successfully transferred to the decoding queue, the terminal device 101 records an input buffer time of the video frame, where the input buffer time of the video frame is used to indicate a time spent from the first request of the video frame to be added to the input buffer to be successfully transferred to the decoding queue, that is, the input buffer time of the video frame is calculated based on a time spent by the first request of the video frame to be added to the input buffer and a time spent by the video frame to be successfully transferred to the decoding queue. In addition, the terminal device 101 may also count the number of video frames in the decoding queue in real time to obtain a decoded bin number, calculate a time interval between adjacent video frames in the decoding queue according to the time when the video frames in the decoding queue are written into the decoding queue, and further count a frame number, where the adjacent video frames refer to video frames arranged in adjacent positions in the decoding queue, for example, assuming that the decoding queue includes video frames 1-3 and video frames 1-3 are arranged according to sequence numbers, video frame 1 and video frame 2 are adjacent frames, and video frame 2 and video frame 3 are adjacent frames.

Further, when the video frame decoding is completed, the terminal device 101 may calculate a rendering time interval based on the time when the video frame is completed to decode and the time when the third video frame is rendered, wherein the third video frame is a video frame with the shortest time interval between the time when the video frame is completed to decode, and the time when the video frame is completed to decode, for example, assuming that the time when the video frame 1 is completed to decode is 3 rd second, the video frame 2 is rendered at 0.5 th second, the video frame 3 is rendered at 1.1 th second, the video frame 4 is rendered at 1.9 th second, the video frame 5 is rendered at 2.1 th second, and the video frame 6 is rendered at 2.7 th second, then for the video frame 1, the third video frame is the video frame 6, and the rendering time interval=3-2.7=0.3 second. It should be noted that the values and time units in the above examples are only for example, and in practical applications, the rendering time interval unit may be milliseconds.

(2) In response to the completion of decoding the first video frame in the video to be processed, the terminal device 101 obtains a speed variation parameter in a decoding queue corresponding to the video to be processed, where the speed variation parameter is used to indicate whether a video frame writing speed of the decoding queue is greater than a video frame decoding speed. In one embodiment, the speed change parameter in the decoding queue corresponding to the video to be processed includes a frame number, and if the frame number is greater than or equal to a preset frame number (e.g. 2), the terminal device 101 determines that the video frame writing speed of the decoding queue is greater than the video frame decoding speed. In another embodiment, the terminal device 101 may directly count the video frame writing speed and the video frame decoding speed of the decoding queue in a preset period (for example, in the first 10 seconds of the current time), and take them as the speed variation parameters in the decoding queue.

(3) The terminal device 101 obtains rendering parameters of the video to be processed, where the rendering parameters are used to indicate a time interval between video frames, and through the time interval between video frames, the terminal device 101 can determine whether play continuity of the video to be processed can be ensured after rendering frame loss processing is performed on the first video frame. In one embodiment, the rendering parameter includes a rendering time interval, and if the rendering time interval between the first video frame and the third video frame is less than or equal to the second time interval threshold (e.g. 50 ms), the terminal device 101 determines that the video to be processed after the rendering frame loss processing is performed on the first video frame can ensure the playing continuity (i.e. the time interval between the video frames meets the rendering frame loss condition). It should be noted that, the second time interval threshold is used to ensure continuity of the video, and the second time interval threshold may be a fixed value or may be calculated based on a target decoding frame rate corresponding to the video to be processed, for example, the second time interval threshold=1000/target decoding frame rate×2.

(4) If the speed change parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed (for example, the number of frames is greater than or equal to the preset number of frames), and the time interval between the video frames indicated by the rendering parameter satisfies the rendering frame loss condition (for example, the rendering time interval is less than or equal to the second time interval threshold), the terminal device 101 performs rendering frame loss processing on the first video frame.

Accordingly, if the speed change parameter indicates that the video frame writing speed of the decoding queue is less than or equal to the video frame decoding speed (for example, the number of frames sent is less than the preset number of frames), and the time interval between the video frames indicated by the rendering parameter satisfies the rendering frame loss condition, the terminal device 101 may acquire the buffering parameter, where the buffering parameter is used to indicate whether the buffering resource corresponding to the decoding queue satisfies the decoding requirement. In one embodiment, the buffering parameter includes an input buffering time of a second video frame, the second video frame being a last enqueued video frame in the decoding queue, for example, assuming that the decoding queue includes video frames 1-5 and the enqueuing order of video frames 1-5 is the same as the sequence number, the second video frame is referred to as video frame 5. If the input buffer time consumption of the second video frame is greater than the buffer time consumption threshold (e.g. 10 ms), the terminal device 101 determines that the buffer resource corresponding to the decoding queue does not meet the decoding requirement. Further, if the buffer parameter indicates that the buffer resource corresponding to the decoding queue does not meet the decoding requirement, the terminal device 101 determines a rendering frame loss frequency based on the buffer parameter (e.g., the terminal device 101 may determine the rendering frame loss frequency based on the corresponding relationship between the buffer parameter and the rendering frame loss frequency), and performs rendering frame loss processing on the video frame decoded in the target time period according to the rendering frame loss frequency. For example, if the fourth video frame is a video frame after the second video frame, and when the fourth video frame is added to the decoding queue, if the terminal device 101 detects that the input buffering time of the fourth video frame is less than the buffering time threshold, the terminal device 101 may determine the time period from the time when the second video frame is added to the decoding queue to the time when the fourth video frame is added to the decoding queue as the target time period.

If the speed variation parameter indicates that the video frame writing speed of the decoding queue is less than or equal to the video frame decoding speed (for example, the number of frames sent is less than the preset number of frames), and the time interval between the video frames indicated by the rendering parameter satisfies the rendering frame loss condition, the terminal device 101 may further obtain a decoding performance parameter, where the decoding performance parameter is used to indicate whether the decoding performance of the decoder satisfies the decoding requirement. In one embodiment, the decoding performance parameter includes a number of decoding bins at a first time, where the number of decoding bins at the first time is used to indicate a number of video frames in the decoding queue at the first time, and if the number of decoding bins at the first time is greater than a threshold of the number of decoding bins at the first time, the terminal device 101 determines that the decoding performance of the decoder at the first time does not meet the decoding requirement. Further, if the decoding performance parameter indicates that the decoding performance of the decoder does not meet the decoding requirement, the terminal device 101 determines the rendering frame loss frequency as the first frequency, and performs rendering frame loss processing on the video frames decoded in the first period according to the first frequency. The first period may be a preset fixed period, or a period dynamically determined by the terminal device based on the related parameters, and in the subsequent process, the terminal device 101 may also adjust (increase or decrease) the first frequency based on the decoding bin number at the current time.

Based on the video processing scheme, the embodiment of the application provides a more detailed video processing method, and the video processing method provided by the embodiment of the application will be described in detail with reference to the accompanying drawings.

Referring to fig. 2, fig. 2 is a flowchart of a video processing method according to an embodiment of the present application, where the video processing method may be performed by a computer device, and the computer device may be the terminal device 101 shown in fig. 1 a. As shown in fig. 2, the video processing method may include steps S201 to S203:

S201, responding to the completion of decoding of a first video frame in the video to be processed, and acquiring a speed change parameter in a decoding queue corresponding to the video to be processed.

The speed change parameter in the decoding queue corresponding to the video to be processed is used for indicating whether the video frame writing speed of the decoding queue is greater than the video frame decoding speed. In one embodiment, the speed variation parameter in the decoding queue corresponding to the video to be processed includes a frame number indicating a number of video frames written in the decoding queue with a time interval less than a first time interval threshold. If the number of frames is greater than or equal to the preset number of frames (e.g., 2), the computer device determines that the video frame writing speed of the decoding queue is greater than the video frame decoding speed.

In one embodiment, the decoding queue contains M video frames, M being an integer greater than 1. The specific implementation mode of the computer equipment for obtaining the frame sending number is that the time for writing M video frames into a decoding queue is obtained, the writing time interval between adjacent video frames in the M video frames is calculated based on the time for writing the M video frames into the decoding queue, the adjacent video frames are video frames which are arranged in adjacent positions in the decoding queue, for example, the decoding queue is assumed to comprise video frames 1-3, and the video frames 1-3 are arranged according to the sequence number, the video frames 1 and 2 are adjacent frames, and the video frames 2 and 3 are adjacent frames. It will be appreciated that M video frames correspond to the write time interval between M-1 adjacent video frames. Further, the computer device may count the number of frames to be transmitted according to the write time interval and the first time interval threshold value between M-1 adjacent video frames corresponding to the M video frames, specifically, the number of frames to be transmitted=k+1, where K is the number of write time intervals greater than the first time interval threshold value among the write time intervals between M-1 adjacent video frames, and K is a positive integer less than M. For example, assuming that the decoding queue includes video frames 1-3, and video frames 1-3 are arranged in sequence number, the write time interval between video frame 1 and video frame 2 is greater than a first time interval threshold, and the write time interval between video frame 2 and video frame 3 is less than the first time interval threshold, then the number of frames=1+1=2.

S202, obtaining rendering parameters of the video to be processed.

The rendering parameters are used to indicate the time interval between video frames. The time interval may specifically refer to a rendering time interval or a playing time interval. The computer equipment can judge whether the playing continuity of the video to be processed can be ensured after the first video frame is subjected to rendering frame loss processing through the time interval between the video frames.

In one embodiment, the rendering parameters include a rendering time interval. The rendering time interval is calculated based on the time when the first video frame finishes decoding and the time when the third video frame finishes decoding, wherein the third video frame is the video frame with the shortest time interval between the time when the first video frame finishes decoding in the rendered video frames, for example, assuming that the time when the video frame 1 finishes decoding is 3 rd second, the rendered video frame comprises that the video frame 2 is rendered at 0.5 th second, the video frame 3 is rendered at 1.1 st second, the video frame 4 is rendered at 1.9 th second, the video frame 5 is rendered at 2.1 th second, and the video frame 6 is rendered at 2.7 th second, and then the third video frame is the video frame 6, and the rendering time interval=3-2.7=0.3 second for the video frame 1. It should be noted that the values and time units in the above examples are only for example, and in practical applications, the rendering time interval unit may be milliseconds. Further, if the rendering time interval between the first video frame and the third video frame is less than or equal to the second time interval threshold (e.g., 50 ms), the computer device determines that the video to be processed after the rendering frame loss processing is performed on the first video frame can ensure the playing continuity (i.e., the time interval between the video frames satisfies the rendering frame loss condition). It should be noted that, the second time interval threshold is used to ensure continuity of the video, and the second time interval threshold may be a fixed value or may be calculated based on a target decoding frame rate corresponding to the video to be processed, for example, the second time interval threshold=1000/target decoding frame rate×2 (which indicates that the rendering frame rate of the video to be processed after the rendering frame loss processing is reduced to half of the decoding frame rate of the video to be processed).

Optionally, the rendering parameter includes a playing time interval, and the computer device may calculate the playing time interval based on the playing time of the first video frame in the video to be processed and the playing time of the target video frame in the video to be processed, where if the playing time interval of the first video frame and the target video frame is less than or equal to a second time interval threshold (e.g. 50 ms), the computer device determines that the video to be processed after the frame loss processing is performed on the first video frame can ensure playing continuity (i.e. the time interval between the video frames satisfies the rendering frame loss condition). The target video frame is a video frame with the shortest playing time interval between the playing time of the video frame 1 in the video to be processed and the playing time of the first video frame in the video to be processed, for example, the playing time of the video frame 1 in the video to be processed is assumed to be 2 nd second, the video frame 2 with the playing time of the video to be processed being 0.5 th second, the video frame 3 with the playing time of the video to be processed being 1 st second, the video frame 4 with the playing time of the video to be processed being 1.2 th second and the video frame 5 with the playing time of the video to be processed being 1.7 th second are assumed to be included in the rendered video frame, and then the target video frame is the video frame 5 with the playing time interval=2-1.7=0.3 second. It should be noted that the values and time units in the above examples are only for example, and in practical applications, the playing time interval unit may be milliseconds.

S203, if the speed change parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed and the time interval between the video frames indicated by the rendering parameter meets the rendering frame loss condition, rendering frame loss processing is performed on the first video frame.

In one embodiment, the speed change parameter in the decoding queue corresponding to the processed video includes a frame number, the rendering parameter includes a rendering time interval, the rendering time interval is used for indicating a time interval between a time when the first video frame finishes decoding and a time when the third video frame performs rendering, and the third video frame is a video frame with a shortest time interval between a time when the first video frame finishes decoding and a time when the third video frame performs rendering in the video frames where the video to be processed is rendered. If the number of frames sent is greater than or equal to the preset number of frames and the rendering time interval is less than or equal to the second time interval threshold (i.e. the time interval between video frames meets the rendering frame loss condition), the computer equipment performs rendering frame loss processing on the first video frame.

Optionally, for consecutive S video frames with write time intervals smaller than the first time interval threshold in the decoding queue, S is an integer greater than 1, the computer may keep any one of the S decoded video frames corresponding to the S video frames (e.g., the first video frame in the S video frames, the last video frame in the S video frames, etc.) for rendering, and discard the other S-1 decoded video frames, and the computer device may also discard all the S decoded video frames corresponding to the S video frames.

Referring to fig. 3, fig. 3 is a flowchart of another video processing method according to an embodiment of the present application, where the video processing method may be performed by a computer device, and the computer device may be the terminal device 101 shown in fig. 1 a. As shown in fig. 3, the video processing method may include steps S301 to S308:

S301, responding to the completion of decoding of a first video frame in the video to be processed, and acquiring a speed change parameter in a decoding queue corresponding to the video to be processed.

S302, obtaining rendering parameters of the video to be processed.

The specific embodiments of step S301 and step S302 may refer to the embodiments of step S201 and step S202 in fig. 2, and will not be described herein.

In one embodiment, if the speed variation parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed (e.g., the speed variation parameter includes a frame number greater than or equal to a preset frame number), and the time interval between the video frames indicated by the rendering parameter satisfies the rendering frame loss condition (e.g., the rendering parameter includes a rendering time interval, and the rendering time interval is less than or equal to a second time interval threshold), the computer device performs rendering frame loss processing on the first video frame.

In another embodiment, the speed change parameter in the decoding queue for the processed video includes a frame number. If the number of frames sent is less than the preset number of frames and the time interval between video frames indicated by the rendering parameters satisfies the rendering frame loss condition, the computer device continues to execute step S303.

In yet another embodiment, the speed change parameter in the decode queue for the processed video includes a frame number. If the number of frames sent is less than the preset number of frames and the time interval between video frames indicated by the rendering parameters satisfies the rendering frame loss condition, the computer device continues to execute step S306.

In yet another embodiment, if the speed change parameter indicates that the video frame writing speed of the decoding queue is less than or equal to the video frame decoding speed, the computer device may render the first video frame and output the first video frame.

In yet another embodiment, if the time interval between video frames indicated by the rendering parameter does not satisfy the rendering frame loss condition (e.g., the rendering parameter includes a rendering time interval, and the rendering time interval is greater than the second time interval threshold), the computer device renders the first video frame and outputs the first video frame.

S303, obtaining a cache parameter.

The buffer parameter is used for indicating whether the buffer resource corresponding to the decoding queue (i.e. the input buffer allocated to the decoder by the computer device) meets the decoding requirement. In one embodiment, the buffering parameter includes an input buffering time of a second video frame, the second video frame being a last enqueued video frame in the decoding queue, for example, assuming that the decoding queue includes video frames 1-5 and the enqueuing order of video frames 1-5 is the same as the sequence number, the second video frame is referred to as video frame 5. If the input buffer time consumption of the second video frame is greater than the buffer time consumption threshold (e.g., 10 ms), the computer device determines that the buffer resource corresponding to the decoding queue does not meet the decoding requirement.

In one embodiment, the second video frame is transferred from the input buffer to the decode queue, the computer device obtains a time when the second video frame first requests to join the input buffer and a time when the second video frame is successfully transferred to the decode queue, and calculates an input buffer time for the second video frame based on the time when the second video frame first requests to join the input buffer and the time when the second video frame is successfully transferred to the decode queue. Specifically, the input buffering time of the second video frame = the time the second video frame was successfully delivered to the decode queue-the time the second video frame first requested to join the input buffer.

And S304, if the buffer parameters indicate that the buffer resources corresponding to the decoding queues do not meet the decoding requirements, determining the rendering frame loss frequency based on the buffer parameters.

In one embodiment, the buffering parameter includes an input buffering time of a second video frame, the second video frame being a last enqueued video frame in the decoding queue. If the input buffering time consumption of the second video frame is greater than a buffering time consumption threshold (e.g., 10 ms), the computer device determines a rendering frame loss frequency based on the input buffering time consumption of the second video frame. Specifically, the computer device may determine a time-consuming interval to which the input buffer time of the second video frame belongs, and determine, according to a correspondence between the time-consuming interval and the frame loss frequency, to be the rendering frame loss frequency. In one implementation, the correspondence between the time-consuming interval and the frame loss frequency may be as shown in table 1 below:

TABLE 1

Time-consuming interval	Frame loss frequency	Description of the invention
			(50ms,∞)	1/2	Every two decoded video frames, one of them is discarded
(40ms,50ms]	1/3	Every three video frames are decoded, one of the video frames is discarded
			(30ms,40ms]	1/4	Every four video frames are decoded, one of the video frames is discarded
(20ms,30ms]	1/5	Every five decoded video frames, one of the video frames is discarded
			(10ms,20ms]	1/6	Every six video frames are decoded, one of the video frames is discarded

As can be seen from table 1, different time intervals correspond to different frame loss frequencies, and the frame loss frequencies are proportional to the input buffer time. It will be appreciated that the time-consuming intervals and frame loss frequencies in table 1 are for illustration only and do not constitute a practical limitation of the present application.

And S305, performing rendering frame loss processing on the video frames decoded in the target time period according to the rendering frame loss frequency.

For example, if the fourth video frame is a video frame after the second video frame, and when the fourth video frame is added to the decoding queue, if the computer device detects that the input buffer time consumption of the fourth video frame is less than or equal to the buffer time consumption threshold, the computer device may determine the time period from when the second video frame is added to the decoding queue to when the fourth video frame is added to the decoding queue as the target time period.

Further, after the target period, the computer device may obtain the input buffer time spent for the last video frame (e.g., a fourth video frame) in the current decoding queue in decoding (enqueuing) order, the fourth video frame being a video frame subsequent to the second video frame. If the computer device detects that the input buffer time consumption of the fourth video frame is greater than the buffer time consumption threshold, the computer device can determine a rendering frame loss frequency corresponding to a next time period after the target time period according to the implementation manner in step S304-step S305, and render the frame loss processing to the video frame decoded in the next time period after the target time period according to the rendering frame loss frequency.

Accordingly, after the buffer parameter is obtained, if the buffer parameter indicates that the buffer resource corresponding to the decoding queue meets the decoding requirement (for example, the input buffer time of the second video frame is less than or equal to the buffer time threshold), the computer device renders the first video frame and outputs the first video frame (i.e., no rendering frame loss processing is required).

S306, obtaining decoding performance parameters.

The decoding performance parameter is used to indicate whether the decoding performance of the decoder meets the decoding requirements. In one embodiment, the decoding performance parameter includes a number of frames in the first time and a threshold number of frames in the first time, where the number of frames in the first time is used to indicate a number of video frames in the decoding queue, and the threshold number of frames in the first time is determined based on a decoding performance of the decoder at the first time. If the number of decoding bin frames at the first time is greater than the threshold number of decoding bin frames at the first time, the computer device determines that the decoding performance of the decoder at the first time does not meet the decoding requirement.

S307, if the decoding performance parameter indicates that the decoding performance of the decoder does not meet the decoding requirement, the rendering frame loss frequency is determined as the first frequency.

In one embodiment, the decoding performance parameters include a decoding bin number at the first time and a decoding bin number threshold at the first time, and the computer device determines the rendering frame loss frequency as the first frequency if the decoding bin number at the first time is greater than the decoding bin number threshold at the first time. The first frequency may be a preset frequency, or may be calculated based on a difference between the number of frames stored in the first time and a threshold value of the number of frames stored in the first time, where the first frequency is proportional to the difference between the number of frames stored in the first time and the threshold value of the number of frames stored in the first time, that is, the more the number of frames stored in the first time exceeds the threshold value of the number of frames stored in the first time, the greater the first frequency.

And S308, performing rendering frame loss processing on the video frames decoded in the first time period according to the first frequency.

The first time period may be a preset fixed time period or a time period dynamically determined by the terminal device based on the related parameter, for example, if the computer device detects that the decoding bin number at the second time is less than or equal to the decoding bin number threshold at the second time, the first time period may be a time period corresponding to the first time to the second time.

Further, the decoding performance parameter further includes a decoding bin number at the second time and a decoding bin number threshold at the second time. And according to the second frequency, rendering and frame loss processing is carried out on video frames decoded in a second time period, wherein the second frequency is larger than the first frequency, and the second time period is after the first time period. According to the above method, if after the rendering frame loss processing is performed, the decoding performance parameter still indicates that the decoding performance of the decoder does not meet the decoding requirement, the computer device may further increase the frequency of rendering frame loss until the frequency of rendering frame loss reaches the frequency threshold (which may be determined by the rendering parameters of the video to be processed).

Correspondingly, if the decoding bin number at the second moment is smaller than or equal to the decoding bin number threshold at the second moment, the computer equipment determines the rendering frame loss frequency as a third frequency, and performs rendering frame loss processing on the video frames decoded in the second time period according to the third frequency, wherein the third frequency is smaller than the first frequency, and the second time period is after the first time period. According to the method, if the decoding performance parameter indicates that the decoding performance of the decoder can meet the decoding requirement after the rendering frame loss processing is performed, the computer device can gradually reduce the frequency of the rendering frame loss until the frequency of the rendering frame loss is reduced to zero, or can directly reduce the frequency of the rendering frame loss to zero (i.e. the third frequency can be 0).

On the basis of the embodiment of fig. 2, the embodiment of the application judges whether the video frame writing speed of the decoding queue is higher than the video frame decoding speed through frame sending, reduces video time delay through frame loss by rendering when the video frame writing speed of the decoding queue is higher than the video frame decoding speed (namely, the frame sending number is higher than or equal to a preset frame number) is detected, judges whether the buffer resources corresponding to the decoding queue meet the decoding requirement through the input buffer time consumption, reduces video time delay by rendering the frame loss processing on the video frames which are finished to be decoded when the buffer resources corresponding to the decoding queue are detected to not meet the decoding requirement (namely, the input buffer time consumption of the video frames is higher than the buffer time consumption threshold), and reduces video time delay by rendering the frame loss processing on the video frames which are finished to be decoded according to the set frequency when the decoding performance of the decoding queue is detected to not meet the decoding requirement (namely, the decoding frame number of the current time is higher than the decoding frame storage threshold at the current moment). In addition, the video continuity is ensured through the rendering time interval while the video time delay is reduced, and the video interaction experience is further improved.

The foregoing details of the method of embodiments of the present application are provided for the purpose of better implementing the foregoing aspects of embodiments of the present application, and accordingly, the following provides an apparatus of embodiments of the present application.

Referring to fig. 4, fig. 4 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present application, where the video processing apparatus shown in fig. 4 may be installed in a computer device, and the computer device may specifically be the terminal device 101 shown in fig. 1 a. The video processing device may be adapted to perform some or all of the functions of the method embodiments described above with respect to fig. 2 and 3. Referring to fig. 4, the video processing apparatus includes:

An obtaining unit 401, configured to obtain, in response to completion of decoding a first video frame in the video to be processed, a speed variation parameter in a decoding queue corresponding to the video to be processed, where the speed variation parameter is used to indicate whether a video frame writing speed of the decoding queue is greater than a video frame decoding speed;

the processing unit 402 is configured to perform rendering frame loss processing on the first video frame if the speed variation parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed and the time interval between the video frames indicated by the rendering parameter satisfies the rendering frame loss condition.

In one embodiment, the speed variation parameter in the decoding queue corresponding to the video to be processed includes a frame number indicating a number of video frames written in the decoding queue with a time interval less than a first time interval threshold, and the processing unit 402 is further configured to:

In one embodiment, the processing unit 402 is further configured to:

In one embodiment, the buffering parameter includes an input buffering time of a second video frame, the second video frame being a last enqueued video frame in the decoding queue, and the processing unit 402 is further configured to:

In one embodiment, the buffering parameters include an input buffering time of a second video frame, the second video frame being a last enqueued video frame in the decoding queue, the second video frame being transferred from the input buffering to the decoding queue, and the processing unit 402 is configured to obtain the buffering parameters, specifically configured to:

In one embodiment, the buffering parameter includes an input buffering time of a second video frame, the second video frame being a last enqueued video frame in the decoding queue, and the processing unit 402 is configured to determine a rendering frame loss frequency based on the buffering parameter, specifically configured to:

In one embodiment, the processing unit 402 is further configured to:

In one embodiment, the decoding performance parameter includes a decoding bin number at a first time and a decoding bin number threshold at the first time, the decoding bin number at the first time being indicative of a number of video frames in the decoding queue at the first time, the decoding bin number threshold at the first time being determined based on a decoding performance of the decoder at the first time, the processing unit 402 further configured to:

In one embodiment, the decoding performance parameters further include a decoding bin number at a second time and a decoding bin number threshold at a second time, the second time being after the first time, the processing unit 402 further configured to:

In one embodiment, the processing unit 402 is further configured to:

In one embodiment, the decoding queue contains M video frames, M is an integer greater than 1, and the processing unit 402 is configured to obtain a speed variation parameter in the decoding queue corresponding to the video to be processed, specifically configured to:

Acquiring the time of writing M video frames into a decoding queue;

In one embodiment, the processing unit 402 is further configured to:

According to one embodiment of the present application, some of the steps involved in the video processing methods shown in fig. 2 and 3 may be performed by respective units in the video processing apparatus shown in fig. 4. For example, step S201 and step S202 shown in fig. 2 may be performed by the acquisition unit 401 shown in fig. 4, step S203 shown in fig. 2 may be performed by the processing unit 402 shown in fig. 4, step S301 to step S303 and step S306 shown in fig. 3 may be performed by the acquisition unit 401 shown in fig. 4, and step S304, step S305, step S307 and step S308 may be performed by the processing unit 402 shown in fig. 4. The respective units in the video processing apparatus shown in fig. 4 may be individually or collectively combined into one or several other units, or some unit(s) thereof may be further split into a plurality of units having smaller functions, which can achieve the same operation without affecting the achievement of the technical effects of the embodiments of the present application. The above units are divided based on logic functions, and in practical applications, the functions of one unit may be implemented by a plurality of units, or the functions of a plurality of units may be implemented by one unit. In other embodiments of the present application, the video processing apparatus may also include other units, and in practical applications, these functions may also be implemented with assistance from other units, and may be implemented by cooperation of a plurality of units.

According to another embodiment of the present application, a video processing apparatus as shown in fig. 4 may be constructed by running a computer program (including program code) capable of executing the steps involved in the respective methods as shown in fig. 2 and 3 on a general-purpose computing apparatus such as a computer device including a processing element such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read only storage medium (ROM), and the like, and a storage element, and implementing the video processing method of the embodiment of the present application. The computer program may be recorded on, for example, a computer-readable recording medium, and loaded into and run in the above-described computing device through the computer-readable recording medium.

Based on the same inventive concept, the principles and beneficial effects of the video processing apparatus provided in the embodiments of the present application for solving the problems are similar to those of the video processing method in the embodiments of the present application, and may refer to the principles and beneficial effects of implementation of the method, which are not described herein for brevity.

Referring to fig. 5, fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present application, where the computer device may be a terminal device or a server. As shown in fig. 5, the computer device comprises at least a processor 501, a communication interface 502 and a memory 503. Wherein the processor 501, the communication interface 502 and the memory 503 may be connected by a bus or other means. The processor 501 (or called central processing unit (Central Processing Unit, CPU)) is a computing core and a control core of the computer device, and can analyze various instructions in the computer device and process various data of the computer device, for example, the CPU can be used for analyzing an on-off instruction sent by an object to the computer device and controlling the computer device to perform on-off operation, for example, the CPU can transmit various interactive data between internal structures of the computer device, and the like. Communication interface 502 may optionally include standard wired interfaces, wireless interfaces (e.g., WI-FI, mobile communication interfaces, etc.), and may be controlled by processor 501 to receive and transmit data, and communication interface 502 may also be used for transmission and interaction of data within a computer device. The Memory 503 (Memory) is a Memory device in a computer device for storing programs and data. It will be appreciated that the memory 503 herein may include both built-in memory of the computer device and extended memory supported by the computer device. Memory 503 provides storage space that stores the operating system of the computer device, which may include, but is not limited to, the Android system, the internet operating system (Internetworking Operating System, IOS), and the like, as the application is not limited in this regard.

The embodiment of the application also provides a computer readable storage medium (Memory), which is a Memory device in the computer device and is used for storing programs and data. It is understood that the computer readable storage medium herein may include both built-in storage media in a computer device and extended storage media supported by the computer device. The computer readable storage medium provides storage space that stores a processing system of a computer device. In this memory space, a computer program suitable for being loaded and executed by the processor 501 is stored. The computer readable storage medium may be a high-speed RAM memory, or may be a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory, or may alternatively be at least one computer readable storage medium located remotely from the processor.

In one embodiment, the processor 501 performs the following by running a computer program in the memory 503:

As an optional embodiment, the speed change parameter in the decoding queue corresponding to the video to be processed includes a frame number, where the frame number is used to indicate the number of video frames written in the decoding queue with a time interval less than the first time interval threshold;

The processor 501, by running a computer program in the memory 503, also performs the following operations:

As an alternative embodiment, the processor 501, by running a computer program in the memory 503, also performs the following operations:

As an alternative embodiment, the buffering parameters include the input buffering time of the second video frame, which is the last enqueued video frame in the decode queue, and the processor 501, by running the computer program in the memory 503, also performs the following operations:

As an alternative embodiment, the buffering parameters include the input buffering time of the second video frame, the second video frame being the last enqueued video frame in the decoding queue, the second video frame being transferred from the input buffer to the decoding queue, and the specific embodiment of the processor 501 obtaining the buffering parameters is:

As an alternative embodiment, the buffer parameters include the input buffer time consumption of the second video frame, which is the last enqueued video frame in the decoding queue, and the processor 501 determines, based on the buffer parameters, that the specific embodiment of rendering the frame loss frequency is:

As an alternative embodiment, the decoding performance parameter includes a decoding bin number at the first time and a decoding bin number threshold at the first time, where the decoding bin number at the first time is used to indicate the number of video frames in the decoding queue at the first time, and the decoding bin number threshold at the first time is determined based on the decoding performance of the decoder at the first time;

As an alternative embodiment, the decoding performance parameters further comprise a decoding bin number at a second time and a decoding bin number threshold at the second time, the second time being after the first time, the processor 501 further performs the following by running the computer program in the memory 503:

As an alternative embodiment, the decoding queue contains M video frames, M is an integer greater than 1, and the specific embodiment of the processor 501 obtaining the speed variation parameter in the decoding queue corresponding to the video to be processed is:

Acquiring the time of writing M video frames into a decoding queue;

As an optional embodiment, the rendering parameters of the video to be processed include a rendering time interval, the rendering time interval is used for indicating a time interval between a time when the first video frame finishes decoding and a time when the third video frame renders, the third video frame is a video frame with a shortest time interval between a time when the first video frame finishes decoding in the video frames to be processed rendered, and the time interval between the video frames meets a rendering frame loss condition and includes a rendering time interval being less than or equal to a second time interval threshold.

Based on the same inventive concept, the principle and beneficial effects of the computer device provided in the embodiment of the present application are similar to those of the video processing method in the embodiment of the present application, and may refer to the principle and beneficial effects of implementation of the method, which are not described herein for brevity.

The embodiment of the application also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program is suitable for being loaded by a processor and executing the video processing method of the method embodiment.

The present application also provides a computer program product comprising a computer program adapted to be loaded by a processor and to perform the video processing method of the above method embodiments.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the video processing method described above.

The steps in the method of the embodiment of the application can be sequentially adjusted, combined and deleted according to actual needs.

The modules in the device of the embodiment of the application can be combined, divided and deleted according to actual needs.

Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the above embodiments may be implemented by a program for instructing related hardware, and the program may be stored in a computer readable storage medium, where the readable storage medium may include a flash disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, an optical disk, or the like.

The above disclosure is illustrative of a preferred embodiment of the present application, and it is not to be construed as limiting the scope of the application, but rather as providing for the full or partial flow of the solution to the above-described embodiment, and equivalent variations according to the appended claims, will be apparent to those skilled in the art.

Claims

1. A video processing method, characterized in that the method comprises:

In response to the completion of decoding of the first video frame in the video to be processed, a speed change parameter in a decoding queue corresponding to the video to be processed is obtained, the speed change parameter being used to indicate whether a video frame writing speed of the decoding queue is greater than a video frame decoding speed; the speed change parameter includes a number of sent frames, the number of sent frames being used to indicate the number of video frames in the decoding queue whose writing time interval is less than a first time interval threshold;

Acquire rendering parameters of the video to be processed, where the rendering parameters are used to indicate the time interval between video frames; and determine the playback continuity of the video to be processed after rendering and dropping the first video frame according to the time interval between the video frames;

If the speed change parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed, and the time interval between the video frames indicated by the rendering parameter meets the rendering frame loss condition, then rendering frame loss processing is performed on the first video frame;

If the number of sent frames is less than the preset number of frames, and the time interval between the video frames indicated by the rendering parameter meets the rendering frame loss condition, then obtain a cache parameter, where the cache parameter is used to indicate whether the cache resources corresponding to the decoding queue meet the decoding requirements;

If the cache parameter indicates that the cache resources corresponding to the decoding queue do not meet the decoding requirements, determining a rendering frame loss frequency based on the cache parameter;

According to the rendering frame loss frequency, rendering frame loss processing is performed on the video frames decoded within the target time period.

2. The method according to claim 1, characterized in that the method further comprises:

If the number of sent frames is greater than or equal to the preset number of frames, it is determined that the video frame writing speed of the decoding queue is greater than the video frame decoding speed.

3. The method of claim 1, wherein the cache parameter comprises an input cache time consumption of a second video frame, the second video frame being the last video frame to be queued in the decoding queue; the method further comprising:

If the input cache time consumption of the second video frame is greater than the cache time consumption threshold, it is determined that the cache resources corresponding to the decoding queue do not meet the decoding requirements.

4. The method according to claim 1, wherein the cache parameter includes an input cache time consumption of a second video frame, the second video frame is a video frame that is last queued in the decoding queue; the second video frame is transferred from the input cache to the decoding queue, and the obtaining of the cache parameter includes:

Obtaining the time when the second video frame is first requested to be added to the input buffer, and the time when the second video frame is successfully delivered to the decoding queue;

The input buffer time consumption of the second video frame is calculated according to the time when the second video frame is first requested to be added to the input buffer and the time when the second video frame is successfully transferred to the decoding queue.

5. The method of claim 1, wherein the cache parameter comprises an input cache time consumption of a second video frame, and the second video frame is a last video frame to be queued in the decoding queue; and determining the rendering frame loss frequency based on the cache parameter comprises:

Determine a time consumption interval to which the input buffering time consumption of the second video frame belongs, where different time consumption intervals correspond to different frame loss frequencies;

The frame loss frequency corresponding to the time consumption interval to which the input cache time consumption of the second video frame belongs is determined as the rendering frame loss frequency.

6. The method according to claim 1, characterized in that the method further comprises:

If the cache parameter indicates that the cache resources corresponding to the decoding queue meet the decoding requirements, the first video frame is rendered and output.

7. The method according to claim 2, characterized in that the method further comprises:

If the number of sent frames is less than or equal to the preset number of frames, and the time interval between the video frames indicated by the rendering parameter meets the rendering frame loss condition, then a decoding performance parameter is obtained, where the decoding performance parameter is used to indicate whether the decoding performance of the decoder meets the decoding requirement;

If the decoding performance parameter indicates that the decoding performance of the decoder does not meet the decoding requirement, determining the rendering frame loss frequency as the first frequency;

Perform rendering frame loss processing on the video frames decoded in the first time period according to the first frequency.

8. The method of claim 7, wherein the decoding performance parameter comprises a decoding buffered frame number at a first moment and a decoding buffered frame number threshold at the first moment, the decoding buffered frame number at the first moment is used to indicate the number of video frames in the decoding queue at the first moment, and the decoding buffered frame number threshold at the first moment is determined based on the decoding performance of the decoder at the first moment; the method further comprises:

If the number of decoded frames at the first moment is greater than the threshold of the number of decoded frames at the first moment, it is determined that the decoding performance of the decoder at the first moment does not meet the decoding requirement.

9. The method according to claim 8, wherein the decoding performance parameter further comprises a number of decoded frames at a second moment and a threshold of the number of decoded frames at the second moment, and the second moment is after the first moment; the method further comprises:

If the number of decoded frames at the second moment is greater than the threshold of the number of decoded frames at the second moment, the rendering frame loss frequency is determined to be a second frequency, and the second frequency is greater than the first frequency;

Performing rendering and frame loss processing on video frames decoded in a second time period according to the second frequency, the second time period being after the first time period.

10. The method according to claim 9, characterized in that the method further comprises:

If the number of decoded frames at the second moment is less than or equal to the threshold of the number of decoded frames at the second moment, the rendering frame loss frequency is determined to be a third frequency, and the third frequency is less than the first frequency;

According to the third frequency, rendering and frame loss processing is performed on the video frames decoded in the second time period.

11. The method according to claim 2, wherein the decoding queue contains M video frames, where M is an integer greater than 1; and the step of obtaining the speed change parameter in the decoding queue corresponding to the video to be processed comprises:

Obtain the time when the M video frames are written into the decoding queue;

Based on the time when the M video frames are written into the decoding queue, calculating the writing time interval between adjacent video frames in the M video frames;

The number of sent frames is counted according to the writing time interval between adjacent video frames in the M video frames and the first time interval threshold.

12. The method according to claim 1, further comprising:

If the speed change parameter indicates that the video frame writing speed of the decoding queue is less than or equal to the video frame decoding speed, the first video frame is rendered and output; or

If the time interval between the video frames indicated by the rendering parameters does not satisfy the rendering frame loss condition, the first video frame is rendered and output.

13. The method according to any one of claims 1 to 12 is characterized in that the rendering parameters of the video to be processed include a rendering time interval, and the rendering time interval is used to indicate the time interval between the time when the first video frame completes decoding and the time when the third video frame is rendered, and the third video frame is the video frame with the shortest time interval between the time when the rendered video frames of the video to be processed and the time when the first video frame completes decoding; the time interval between the video frames satisfies the rendering frame loss condition including that the rendering time interval is less than or equal to a second time interval threshold.

14. A video processing device, characterized in that the video processing device comprises:

an acquisition unit, configured to acquire, in response to completion of decoding of a first video frame in a video to be processed, a speed change parameter in a decoding queue corresponding to the video to be processed, the speed change parameter being used to indicate whether a video frame writing speed of the decoding queue is greater than a video frame decoding speed; the speed change parameter includes a number of sent frames, the number of sent frames being used to indicate the number of video frames in the decoding queue whose writing time interval is less than a first time interval threshold;

and for obtaining rendering parameters of the video to be processed, the rendering parameters being used to indicate the time interval between video frames; and judging the playback continuity of the video to be processed after rendering and dropping the first video frame according to the time interval between the video frames;

a processing unit, configured to perform rendering frame loss processing on the first video frame if the speed change parameter indicates that the video frame writing speed of the decoding queue is greater than the video frame decoding speed, and the time interval between the video frames indicated by the rendering parameter satisfies a rendering frame loss condition;

The processing unit is further used to obtain cache parameters if the number of sent frames is less than a preset number of frames and the time interval between the video frames indicated by the rendering parameters meets the rendering frame loss condition, and the cache parameters are used to indicate whether the cache resources corresponding to the decoding queue meet the decoding requirements; if the cache parameters indicate that the cache resources corresponding to the decoding queue do not meet the decoding requirements, determine the rendering frame loss frequency based on the cache parameters; and perform rendering frame loss processing on the video frames decoded within the target time period according to the rendering frame loss frequency.

15. A computer device, comprising: a memory and a processor;

a memory, wherein a computer program is stored in the memory;

A processor, used to load the computer program to implement the video processing method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the video processing method according to any one of claims 1 to 13.