CN116320437B

CN116320437B - Video coding control method, device and equipment

Info

Publication number: CN116320437B
Application number: CN202310354922.1A
Authority: CN
Inventors: 张敏琪
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2023-03-31
Filing date: 2023-03-31
Publication date: 2025-08-12
Anticipated expiration: 2043-03-31
Also published as: CN116320437A

Abstract

The embodiments of this specification disclose a video encoding control method, apparatus, and device. The method includes: obtaining a video frame to be encoded, sending the video frame to a first encoder for encoding; determining whether the first encoder uses hardware encoding; if so, sending the video frame to be encoded to a second encoder for encoding; detecting the encoding performance of the second encoder for the video frame to be encoded, and obtaining corresponding encoding performance data; and determining, based on the encoding performance data, whether to use the second encoder for encoding, over the first encoder and the second encoder, preferentially.

Description

Video coding control method, device and equipment

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to a method, an apparatus, and a device for controlling video coding.

Background

Video coding, which is to convert one video format into another video format, mainly aims at compression and reducing the data volume. Common application scenes are online videos, such as video phones, network live broadcasting and the like, and under the scenes, the video coding capability capable of keeping high definition while having high compression rate has the advantage of being thick on a single day, occupies the least network bandwidth, transmits more video information and achieves narrow-band high definition.

In practical applications, there are a large number of alternative video encoders, but at present, a user often manually sets an encoder according to his own experience or uses a default encoder to encode video, so that the obtained effect is not necessarily good, and the user is hard to perceive and further distinguish, so that a scheme capable of more effectively discriminating an encoder more suitable for the current video encoding task is needed.

Disclosure of Invention

One or more embodiments of the present disclosure provide a video encoding control method, apparatus, device, and storage medium, which are used to solve the technical problem that a scheme capable of more effectively discriminating an encoder more suitable for the current video encoding task is needed.

To solve the above technical problems, one or more embodiments of the present specification are implemented as follows:

One or more embodiments of the present disclosure provide a video coding control method, including:

obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding;

judging whether the first encoder adopts a hardware coding mode or not;

if yes, the video frame to be encoded is also sent to a second encoder for encoding;

detecting the coding performance of the second encoder on the video frame to be coded to obtain corresponding coding performance data;

And judging whether the first encoder and the second encoder are used for encoding preferentially according to the encoding performance data.

One or more embodiments of the present disclosure provide a video encoding control apparatus, including:

The video frame main coding module acquires a video frame to be coded, and sends the video frame to be coded to a first coder for coding;

The coding mode judging module judges whether the first coder adopts a hardware coding mode or not;

the video frame bypass coding module is used for sending the video frame to be coded to a second coder for coding if the video frame to be coded is the video frame bypass coding module;

The bypass performance detection module detects the coding performance of the second encoder on the video frame to be coded to obtain corresponding coding performance data;

And the encoder switching selection module is used for judging whether the first encoder and the second encoder are preferentially used for encoding according to the encoding performance data.

One or more embodiments of the present specification provide a video encoding control apparatus including:

at least one processor, and

A memory communicatively coupled to the at least one processor, wherein,

The memory stores instructions executable by the at least one processor, the instructions are executable by the at least one processor to enable the at least one processor to:

judging whether the first encoder adopts a hardware coding mode or not;

One or more embodiments of the present specification provide a non-volatile computer storage medium storing computer-executable instructions configured to:

judging whether the first encoder adopts a hardware coding mode or not;

The at least one technical scheme adopted by one or more embodiments of the present disclosure can achieve the following beneficial effects that, based on filtering of a hardware coding mode, a suitable detection state that does not affect the video coding task performed by a current main-path encoder as much as possible can be ensured, and in the suitable detection state, the coding performance of other encoders under a bypass detection specified specification is accumulated, so that the encoder with the current performance meeting the requirement can be dynamically screened out in time, and further, the encoder which is possibly better and more suitable for the next video coding task can be timely started as the main-path encoder.

Drawings

In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some of the embodiments described in the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a video coding control method according to one or more embodiments of the present disclosure;

FIG. 2 is a schematic flow diagram of one embodiment of the method of FIG. 1 in one application scenario provided by one or more embodiments of the present disclosure;

FIG. 3 is a flow diagram of an encoding mode branching control scheme provided in one or more embodiments of the present disclosure;

FIG. 4 is a flow diagram of a joint encoding scheme of a main encoder and a bypass encoder provided in one or more embodiments of the present disclosure;

fig. 5 is a schematic structural diagram of a video encoding control device according to one or more embodiments of the present disclosure;

Fig. 6 is a schematic structural diagram of a video encoding control device according to one or more embodiments of the present disclosure.

Detailed Description

The embodiment of the specification provides a video coding control method, a video coding control device, video coding control equipment and a storage medium.

In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.

As mentioned in the background, there are a number of alternative video encoders in practical use. Such as h.264, h.265, AV1, h.266, etc. The compression ratio of the h.265, AV1 and h.266 is improved compared with that of the h.264, but the supporting strength of hardware coding of the encoders is poor, for example, AV1 is taken as an example, most of machines do not support hardware coding of AV1, and software coding is needed, so that coding time spent on coding of the same specification on different machines is different, in the field of Real-time audio-video communication (Real-Time Communication, RTC), the transmission frame per Second (FRAMES PER Second, FPS) of the communication cannot meet the design specification due to the overlong coding time, and therefore, the product requirement cannot be met, but part of machines have better performance, so that the encoder can obtain good performance, and the advantages of the encoder can be utilized to keep high compression ratio and high definition.

In view of this, the present application provides a targeted solution to discriminate whether a target machine can support a target encoder to obtain good performance on the machine, accumulate more performance data to increase the usage coverage rate of the encoder as much as possible, and globally help to perform video encoding tasks on the machine more efficiently and better. The following continues to describe the solution of the application in detail.

Fig. 1 is a flowchart of a video coding control method according to one or more embodiments of the present disclosure. The process can be performed on machines with video coding requirements, such as smartphones, tablet computers, monitoring cameras, etc., and is particularly suitable for being performed on machines with better performance. Some input parameters or intermediate results in the flow allow for manual intervention adjustments to help improve accuracy.

The flow in fig. 1 includes the following steps:

s102, obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding.

There are currently a plurality of video frames to be encoded, and this requirement is taken as a current video encoding task, where the video frames are usually orderly consecutive frames or sampling frames in the same video, and the video frames to be encoded in step S102 belong to the video frames, for example, the first frame or consecutive frames with a previous timing in the video frames, and of course, other video frames to be encoded may exist after this step.

In one or more embodiments of the present disclosure, for a video encoding task, the first encoder is the default encoder or the encoder currently being used, and sending a video frame to be encoded to the first encoder for encoding indicates that the first encoder is currently responsible for performing the video encoding task preferentially, and during a period in which the first encoder remains prioritized, the encoding result of the first encoder will be the formal result, and during this period, the encoding performance exhibited by the first encoder substantially determines the corresponding business effect and user experience.

S104, judging whether the first encoder adopts a hardware coding mode or not.

In one or more embodiments of the present disclosure, the first encoder actively makes a determination as to whether to use a hardware encoding mode, where two main considerations are:

In the first aspect, during the period that the first encoder keeps the priority, the bypass encoder is to be introduced at the same time, the encoding performance of the bypass encoder is also detected by using the current video encoding task, in this case, there may be two or more encoders including the first encoder on the machine where the first encoder is located to repeatedly execute the same video encoding task, in the case that the first encoder adopts a hardware encoding mode, the bypass encoders are introduced to perform bypass detection, so that the influence on the performance is less, and if the first encoder adopts a software encoding mode, the CPU resource may be preempted because the other encoders introduced by performing bypass detection may also adopt a software encoding mode, which is not necessarily beneficial to task completion.

In the second aspect, for a machine with better performance, especially a machine with powerful CPU, in the case that the first encoder adopts a hardware coding mode, it is difficult to fully utilize the capability of the CPU, in this case, other encoders introduced by performing bypass detection may adopt a software coding mode, and may further exhibit better performance effects, so that the trial value is better, which is why the above-mentioned scheme is particularly suitable for being executed on a machine with better performance.

And S106, if yes, sending the video frame to be encoded to a second encoder for encoding.

In one or more embodiments of the present disclosure, as described above, in the case where the first encoder adopts a hardware encoding method, the second encoder is introduced as a bypass encoder, so that the second encoder may also perform encoding tasks performed by the first encoder. The first encoder may be preset as a main encoder, and the second encoder may be set as a bypass encoder for performance to be detected, so as to determine respective positioning of the two during task execution.

Before encoding, a corresponding encoding specification needs to be set for a corresponding encoder, where the encoding specification includes, for example, an encoding resolution (for example, 480P, 720P, 1080P, 2K, 4K, etc.), an encoding rate (for example, 1Mbps, 10 Mbps, 40MMbps, etc.), and the bypass detection is performed to discover an encoder with higher encoding performance, so that a requirement and an expectation for a second encoder can be increased, and a higher encoding specification can be set for the second encoder. If the coding specification set for the first encoder is used as a reference, a higher coding specification is set for the second encoder, for example, the second encoder may be set with the same coding resolution as the first encoder and a higher coding rate than the first encoder, for example, so as to help more efficiently discover other encoders with significantly better performance than the first encoder, and help to find the upper limit of the coding specification applicable to the current coding task together.

S108, detecting the coding performance of the second coder on the video frame to be coded to obtain corresponding coding performance data.

In one or more embodiments of the present disclosure, the encoding performance data includes at least an encoding duration, and may further include, for example, a memory occupancy rate, a CPU occupancy rate, an error rate, and the like of the corresponding process. The encoding performance data can directly influence the subsequent application effect corresponding to the video frame, for example, influence the smoothness degree of video presentation after decoding and decoding, taking video communication between users as an example, even if the network bandwidth is enough, the phenomenon that the video is blocked and dropped can still occur, and other applications in simultaneous operation can be influenced.

S110, judging whether the first encoder and the second encoder are preferentially used for encoding according to the encoding performance data.

In one or more embodiments of the present disclosure, a desired coding performance threshold is preset, and then the coding performance data of the second encoder is compared with the coding performance threshold. If desired, the first encoder may be switched to the second encoder (for example, in the next frame or the next encoding task), the second encoder is used as the formal executor of the encoding task, and after the switching, the first encoder loses priority, or of course, only the relevant data may be recorded, and then decision is made. This approach based on the desire to set the threshold alone has the additional advantage that there is no need to collect the actual encoding performance data of the first encoder.

If the encoder is to be switched dynamically and timely, the real-time coding performance of the first encoder and the second encoder can be compared dynamically, and the encoder with better performance is adopted as the main-path encoder timely, so that one or more times of switching between the first encoder and the second encoder is possible.

In one or more embodiments of the present disclosure, considering that in practical applications, the information amounts of different video frames are different, and the coding environments under different coding times may also change, the coding performance of the same encoder for coding different video frames may also fluctuate, and after a certain degree of accumulation of the coding performance data of the second encoder is required, a more accurate decision can be made. Based on this, since the video frame to be encoded is only one frame or a few frames with a forward timing, after detecting the encoding performance of the second encoder for the video frame to be encoded, the encoding performance of the second encoder can be further detected by using other video frames to be encoded subsequent to the video frame to be encoded, so as to obtain corresponding encoding performance data, until the specified detection task is completed, for example, after the total number of the specified video frames is accumulated, or after the specified certain video is encoded, the specified detection task is considered to be completed.

By the method of fig. 1, filtering based on a hardware coding mode can ensure that the current main encoder is in a suitable detection state which does not affect the video coding task execution as much as possible, and in the suitable detection state, the bypass detection is used for detecting the coding performance of other encoders under the specified specification, and the coding performance data is accumulated, so that the encoder with the current performance meeting the requirement can be dynamically screened out in time, and further, the encoder which is possibly better and more suitable for the next video coding task can be started in time to serve as the main encoder.

Based on the method of fig. 1, the present specification also provides some specific embodiments and extensions of the method, and the following description will proceed.

More intuitively, in one application scenario provided by one or more embodiments of the present disclosure, a schematic flow chart of a specific implementation of the method in fig. 1 is shown in fig. 2. The flow of fig. 2 relates to a video input module, a video coding control module, a first encoder, a second encoder, a coding switching module, and a coding performance statistics platform.

The flow of fig. 2 includes the steps of:

by operating the video coding control module, the coding control is initialized, the first encoder is currently selected as a formal encoder, and is also used as a main encoder, the target detects the second encoder, namely the second encoder is used as a bypass encoder, and the second encoder can be especially an encoder adopting a software coding mode.

The video input module triggers the video coding control module to start executing control actions aiming at specific frames through a first video frame V1 to be coded.

The video coding control module initializes the first encoder and sets corresponding coding specifications, such as coding resolution, coding rate, etc.

The first encoder may adopt a hardware encoding mode or a software encoding mode, and the video encoding control module judges the encoding mode adopted by the first encoder. If the hardware coding mode is adopted, the current coding brings little pressure to the CPU of the machine where the current coding is, the second coder can be considered to be started for bypass detection, specifically, the second coder is created and initialized, corresponding coding specifications are set, for example, the coding specifications are not lower than those of the first coder, and the total number of expected detection coding frames can be set, which is called detection target times for short.

After the first encoder is ready (present and initialized), the video encoding control module sends the video frame V1 to be encoded to the first encoder for encoding, and if the second encoder is also ready, the video encoding control module sends the video frame V1 to be encoded to the second encoder for encoding, and adds 1 to the count of the detected encoding frames, which indicates that the new frame V1 to be encoded is added to be detected.

Other video frames to be encoded after the video frame to be encoded V1 may be sequentially sent to the first encoder for encoding and the second encoder for encoding detection. For example, for the nth video frame Vn to be encoded, the video frame Vn to be encoded is sent to the first encoder to be encoded, if the second encoder is ready and the detection task has not yet ended, the video frame Vn to be encoded is also sent to the second encoder to be encoded, the statistics of the number of detection encoding frames is added by 1, in the detection process, the encoding performance data of the second encoder is recorded, when the statistics number of the number of detection encoding frames reaches the detection target number of times, the detection can be ended, and the performance detection result is generated according to the successively recorded encoding performance data.

Assuming that the encoding performance is mainly measured from the encoding time length angle, the average encoding time length of the video frames corresponding to the second encoder can be determined according to the corresponding encoding time length of the video frames encoded in the detection task, and if the average encoding time length of the video frames is smaller than a set threshold (representing the expected performance), the next priority of encoding by using the second encoder can be judged. If the performance of the second encoder meets the requirement, for example, the average encoding duration of the video frames is smaller than the set threshold, the encoding switching module can switch the main encoder from the first encoder to the second encoder immediately, and in addition, the performance data can be reported to the encoding performance statistics platform, so that the second encoder is formally used for the first time as the encoder with default priority when the subsequent decision is made whether to execute the encoding task later. If the second encoder is a novel encoder, there may be performance advantages, especially in this case, the coverage rate of the second encoder for the overall encoding task of the platform may be gradually enlarged based on the performance data accumulated by the platform, which helps to more stably and reliably popularize the use of the second encoder in actual services (such as live game, video conference, etc.).

In one or more embodiments of the present disclosure, in the method of fig. 1, mainly a case where the first encoder adopts a hardware encoding manner is considered. It is also mentioned that the supporting strength of hardware coding of some encoders is lacking, and in practical applications, it will be the case that the first encoder adopts a software coding mode, and for this case, considering that the current frame is subjected to intra-frame region dynamic segmentation, the second encoder also tries to participate in the formal coding work of the small region, and will perform bypass performance detection based on the small region. Based on this, one or more embodiments of the present disclosure further provide a flow diagram of an encoding mode branching control scheme, as shown in fig. 3.

The flow in fig. 3 includes the following steps:

S302, obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding.

S304, judging whether the first encoder adopts a hardware encoding mode or not.

And S306, if yes, sending the video frame to be encoded to a second encoder to perform full-area detection encoding, and obtaining corresponding encoding performance data.

Full-area detection encoding refers to the fact that for any video frame to be encoded, which is involved in a detection task, the second encoder encodes all areas within the frame, but the resulting encoding result is only for detection encoding performance and is not used as a formal encoding result for the frame.

And S308, otherwise, sending the video frame to be encoded to a second encoder for local region joint formal encoding to obtain corresponding encoding performance data.

If the hardware coding mode is not adopted, the software coding mode is considered to be adopted, and the second coder and the first coder are likely to adopt the software coding mode, in this case, the first coder and the second coder mainly depend on the computing resources of the CPU on the same machine to carry out coding, and even there may be a resource competition relationship between the two, in this case, the first coder and the second coder are considered to combine to carry out formal coding, and for any video frame to be coded, the two are respectively responsible for coding a part of areas, and coding results obtained by the two are used as formal coding results of the frame.

Further, based on the idea of local region joint formal coding, one or more embodiments of the present disclosure further provide a flow chart of a joint coding scheme of a main encoder and a bypass encoder, as shown in fig. 4.

The flow in fig. 4 includes the steps of:

S402, obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding.

S404, judging whether the first encoder adopts a hardware coding mode or not.

If not, and the first encoder is determined to adopt a software coding mode, the method is executed, namely a first area and a second area are divided in the video frame to be coded (other video frames to be coded can be divided in the same mode in sequence), the first area is coded by the first encoder, and the second area is coded by the second encoder.

For the second encoder, since the performance is less clear at the initial joint encoding, the second encoder may first try to encode the smaller region, and the second region may be initially smaller or even much smaller than the first region. When dividing, for example, each frame can be directly divided by a transverse line or a longitudinal line, the mode can conveniently divide the region proportion directly according to each side length of the video frame to be encoded, if the image recognition and division capability is supported, the foreground and the background in the image can be considered to be dynamically recognized and are respectively encoded by the first encoder and the second encoder, and the scheme has the advantages that the background is relatively unimportant, the background is encoded by the second encoder, and the fault tolerance is good.

And combining the coding results of the same frame by the first coder and the second coder respectively to obtain the whole coding result of the frame, distinguishing the first area from the second area when decoding is carried out later, adopting different decoders to decode, and combining the decoding results.

S408, detecting the coding performance of the second coder on the second region to obtain corresponding coding performance data;

and S410, judging whether to update the duty ratio of the first area and the second area according to the coding performance data so as to code the subsequent video frames according to the updated duty ratio.

If the corresponding coding performance data of the second encoder continuously meets the set performance requirement, the duty ratio of the second area relative to the first area can be correspondingly enlarged gradually until the second encoder is completely switched to be independently coded by the second encoder.

Further, in joint encoding, it is difficult to accurately set the region ratio, so that the first encoder and the second encoder complete encoding the corresponding region of the same frame at the same time, and the encoder that completes first may need to wait for the other encoder to complete the current frame, and if the waiting time is long, the encoding time of the whole frame may be compromised. In order to reduce the latency, an area is additionally divided in the video frame to be encoded, called a third area, as an advantageous preemptive encoding area, in which case the first, second and third areas in the same frame together constitute the complete frame. For the same frame, either the first encoder or the second encoder may complete encoding the corresponding region of the frame first, the encoder that completes first is called the current frame dominant encoder, the current frame dominant encoder encodes the third region of the frame, and the third region may be set to be a region with smaller proportion, so that the time when the later encoder completes encoding the corresponding region is closer to the time when the current frame dominant encoder completes encoding the third region, which may help reduce the waiting time.

In one or more embodiments of the present disclosure, the second region has a smaller duty cycle than the first region, the third region has a smaller duty cycle than the second region, e.g., the first region is set to a five-tenth duty cycle, the second region is set to a four-tenth duty cycle, and the third region is set to a one-tenth duty cycle.

In one or more embodiments of the present disclosure, since the encoder that is completed first is uncertain, in order to avoid collisions or repetitively encoding the third region as much as possible, a status indication bit is set for the third region. Taking the example that the status indication bit is a binary bit (for example, the value is 0 or 1), after the first encoder or the second encoder finishes encoding the corresponding region, inquiring whether the status indication bit is at a first value (representing that the third region of the current frame is idle and not yet begins encoding), if so, setting the status indication bit to be a second value (representing that the third region is occupied and begins encoding), determining the status indication bit as a dominant encoder of the current frame, encoding the third region, and setting the status indication bit to be a first value (representing that the third region of the next frame is idle and not yet begins encoding) after the encoding of the current frame is finished, otherwise, waiting for encoding the next frame if the status indication bit is at the second value.

Based on the same thought, one or more embodiments of the present disclosure further provide apparatuses and devices corresponding to the above method, as shown in fig. 5 and fig. 6. The apparatus and device are capable of performing the above method and related alternatives accordingly.

Fig. 5 is a schematic structural diagram of a video coding control device according to one or more embodiments of the present disclosure, where the device includes:

The video frame main path coding module 502 acquires a video frame to be coded, and sends the video frame to be coded to the first coder for coding;

An encoding mode determining module 504, configured to determine whether the first encoder adopts a hardware encoding mode;

The video frame bypass encoding module 506 sends the video frame to be encoded to a second encoder for encoding if yes;

the bypass performance detection module 508 detects the coding performance of the second encoder on the video frame to be coded to obtain corresponding coding performance data;

the encoder switching selection module 510 determines whether the first encoder and the second encoder are preferentially encoded by using the second encoder according to the encoding performance data.

Optionally, before the video frame to be encoded is sent to a first encoder for encoding, the video frame main encoding module 502 sets the first encoder as a main encoder, initializes the first encoder, and sets a corresponding encoding specification;

the video frame bypass encoding module 506 creates and initializes a second encoder as a bypass encoder for performance to be detected, and sets a corresponding encoding specification;

The video frame to be encoded is also sent to the second encoder ready for encoding.

Optionally, the coding specification includes a coding resolution and a coding rate;

the coding specification correspondingly set for the bypass encoder is higher than the coding specification correspondingly set for the main encoder.

Optionally, the encoder switching selection module 510 continuously detects the encoding performance of the second encoder by using the video frame subsequent to the video frame to be encoded, to obtain corresponding encoding performance data, until a specified detection task is completed;

And judging whether the second encoder is used for encoding preferentially next time according to the encoding performance data corresponding to the video frames encoded in the detection task.

Optionally, the encoder switch selection module 510 determines an average encoding duration of the video frames corresponding to the second encoder according to the encoding duration corresponding to the video frames encoded in the detection task;

and if the average encoding duration of the video frames is smaller than a set threshold value, judging that the second encoder is used for encoding preferentially next time.

Optionally, the method further comprises:

After the determining whether the first encoder adopts the hardware encoding mode, the single frame joint encoding module 512 performs:

dividing a first region and a second region in the video frame to be encoded;

The first region is encoded with the first encoder and the second region is encoded with the second encoder.

Optionally, the bypass performance detecting module 508 detects the encoding performance of the second encoder on the second area after the second encoder is used to encode the second area, so as to obtain corresponding encoding performance data;

The encoder switch selection module 510 determines whether to update the duty ratio of the first area and the second area according to the encoding performance data, so as to encode the subsequent video frame according to the updated duty ratio.

Optionally, the encoder switch selection module 510 gradually enlarges the duty ratio of the second area relative to the first area correspondingly until the second encoder is completely switched to be encoded by the second encoder alone if the encoding performance data corresponding to the second encoder continuously meets the set performance requirement.

Optionally, the single frame joint encoding module 512 divides a third region in the video frame to be encoded, and sets a status indication bit for the third region;

after the first encoder is adopted to encode the first region and the second encoder is adopted to encode the second region, after the first encoder or the second encoder finishes encoding the corresponding region, inquiring whether the state indicating bit is in a first value;

if yes, the state indication position is taken as a second value, the state indication position is determined to be taken as a current frame dominant encoder, the third area is encoded, and after the encoding of the current frame is finished, the state indication position is taken as the first value;

Otherwise, if the status indication bit is at the second value, waiting for encoding for a next frame.

Optionally, the second region has a smaller duty cycle than the first region, and the third region has a smaller duty cycle than the second region.

Fig. 6 is a schematic structural diagram of a video coding control device according to one or more embodiments of the present disclosure, where the device includes:

at least one processor, and

A memory communicatively coupled to the at least one processor, wherein,

judging whether the first encoder adopts a hardware coding mode or not;

Based on the same considerations, one or more embodiments of the present specification further provide a non-volatile computer storage medium storing computer-executable instructions configured to:

judging whether the first encoder adopts a hardware coding mode or not;

In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable GATE ARRAY, FPGA)) is an integrated circuit whose logic functions are determined by user programming of the device. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented with "logic compiler (logic compiler)" software, which is similar to the software compiler used in program development and writing, and the original code before being compiled is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but HDL is not just one, but a plurality of kinds, such as ABEL(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby Hardware Description Language), and VHDL (Very-High-SPEED INTEGRATED Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application SPECIFIC INTEGRATED Circuits (ASICs), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present specification.

It will be appreciated by those skilled in the art that the present description may be provided as a method, system, or computer program product. Accordingly, the present specification embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description embodiments may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.

The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus, devices, non-volatile computer storage medium embodiments, the description is relatively simple, as it is substantially similar to method embodiments, with reference to the section of the method embodiments being relevant.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The foregoing is merely one or more embodiments of the present description and is not intended to limit the present description. Various modifications and alterations to one or more embodiments of this description will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, or the like, which is within the spirit and principles of one or more embodiments of the present description, is intended to be included within the scope of the claims of the present description.

Claims

1. A video coding control method, comprising:

judging whether the first encoder adopts a hardware coding mode or not;

2. The method of claim 1, the method further comprising, prior to sending the video frame to be encoded to a first encoder for encoding:

setting a first encoder as a main encoder, initializing the first encoder, and setting corresponding encoding specifications;

The step of sending the video frame to be encoded to a second encoder for encoding specifically includes:

creating and initializing a second encoder serving as a bypass encoder of the performance to be detected, and setting corresponding encoding specifications;

3. The method of claim 2, the coding specification comprising a coding resolution, a coding rate;

4. The method according to claim 1, wherein said determining whether to preferentially use the second encoder for encoding in the first encoder and the second encoder according to the encoding performance data specifically comprises:

Continuing to detect the coding performance of the second coder by using the video frames subsequent to the video frame to be coded to obtain corresponding coding performance data until a specified detection task is completed;

5. The method according to claim 4, wherein the determining whether to use the second encoder for encoding is performed next preferentially according to the encoding performance data corresponding to the video frame encoded in the probing task, specifically includes:

Determining the average encoding time length of the video frames corresponding to the second encoder according to the corresponding encoding time length of the video frames encoded in the detection task;

6. The method of claim 1, wherein after determining whether the first encoder employs a hardware encoding scheme, the method further comprises:

if not, and the first encoder is determined to adopt a software coding mode, executing:

dividing a first region and a second region in the video frame to be encoded;

7. The method of claim 6, after said encoding said second region with said second encoder, further comprising:

detecting the coding performance of the second coder on the second region to obtain corresponding coding performance data;

and judging whether to update the duty ratio of the first area and the second area according to the coding performance data so as to code the subsequent video frames according to the updated duty ratio.

8. The method of claim 7, wherein the determining whether to update the duty ratio of the first area to the second area according to the coding performance data specifically comprises:

if the corresponding coding performance data of the second encoder continuously meets the set performance requirement, the duty ratio of the second area relative to the first area is correspondingly gradually enlarged until the second encoder is completely switched to be independently coded by the second encoder.

9. The method of claim 6, wherein the dividing the first region and the second region in the video frame to be encoded further comprises:

Dividing a third region in the video frame to be encoded, and setting a state indicating bit for the third region;

After the encoding of the first region with the first encoder and the encoding of the second region with the second encoder, the method further comprises:

After the first encoder or the second encoder completes the corresponding region encoding, inquiring whether the state indicating bit is in a first value;

10. The method of claim 9, wherein the second region has a smaller duty cycle than the first region and the third region has a smaller duty cycle than the second region.

11. A video encoding control apparatus comprising:

12. The apparatus of claim 11, the video frame main encoding module setting a first encoder as a main encoder, initializing the first encoder, and setting a corresponding encoding specification before the video frame to be encoded is sent to the first encoder for encoding;

The video frame bypass coding module creates and initializes a second coder as a bypass coder with performance to be detected, and sets corresponding coding specifications;

13. The apparatus of claim 12, the coding specification comprising a coding resolution, a coding rate;

14. The apparatus of claim 11, wherein the encoder switch selection module continues to detect the encoding performance of the second encoder using a video frame subsequent to the video frame to be encoded to obtain corresponding encoding performance data until a specified detection task is completed;

15. The apparatus of claim 14, wherein the encoder switch selection module determines an average encoding duration of the video frames corresponding to the second encoder according to the encoding durations of the video frames encoded in the probing task;

16. The apparatus of claim 11, further comprising:

And after the judging whether the first encoder adopts a hardware coding mode, if not, and determining that the first encoder adopts a software coding mode, executing:

dividing a first region and a second region in the video frame to be encoded;

17. The apparatus of claim 16, the bypass performance detection module detecting a coding performance of the second encoder for the second region after the second encoder is used to encode the second region, to obtain corresponding coding performance data;

And the encoder switching selection module judges whether to update the duty ratio of the first area and the second area according to the coding performance data so as to code the subsequent video frames according to the updated duty ratio.

18. The apparatus of claim 17, the encoder switch selection module to, if the encoding performance data corresponding to the second encoder continues to meet a set performance requirement, correspondingly gradually expand the duty cycle of the second region relative to the first region until a complete switch to encoding by the second encoder alone.

19. The apparatus of claim 16, the single frame joint coding module to divide a third region in the video frame to be coded and set a status indication bit for the third region;

20. The apparatus of claim 19, the second region having a smaller duty cycle than the first region and the third region having a smaller duty cycle than the second region.

21. A video encoding control apparatus comprising:

at least one processor, and

A memory communicatively coupled to the at least one processor, wherein,

The memory stores instructions executable by the at least one processor, the instructions are executable by the at least one processor to enable the at least one processor to perform:

judging whether the first encoder adopts a hardware coding mode or not;