Disclosure of Invention
The embodiment of the application provides an image coding control method, an image coding control device, image coding control equipment and a computer readable storage medium, which can solve the problem of long image coding time delay. The technical scheme is as follows:
According to a first aspect of an embodiment of the present application, there is provided an image encoding control method, including:
counting the network time delay of each frame of image in the preset number of frame of images;
Calculating the average value of the network delays of the preset number of frame images according to the network delay of each frame image;
judging whether the average value of the network time delay is larger than a preset threshold value or not;
When the average value of the network time delay is larger than the preset threshold value, determining a frame loss number M according to a preset rule, wherein M is an integer;
consecutive M-frame images that are not currently beginning to be encoded are discarded.
In one embodiment, the determining the frame loss number M according to the preset rule includes:
and determining the frame loss number M according to the difference value between the average value of the network time delay and the preset threshold value.
In one embodiment, the determining the frame loss number M according to the difference between the average value of the network delay and the preset threshold value includes:
determining a frame loss number M according to a first formula, wherein the first formula is as follows: m= [ (T Average of -A)/(t Acquisition of +t Encoding ) ];
wherein T Average of is an average value of network delay, a is a preset threshold, T Acquisition of is an average time length required for acquiring one frame of image, and T Encoding is an average time length required for encoding one frame of image.
In one embodiment, the determining the frame loss number M according to the difference between the average value of the network delay and the preset threshold value includes:
determining the frame loss number M according to a second formula, wherein the second formula is as follows:
M=[(T Average of -A)/(t Acquisition of +t Encoding )]+1;
wherein T Average of is an average value of network delay, a is a preset threshold, T Acquisition of is an average time length required for acquiring one frame of image, and T Encoding is an average time length required for encoding one frame of image.
In one embodiment, the method further comprises:
Detecting a setting operation for setting a preset number;
And determining the preset quantity according to the setting operation.
According to the image coding control method provided by the embodiment of the application, the network time delay of each frame of images in the preset number of frame images is counted in real time in the image transmission process, the average value of the network time delays of the preset number of frame images is calculated, when the average value is larger than a preset threshold value, the frame loss number M is determined according to a preset rule, and the continuous M frame images which are not coded at present are discarded. The method can determine the frame loss number before encoding and discard the corresponding number of images before encoding, thereby relieving the network pressure, shortening the image encoding time delay and reducing the encoding load of the CPU.
According to a second aspect of an embodiment of the present application, there is provided an image encoding control apparatus including:
The statistics module is used for counting the network time delay of each frame of images in the preset number of frames of images;
The calculation module is used for calculating the average value of the network time delays of the preset number of frame images according to the network time delay of each frame image;
the first determining module is used for judging whether the average value of the network time delay is larger than a preset threshold value, and determining the number of lost frames M according to a preset rule when the average value of the network time delay is larger than the preset threshold value, wherein M is an integer;
And the frame discarding module is used for discarding continuous M frame images which are not started to be encoded currently.
In one embodiment, the first determining module is specifically configured to determine the frame loss number M according to a difference between the average value of the network delay and the preset threshold.
In one embodiment, the first determining module is specifically configured to determine the frame loss number M according to a first formula, where the first formula is: m= [ (T Average of -A)/(t Acquisition of +t Encoding ) ];
wherein T Average of is an average value of network delay, a is a preset threshold, T Acquisition of is an average time length required for acquiring one frame of image, and T Encoding is an average time length required for encoding one frame of image.
In one embodiment, the first determining module is specifically configured to determine the frame loss number M according to a second formula, where the second formula is:
M=[(T Average of -A)/(t Acquisition of +t Encoding )]+1;
wherein T Average of is an average value of network delay, a is a preset threshold, T Acquisition of is an average time length required for acquiring one frame of image, and T Encoding is an average time length required for encoding one frame of image.
In one embodiment, the apparatus further comprises:
the detection module is used for detecting setting operation for setting the preset quantity;
and the second determining module is used for determining the preset quantity according to the setting operation.
According to the image coding control device provided by the embodiment of the application, the network time delay of each frame image in the preset number of frame images is counted by the counting module, the calculating module calculates the average value of the network time delays of the preset number of frame images, and when the average value is judged to be larger than the preset threshold value, the first determining module determines the frame loss number M according to the preset rule, and the frame loss module discards the continuous M frame images which are not coded currently. The method can determine the frame loss number before encoding and discard the corresponding number of images before encoding, thereby relieving the network pressure, shortening the image encoding time delay and reducing the encoding load of the CPU.
According to a third aspect of embodiments of the present application, there is provided an image encoding control apparatus including a processor and a memory having stored therein at least one computer instruction loaded and executed by the processor to implement the steps performed in the image encoding control method as described above.
According to a fourth aspect of embodiments of the present application, there is provided a computer-readable storage medium having stored therein at least one computer instruction loaded and executed by a processor to implement the steps performed in the image encoding control method as described above.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.
An embodiment of the present application provides an image encoding control method, as shown in fig. 1, which may include the following steps:
step 101: and counting the network time delay of the preset number of frame images.
In the image transmission process, the network time delay of each frame image in the preset number of frame images which are transmitted recently is counted in real time. Specifically, the network delay of a frame of image is equal to the difference between the time stamp of the image received by the receiving end and the time stamp of the image when the image is sent by the sending end.
Step 102: and calculating the average value of the network time delays of the preset number of frame images.
And counting to obtain the network delay of each frame of image in the preset number of frame images which are transmitted recently, and calculating the average value of the network delay of the preset number of frame images according to the counted network delay of each frame of image. If the preset number is N, continuously counting the network delay of each of the N frames of images, and then calculating the average value of the network delays of the N frames of images.
Step 103: and judging whether the average value is larger than a preset threshold value.
After calculating the average value of the network time delays of the preset number of frame images, judging whether the average value of the network time delays is larger than a preset threshold value, if so, executing step 104, otherwise, turning to step 101 to continue counting.
Step 104: and determining the frame loss number M according to a preset rule.
When the average value of network time delays of the preset number of frame images is larger than a preset threshold value, determining the number of lost frames M according to a preset rule, wherein M is an integer. For example, the frame loss number M may be determined according to a difference between an average value of network delays and a preset threshold. In one embodiment, the difference and the ratio of the average time required to capture a frame of image to the sum of the average time required to encode a frame of image may be used to obtain the frame loss number M.
Step 105: consecutive M-frame images that are not currently beginning to be encoded are discarded.
After determining the frame loss number M, the continuous M frame images which are not currently coded are directly discarded, and then the step 101 is transferred to continue the statistics of network delay. In this step, no discarding is performed for the image that has started transmission or encoding, and the subsequent processing is still normally performed, and only the consecutive M-frame images that have not started encoding are discarded.
According to the image coding control method provided by the embodiment of the application, the network time delay of each frame of images in the preset number of frame images is counted in real time in the image transmission process, the average value of the network time delays of the preset number of frame images is calculated, when the average value is larger than a preset threshold value, the frame loss number M is determined according to a preset rule, and the continuous M frame images which are not coded at present are discarded. The method can determine the frame loss number before encoding and discard the corresponding number of images before encoding, thereby relieving the network pressure, shortening the image encoding time delay and reducing the encoding load of the CPU.
Based on the image coding control method provided in the embodiment corresponding to fig. 1, another embodiment of the present application provides a specific image coding control method, where the method may be applied to a video image coding end device, for example, the video image coding end device may be a terminal device or a network server. Referring to fig. 2, the image encoding control method provided in this embodiment may include the following steps:
step 201: and counting the network time delay of the preset number of frame images.
In the image transmission process, the network delay average value of each frame image in the preset number of frame images which are transmitted recently is counted in real time. The preset number can be set according to actual needs, for example, 10 frames, 20 frames, 30 frames, and the like. The network delay of a frame of image refers to the difference between the time stamp of the image received by the receiving end and the time stamp of the image when the image is sent by the sending end.
In one embodiment, a preset number of setting functions may be provided, specifically, the method may be implemented as follows: detecting a setting operation for setting a preset number; a preset number is determined according to the setting operation. For example, a user may be provided with a setting interface for setting a preset number, and the user may input the preset number, such as 10 frames, 20 frames, 30 frames, etc., according to the interface prompt; or the preset number of the shortcut keys can be set.
Step 202: and calculating the average value of the network time delays of the preset number of frame images.
And counting to obtain the network delay of each frame of images in the preset number of frames of images which are transmitted recently, and calculating the average value of the network delay of the preset number of frames of images according to the network delay of each frame of images. If the preset number is N, continuously counting the network delay of each of the N frames of images, and then calculating the average value of the network delays of the N frames of images.
Specifically, the network delay average value T Average of of the continuous N-frame image can be obtained by calculation according to the following calculation formula:
Wherein t x is the network delay of the x-th frame image, x is an integer greater than 0 and less than or equal to N, Wherein Rt x represents a time stamp of transmission of the x-th frame image to the receiving end, and St x represents a time stamp of transmission of the x-th frame image at the transmitting end.
Step 203: and judging whether the average value is larger than a preset threshold value.
Judging whether the average value of the network time delays of the counted preset number of frame images is larger than a preset threshold value, if so, executing step 204, otherwise, turning to step 201 to continue counting.
In the step, comparing the calculated average value of the network time delays of the preset number of frame images with a preset threshold value, and judging whether the average value is larger than the preset threshold value. The preset threshold is a preset value, which can be set empirically, for example, can be set to 100ms, 90ms, 80ms, etc., or can be set to other values. The higher the preset threshold value is set, the higher the tolerance to the delay is indicated, and thus, the setting of the preset threshold value can be dependent on the use scenario.
In one embodiment, a preset threshold setting function may be provided, specifically, the method may be implemented as follows: detecting a setting operation for setting a preset threshold; a preset threshold is determined according to the setting operation. For example, a user may be provided with a setting interface for setting a preset threshold, and the user may input the preset threshold according to the interface prompt, such as setting to 100ms, 90ms, 80ms, etc.; or the preset threshold value can be set by a shortcut key of the preset threshold value.
Step 204: and determining the frame loss number M according to the difference value between the average value of the network time delay and the preset threshold value.
Assuming that the average value of the network delay obtained by statistics is T Average of , the number of frames lost M may be calculated by the following first formula or by the following second formula.
Wherein, the first formula is: m= [ (T Average of -A)/(t Acquisition of +t Encoding ) ];
The second formula is: m= [ (T Average of -A)/(t Acquisition of +t Encoding ) ]+1;
in the above formula, T Average of represents an average value of network delay, a represents a preset threshold, T Acquisition of is an average time length required for capturing one frame of image, T Encoding is an average time length required for encoding one frame of image, and both T Acquisition of and T Encoding can be set according to empirical values. Wherein the symbol [ ] represents a rounded symbol, i.e., [ (T Average of -A)/(t Acquisition of +t Encoding ) ] is equal to (T Average of -A)/(t Acquisition of +t Encoding ) the integer portion of the result.
Step 205: consecutive M-frame images that are not currently beginning to be encoded are discarded.
After determining the number M of lost frames, the continuous M frame images which are not started to be encoded at present are directly discarded, and then the step 101 is transferred to continue the statistics of network delay.
In this step, consecutive M-frame images that have not been started to be encoded at present are directly discarded, and at the same time, no discarding is performed for images that have been started to be transmitted or encoded, and the subsequent processing is normally performed, and what is discarded is consecutive M-frame images that have not been started to be encoded.
According to the image coding control method provided by the embodiment of the application, the network time delay of each frame of images in the preset number of frame images is counted in real time in the image transmission process, the average value of the network time delays of the preset number of frame images is calculated, when the average value is larger than a preset threshold value, the frame loss number M is determined according to the difference value between the average value of the network time delays and the preset threshold value, and then the continuous M frame images which are not coded at present are discarded. The method can determine the frame loss number before encoding and discard the corresponding number of images before encoding, thereby relieving the network pressure, shortening the image encoding time delay and reducing the encoding load of the CPU.
Based on the image coding control method described in the above-described embodiments corresponding to fig. 1 and fig. 2, the following is an embodiment of the apparatus of the present application, which may be used to execute the method embodiment of the present application.
An embodiment of the present application provides an image encoding control device, as shown in fig. 3, the image encoding control device 30 includes: a statistics module 301, a computation module 302, a first determination module 303, and a frame dropping module 304. Wherein,
The statistics module 301 is configured to count a network delay of each frame image in a preset number of frame images;
the calculating module 302 is configured to calculate an average value of network delays of a preset number of frame images according to the network delay of each frame image obtained by the statistics module 301;
The first determining module 303 is configured to determine whether the average value of the network delay calculated by the calculating module 302 is greater than a preset threshold, and determine a frame loss number M according to a preset rule when the average value of the network delay is greater than the preset threshold, where M is an integer;
the frame discarding module 304 is configured to discard consecutive M-frame images that are not currently started to be encoded.
In one embodiment, the first determining module 303 is specifically configured to determine the frame loss number M according to a difference between an average value of the network delay and the preset threshold.
In one embodiment, the first determining module 303 is specifically configured to determine the frame loss number M according to a first formula: m= [ (T Average of -A)/(t Acquisition of +t Encoding ) ];
wherein T Average of is an average value of network time delay, a is a preset threshold, T Acquisition of is an average time length required for acquiring one frame of image, and T Encoding is an average time length required for encoding one frame of image.
In one embodiment, the first determining module 303 is specifically configured to determine the frame loss number M according to a second formula, where the second formula is:
M=[(T Average of -A)/(t Acquisition of +t Encoding )]+1;
wherein T Average of is an average value of network time delay, a is a preset threshold, T Acquisition of is an average time length required for acquiring one frame of image, and T Encoding is an average time length required for encoding one frame of image.
In one embodiment, as shown in fig. 4, the image encoding control apparatus further includes a detection module 305 and a second determination module 306. Wherein,
The detection module 305 is configured to detect a setting operation for setting a preset number;
the second determining module 306 is configured to determine the preset number according to the setting operation detected by the detecting module 305.
According to the image coding control device provided by the embodiment of the application, the network time delay of each frame of images in the preset number of frame images is counted in real time in the image transmission process through the counting module, the calculating module calculates the average value of the network time delays of the preset number of frame images, when the first determining module judges that the average value is larger than the preset threshold value, the frame loss number M is determined according to the preset rule, and the frame loss module discards continuous M frame images which are not coded currently. The method can determine the frame loss number before encoding and discard the corresponding number of images before encoding, thereby relieving the network pressure, shortening the image encoding time delay and reducing the encoding load of the CPU.
Referring to fig. 5, an embodiment of the present application further provides an image encoding control apparatus 50, where the image encoding control apparatus 50 includes a memory 501 and a processor 502, at least one computer instruction is stored in the memory 501, and the processor 502 is configured to load and execute the at least one computer instruction to implement the image encoding control method described in the corresponding embodiment of fig. 1 or fig. 2.
Based on the image encoding control method described in the above-described embodiments corresponding to fig. 1 and 2, the embodiments of the present application also provide a computer readable storage medium, for example, a non-transitory computer readable storage medium may be a Read Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the image coding control method described in the above embodiments corresponding to fig. 1 and fig. 2, which are not described herein.
Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.