[go: up one dir, main page]

CN114245196B - Screen recording and stream pushing method and device, electronic equipment and storage medium - Google Patents

Screen recording and stream pushing method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN114245196B
CN114245196B CN202111512331.XA CN202111512331A CN114245196B CN 114245196 B CN114245196 B CN 114245196B CN 202111512331 A CN202111512331 A CN 202111512331A CN 114245196 B CN114245196 B CN 114245196B
Authority
CN
China
Prior art keywords
frame rate
frame
video
target
preview
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111512331.XA
Other languages
Chinese (zh)
Other versions
CN114245196A (en
Inventor
张艳坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhuomi Private Ltd
Original Assignee
Zhuomi Private Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhuomi Private Ltd filed Critical Zhuomi Private Ltd
Priority to CN202111512331.XA priority Critical patent/CN114245196B/en
Publication of CN114245196A publication Critical patent/CN114245196A/en
Application granted granted Critical
Publication of CN114245196B publication Critical patent/CN114245196B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4334—Recording operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The embodiment of the invention provides a screen recording and stream pushing method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a video of a screen display picture and a frame rate, and taking the frame rate as a preview frame rate; based on the size relation between the preview frame rate and the preset frame rate, carrying out frame rate adjustment on the video recorded on the screen to obtain an adjusted video; and encoding the adjusted video to obtain an encoded video, and pushing the encoded video. By adopting the scheme, the preview frame rate of the video recording can be obtained, and then the video recording is subjected to frame rate adjustment based on the preview frame rate and the preset frame rate, so that the frame rate of the video recording is matched with the configured code rate, the occurrence of blurring or cutoff of pictures in the live broadcast process is reduced, the clear and smooth live broadcast pictures are ensured, and the live broadcast effect of the video recording is improved.

Description

Screen recording and stream pushing method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of video processing technologies, and in particular, to a method and apparatus for recording and pushing streams, an electronic device, and a storage medium.
Background
With the rapid development of computer and internet technologies, video live broadcast is becoming more and more common, and live video recording is a very common and convenient live broadcast mode. In the live broadcasting process, the live broadcasting terminal collects screen display data, and then codes the collected screen data and pushes the screen display data. The push stream is to transmit the encoded video data to a server, and the terminal equipment for watching live broadcast can watch live broadcast of the anchor by pulling stream from the server.
The frame rate of the screen display picture of the live broadcast terminal is controlled by an operating system, when the anchor operates the screen display picture, the display picture changes greatly, the frame rate can reach the maximum, and when the screen display picture is unchanged, the frame rate is slowly reduced, even no data is generated to reduce the power consumption.
In the existing live video recording mode, when the frame rate of a screen display picture of a live video terminal is too high, in order to match with a set code rate, the definition of the picture needs to be reduced, so that the picture of a push stream is blurred or even blue; when the frame rate of the screen display picture of the live broadcast terminal is too low, no video stream can be pushed out, and the current is cut off when a user watches live broadcast, so that the live broadcast effect of recording the screen is not ideal.
Disclosure of Invention
The embodiment of the invention aims to provide a screen recording and streaming pushing method, a device, electronic equipment and a storage medium, so as to improve the screen recording and live broadcasting effect. The specific technical scheme is as follows:
In a first aspect, an embodiment of the present invention provides a method for screen recording and stream pushing, where the method includes:
Acquiring a video of a screen display picture and a frame rate, and taking the frame rate as a preview frame rate;
based on the size relation between the preview frame rate and the preset frame rate, carrying out frame rate adjustment on the video recording video to obtain an adjusted video;
And encoding the adjusted video to obtain an encoded video, and pushing the encoded video.
Optionally, the preset frame rate includes a first target frame rate and a second target frame rate, and the first target frame rate is higher than the second target frame rate;
the step of adjusting the frame rate of the video recording based on the size relation between the preview frame rate and the preset frame rate to obtain an adjusted video comprises the following steps:
determining a size relationship between the preview frame rate and the first and second target frame rates, respectively;
if the preview frame rate is larger than the first target frame rate, carrying out frame loss processing on the video recording according to a preset frame loss strategy to obtain an adjusted video;
And if the preview frame rate is smaller than the second target frame rate, carrying out frame supplementing processing on the video recording according to a preset frame supplementing strategy to obtain an adjusted video.
Optionally, the step of performing frame loss processing on the video recording video according to a preset frame loss policy to obtain an adjusted video includes:
determining a frame loss ratio based on the preview frame rate, the first target frame rate and current adjustment parameters for a current image frame included in the video;
If the frame loss ratio is not smaller than a preset ratio and the number of the image frames which are discarded currently is smaller than a first target value, discarding the current image frames and adjusting the current adjusting parameters according to a first preset mode;
taking the next frame of the current image frame as the current image frame, and returning to the step of determining a frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter until the frame loss ratio is smaller than the preset ratio or the number of the currently discarded image frames is not smaller than the first target value;
If the frame loss ratio is smaller than the preset ratio and the number of the current reserved image frames is not smaller than a second target value, discarding the current image frames, setting the number of the current reserved image frames to 0, and adjusting the current adjustment parameters according to a second preset mode;
And taking the next frame of the current image frame as the current image frame, and returning to the step of determining the frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter until the current image frame is the last image frame of the video.
Optionally, the step of adjusting the current adjustment parameter according to a first preset manner includes:
Calculating a difference value overschot based on the preview frame rate, the first target frame rate and the current adjustment parameter;
Calculating according to a formula overshootModifier = - (PREVIEWFPS% overshoot)/3 to obtain the current adjustment parameter overshootModifier, wherein PREVIEWFPS is the preview frame rate;
the step of adjusting the current adjustment parameters according to a second preset mode comprises the following steps:
And calculating to obtain the current adjustment parameter according to a formula overshootModifier =overschoot% outputFps, wherein outputFps is the first target frame rate.
Optionally, the step of calculating a difference value overmethod based on the preview frame rate, the first target frame rate and the current adjustment parameter includes:
the difference overschot was calculated according to the formula overschot= overshootModifier + (PREVIEWFPS-outputFps).
Optionally, the first target value is a quotient of the difference value and the first target frame rate; the second target value is a quotient of the preview frame rate and the difference.
Optionally, the step of performing frame-filling processing on the video with the screen recording to obtain an adjusted video according to a preset frame-filling policy includes:
and taking the acquired last frame of screen display picture as a supplementary video frame to obtain an adjusted video.
In a second aspect, an embodiment of the present invention provides a screen recording and pushing device, where the device includes:
the frame rate acquisition module is used for acquiring the video of the screen display picture and the frame rate, and taking the frame rate as a preview frame rate;
The frame rate control module is used for adjusting the frame rate of the video recording based on the size relation between the preview frame rate and the preset frame rate to obtain an adjusted video;
and the encoding and stream pushing module is used for encoding the adjusted video to obtain an encoded video and pushing the encoded video.
Optionally, the preset frame rate includes a first target frame rate and a second target frame rate, and the first target frame rate is higher than the second target frame rate;
The frame rate control module includes:
a frame rate relationship determination submodule for determining a magnitude relationship between the preview frame rate and the first target frame rate and the second target frame rate, respectively;
the frame loss sub-module is used for carrying out frame loss processing on the video recording according to a preset frame loss strategy if the preview frame rate is larger than the first target frame rate, so as to obtain an adjusted video;
and the frame supplementing sub-module is used for carrying out frame supplementing processing on the video recording according to a preset frame supplementing strategy if the preview frame rate is smaller than the second target frame rate, so as to obtain an adjusted video.
Optionally, the frame loss submodule includes:
the frame loss ratio determining unit is used for determining a frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter for the current image frame included in the video recording;
the first frame loss unit is used for discarding the current image frame and adjusting the current adjusting parameter according to a first preset mode if the frame loss ratio is not smaller than a preset ratio and the number of the current discarded image frames is smaller than a first target value;
A first circulation unit configured to take a next frame of the current image frame as a current image frame, and trigger the frame loss ratio determining unit until if the frame loss ratio is smaller than the preset ratio, or if the number of the currently discarded image frames is not smaller than the first target value;
A second frame loss unit, configured to discard the current image frame if the frame loss ratio is smaller than the preset ratio and the number of the current reserved image frames is not smaller than a second target value, set the number of the current reserved image frames to 0, and adjust the current adjustment parameter according to a second preset manner;
And the second circulation unit is used for taking the next frame of the current image frame as the current image frame and triggering the frame loss ratio determining unit until the current image frame is the last image frame of the screen recording video.
Optionally, the first frame loss unit includes:
the difference value calculating subunit is used for calculating a difference value overhop based on the preview frame rate, the first target frame rate and the current adjusting parameter;
A first adjustment parameter calculating subunit, configured to calculate the current adjustment parameter overshootModifier according to a formula overshootModifier = - (PREVIEWFPS% overschot)/3, where PREVIEWFPS is the preview frame rate;
The second frame loss unit includes:
And a second adjustment parameter calculating subunit, configured to calculate the current adjustment parameter according to a formula overshootModifier =overschoot% outputFps, where outputFps is the first target frame rate.
Optionally, the difference calculating subunit is configured to calculate the difference overschot according to the formula overschot= overshootModifier + (PREVIEWFPS-outputFps).
Optionally, the first target value is a quotient of the difference value and the first target frame rate; the second target value is a quotient of the preview frame rate and the difference.
Optionally, the frame supplementing submodule includes:
and the frame supplementing unit is used for taking the acquired last frame of screen display picture as a supplemental video frame to obtain an adjusted video.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
A memory for storing a computer program;
A processor for implementing the method steps of any of the above first aspects when executing a program stored on a memory.
In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium having a computer program stored therein, which when executed by a processor, implements the method steps of any of the first aspects described above.
The embodiment of the invention has the beneficial effects that:
In the scheme provided by the embodiment of the invention, the electronic equipment can acquire the video of the screen display picture and the frame rate, and takes the frame rate as the preview frame rate; based on the size relation between the preview frame rate and the preset frame rate, carrying out frame rate adjustment on the video recorded on the screen to obtain an adjusted video; and further, the adjusted video is encoded to obtain an encoded video, and the encoded video is subjected to stream pushing. By adopting the scheme, the preview frame rate of the video recording can be obtained, and then the video recording is subjected to frame rate adjustment based on the preview frame rate and the preset frame rate, so that the frame rate of the video recording is matched with the configured code rate, the occurrence of blurring or cutoff of pictures in the live broadcast process is reduced, the clear and smooth live broadcast pictures are ensured, and the live broadcast effect of the video recording is improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other embodiments may be obtained according to these drawings to those skilled in the art.
FIG. 1 is a flowchart of a method for recording and pushing stream according to an embodiment of the present invention;
FIG. 2 is a specific flowchart based on step S102 in the embodiment shown in FIG. 1;
FIG. 3 is a flowchart based on step S202 in the embodiment shown in FIG. 2;
FIG. 4 is a schematic diagram of a frame loss result of the frame loss operation according to the embodiment shown in FIG. 3;
FIG. 5 is a schematic diagram of the overall flow of the screen recording and pushing based on the embodiment shown in FIG. 1;
Fig. 6 is a schematic structural diagram of a screen recording and pushing device according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a specific structure of a frame loss submodule according to the embodiment shown in fig. 6;
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by the person skilled in the art based on the present invention are included in the scope of protection of the present invention.
In order to reduce the occurrence of blurring or blocking of a picture in a live broadcast process, so as to ensure clear and smooth live broadcast pictures, embodiments of the present invention provide a live broadcast streaming method, apparatus, electronic device, computer readable storage medium and computer program product, and a live broadcast streaming method provided by the embodiments of the present invention is first described below.
The screen recording and pushing method provided by the embodiment of the invention can be applied to any electronic equipment needing screen recording and pushing, can acquire the screen recording video and the frame rate of the screen display picture of the live terminal, and can encode the acquired screen recording video and then push, for example, the screen recording and pushing method can be used as a server corresponding to the live terminal. For clarity of description, hereinafter, referred to as an electronic device.
As shown in fig. 1, a screen recording and stream pushing method includes:
S101, acquiring a video recorded on a screen display picture and a frame rate, and taking the frame rate as a preview frame rate.
S102, based on the size relation between the preview frame rate and the preset frame rate, adjusting the frame rate of the video recording video to obtain an adjusted video.
And S103, encoding the adjusted video to obtain an encoded video, and pushing the encoded video.
Therefore, in the scheme provided by the embodiment of the invention, the electronic equipment can acquire the video recorded on the screen of the screen display picture and the frame rate, and takes the frame rate as the preview frame rate; based on the size relation between the preview frame rate and the preset frame rate, carrying out frame rate adjustment on the video recorded on the screen to obtain an adjusted video; and further, the adjusted video is encoded to obtain an encoded video, and the encoded video is subjected to stream pushing. By adopting the scheme, the preview frame rate of the video recording can be obtained, and then the video recording is subjected to frame rate adjustment based on the preview frame rate and the preset frame rate, so that the frame rate of the video recording is matched with the configured code rate, the occurrence of blurring or cutoff of pictures in the live broadcast process is reduced, the clear and smooth live broadcast pictures are ensured, and the live broadcast effect of the video recording is improved.
In order to reduce blurring or cutting off of pictures in the live video recording process, the frame rate of the video recording needs to be matched with the set code rate. The frame rate is the frequency of continuously appearing on the display screen of bitmap images in units of frames, and in the embodiment of the invention, the frame rate is the frequency of continuously appearing on the display screen of image frames included in the video recording, and the higher the frame rate is, the smoother the picture of the video recording. The bit rate is the bit rate, which represents the number of bits transmitted in unit time, and the larger the bit rate, the faster the encoder can transmit bits. For example, a code rate of 800kbps indicates that the encoder can transmit 800 kilobits per second.
In the live broadcasting process of recording, the code rate is usually fixed, and if the frame rate of the video of recording is too high, the definition of the picture needs to be reduced to realize normal coding; if the frame rate of the video is too low, no video stream can be pushed out, so the electronic equipment can adjust the frame rate of the video to ensure the definition and smoothness of the live broadcast picture.
In the above step S101, the electronic device may acquire the video recorded on the screen and the frame rate of the screen display, and take the frame rate as the preview frame rate.
The electronic equipment can record the screen display picture of the live terminal, so as to obtain the recorded video. The frame rate of the screen display is controlled by the operating system, which can change the frame rate according to the operation of the anchor on the screen display, and when the frame rate of the screen display is changed, the frame rate of the video recording is changed. For accurately obtaining the frame rate of the video recording, a time period may be preset, for example, the time period may be 2 seconds, 3 seconds, 5 seconds, and the like, which is not limited herein. The electronic device may obtain the number of image frames included in the video recorded in the time period, and calculate a quotient of the number of image frames and a preset time period as a preview frame rate of the video recorded.
For example, the preset time period is 3 seconds, the electronic device obtains the number of image frames included in the video recorded once in each time period, the number of image frames obtained by the electronic device in the current time period is 90, and then the preview frame rate of the video recorded in the current time period isI.e. 30 frames per second.
After obtaining the video recording and the preview frame rate of the video recording, the electronic device may adjust the frame rate of the video recording based on the size relationship between the preview frame rate and the preset frame rate, so as to obtain an adjusted video, that is, execute step S102.
The preview frame rate of the obtained video in each time period may be different, the highest value can reach the maximum value of the screen display frame rate of the live broadcast terminal, and the smallest value can reach 0; therefore, for each time period, the electronic device can adjust the frame rate of the video to control the frame rate of the video, so that the video can be encoded and continuously pushed without changing the definition. The electronic equipment can control the frame rate of the video recording to be within a certain frame rate range, and in the frame rate range, the video recording cannot cause the blurring of live broadcasting pictures due to the too high frame rate and the live broadcasting interruption due to the too low frame rate.
In one embodiment, a frame rate range may be preset, and when the frame rate of the video recording is within the frame rate range, the video recording may be encoded and continuously streamed without changing sharpness. The electronic device can judge whether the preview frame rate of the video recording is in a preset frame rate range, if the preview frame rate is larger than the maximum value of the preset frame rate range, the preview frame rate is too high, the live broadcast picture can be blurred, and the electronic device can perform frame loss operation on the video recording so that the frame rate of the video after adjustment is in the preset frame rate range. If the preview frame rate is smaller than the minimum value of the preset frame rate range, the preview frame rate is too low, the live broadcast may be cut off, and the electronic device may perform frame supplementing operation on the video recording, so that the frame rate of the adjusted video is within the preset frame rate range. Through carrying out frame rate adjustment on the video recorded on the screen, the electronic equipment obtains the video after adjustment, and the frame rate of the video after adjustment is matched with the set code rate, so that coding and continuous plug flow can be carried out without changing definition.
Next, in the step S103, the electronic device may encode the adjusted video, obtain an encoded video, and push the encoded video.
The frame rate of the adjusted video is matched with the set code rate, and the electronic equipment can code the adjusted video. Video encoding is to convert the adjusted video into a file in another video format by a compression technique, for example, the encoding algorithm may be H261, H263, H264, etc., which is not limited herein. The data volume of the coded video is reduced, the occupation of a CPU and a memory is reduced, and the transmission is more convenient.
The electronic device may push the encoded video, which is the process of transmitting the encoded video to the server. The Protocol of the video push may be RTMP (REAL TIME MESSAGING Protocol, real-time messaging Protocol), etc., which is not limited herein.
By adopting the scheme provided by the embodiment of the invention, the electronic equipment can adjust the frame rate of the video on the basis of the preview frame rate and the preset frame rate of the video on the screen, encode the adjusted video and push the encoded video. The adjusted video frame rate is matched with the set code rate, so that the occurrence of blurring or cutoff of pictures in the live broadcast process is reduced, and the success rate of users entering a live broadcast room is effectively improved because the cutoff is not generated; the blurring of the picture is reduced, the clear and smooth live picture is ensured, the user can interact with the anchor better, the reserved data is higher, and the live video recording effect is improved.
As an implementation manner of the embodiment of the present invention, as shown in fig. 2, the preset frame rate may include a first target frame rate and a second target frame rate, where the first target frame rate is higher than the second target frame rate.
In one embodiment, the first target frame rate is a maximum frame rate at which the adjusted video can be encoded and continuously pushed without changing sharpness, and the second target frame rate is a minimum frame rate at which the adjusted video can be encoded and continuously pushed without changing sharpness. When the frame rate of the video after adjustment is not more than the first target frame rate and not less than the second target frame rate, coding and continuous push can be performed without changing definition.
Correspondingly, the step of adjusting the frame rate of the video recording to obtain an adjusted video based on the size relation between the preview frame rate and the preset frame rate may include:
S201, determining the size relation between the preview frame rate and the first target frame rate and the second target frame rate respectively; if the preview frame rate is greater than the first target frame rate, executing step S202; if the preview frame rate is less than the second target frame rate, step S203 is performed.
After the electronic device acquires the video and the preview frame rate, the size relationship between the preview frame rate and the first target frame rate and the second target frame rate can be determined respectively. If the preview frame rate of the video recording is greater than the first target frame rate, it indicates that the preview frame rate of the video recording is too high, and if the frame rate of the video recording is not adjusted, the live broadcast picture may be blurred or even blue, so that the video recording needs to be subjected to frame loss processing, and step S202 may be executed.
If the preview frame rate of the video recording is smaller than the second target frame rate, it indicates that the preview frame rate of the video recording is too low, and if the video recording is not adjusted, the live video recording may be stopped, so that the video recording needs to be subjected to frame-supplementing processing, and step S203 may be executed.
If the preview frame rate of the video recording is not greater than the first target frame rate and not less than the second target frame rate, the preview frame rate of the video recording is not too high or too low, coding and continuous push can be performed without changing definition, and the preview frame rate of the video recording does not affect the live broadcast picture, so that frame rate adjustment is not needed.
S202, carrying out frame loss processing on the recorded video according to a preset frame loss strategy to obtain an adjusted video.
When the preview frame rate of the video recording is greater than the first target frame rate, the electronic device can perform frame loss processing on the video recording according to a preset frame loss strategy, and part of image frames included in the video recording can be discarded in the frame loss processing, so that the frame rate of the adjusted video is not greater than the first target frame rate and not less than the second target frame rate, the adjusted video can be encoded and continuously pushed without changing definition, and the live broadcast picture can not generate blurring or blue screen.
And S203, carrying out frame supplementing processing on the video recorded by the screen according to a preset frame supplementing strategy to obtain an adjusted video.
When the preview frame rate of the video recording is smaller than the second target frame rate, the electronic equipment can perform frame supplementing processing on the video recording according to a preset frame supplementing strategy. In one embodiment, the frame-supplementing process may be to obtain the last frame of the screen display picture as a supplemental video frame, and add the supplemental video frame to the video recording, so that the frame rate of the adjusted video is not greater than the first target frame rate and not less than the second target frame rate, the adjusted video may be encoded and continuously pushed without changing the sharpness, and the video recording and direct broadcasting will not generate a current interruption phenomenon.
In this embodiment, based on the preset first target frame rate and the second target frame rate, the electronic device may determine whether frame loss processing or frame compensation processing is required for the video recording, and if adjustment is required, the electronic device may perform processing according to a corresponding policy, where the frame rate of the adjusted video is not greater than the first target frame rate and not less than the second target frame rate. Therefore, the frame rate of the adjusted video reaches the preset frame rate range, the video can be encoded and continuously pushed without changing definition, and in the live video recording and broadcasting process, the live video does not generate blurring or blue video, and the live video does not generate cut-off phenomenon.
As shown in fig. 3, the step of performing frame loss processing on the video recorded according to the preset frame loss policy to obtain an adjusted video may include:
S301, determining a frame loss ratio based on the preview frame rate, the first target frame rate and current adjustment parameters for a current image frame included in the video recording; if the frame loss ratio is not less than the preset ratio and the number of the image frames which are discarded currently is less than the first target value, executing step S302; if the frame loss ratio is smaller than the preset ratio and the number of the image frames which are reserved currently is not smaller than a second target value, step S304 is executed.
When the preview frame rate of the video recording is greater than the first target frame rate, the electronic equipment needs to perform frame loss processing on the video recording. If the preview frame rate is far greater than the first target frame rate, it means that the electronic device needs to discard a large number of image frames when performing frame dropping processing on the video recorded by the screen, so that the frame rate of the video after adjustment is not greater than the first target frame rate and not less than the second target frame rate, and therefore a frame dropping strategy capable of rapidly discarding a large number of image frames is needed. If the difference between the preview frame rate and the first target frame rate is smaller, it means that the number of image frames to be discarded is smaller when the electronic device performs frame loss processing on the video recording, so that the frame rate of the adjusted video is not greater than the first target frame rate and not less than the second target frame rate, and therefore a frame loss strategy capable of maintaining the smoothness of the picture as much as possible is needed.
The electronic device may determine a frame loss ratio based on the preview frame rate, the first target frame rate, and the current adjustment parameter. The frame loss ratio indicates the ratio of the image frames to be discarded in the video recording, and the higher the frame loss ratio is, the larger the number of the image frames to be discarded is. The electronic device can determine whether discarding is required for all image frames included in the recorded video one by one.
For the current image frame included in the video recording, if the frame loss ratio is not less than the preset ratio, it indicates that the number of image frames to be discarded is greater, and if the number of image frames to be discarded is less than the first target value, it indicates that the number of image frames to be discarded is still less, so the electronic device may execute step S302 to continue discarding the image frames. If the frame loss ratio is smaller than the preset ratio, which indicates that the number of image frames currently required to be discarded is smaller, and if the number of image frames currently reserved is not smaller than the second target value, which indicates that the number of image frames currently reserved is already larger, the electronic device may execute step S304 to satisfy the requirement of discarding a small number of image frames.
The initial value of the current adjusting parameter can be preset, the current adjusting parameter is calculated after the last image frame discarding process is completed, and the whole frame discarding process can be continuously changed, so that the frame discarding rate is adjusted, and the frame discarding strategy is changed to ensure that the number of the discarded image frames meets the actual requirement.
In one embodiment, the electronic device may calculate a difference between the preview frame rate and the first target frame rate, calculate a sum of the difference and the current adjustment parameter as a frame rate difference, and calculate a quotient of the frame rate difference and the preview frame rate as a frame loss ratio.
The preset ratio may be set according to the requirements of adjusting the frame rate of the video recording, maintaining the smoothness of the picture, etc., for example, the preset ratio may be 0.5. When the frame loss ratio is not less than 0.5, it indicates that the number of image frames to be discarded is greater than the number of image frames to be reserved in the video recording, that is, it may indicate that a plurality of image frames need to be discarded continuously, and then the electronic device may perform the frame loss operation continuously when the number of image frames that have been discarded currently is less than the first target value. When the frame loss ratio is smaller than 0.5, the number of the image frames to be discarded is smaller than the number of the image frames to be reserved in the video recording, and in order to ensure that the display picture of the video recording is smooth and stable, continuous frame loss is not performed, that is, at most one image frame is discarded continuously, so that the electronic device can discard one image frame when the number of the reserved image frames is not smaller than the second target value.
A schematic diagram of the frame loss result may be shown in fig. 4, where dropRatio represents a frame loss ratio, "F" represents an image frame retained in the video recording, and "×" represents an image frame discarded in the video recording. It can be seen that when the frame loss ratio is not less than 0.5, a plurality of image frames may be continuously discarded, after discarding the plurality of image frames, the frame rate of the adjusted video may be not greater than the first target frame rate and not less than the second target frame rate, the electronic device may stop the frame loss operation, and a continuous reserved image frame "F" appears in the video recording. When the frame loss ratio is smaller than 0.5, the frame rate of the adjusted video is not larger than the first target frame rate and not smaller than the second target frame rate without continuous frame loss.
S302, discarding the current image frame, and adjusting the current adjustment parameters according to a first preset mode.
When the frame loss ratio is not less than the preset ratio and the number of currently discarded image frames is less than the first target value, the electronic device may discard the current image frame. The number of discarded image frames is increased by 1, and the number of image frames to be discarded is reduced, so that the electronic device can adjust the current adjustment parameters according to the first preset mode to change the frame loss ratio. Next, the electronic device may continue to perform step S303.
S303, taking the next frame of the current image frame as the current image frame, and returning to the step S301 until the frame loss ratio is smaller than the preset ratio or the number of the image frames which are discarded currently is not smaller than the first target value.
After the electronic device completes the discarding operation of the image frame each time, the next frame of the current image frame may be used as the current image frame, and step S301 is returned, so as to recalculate the frame discarding ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter, and determine whether the current image frame needs to be discarded. Until the frame loss ratio is smaller than the preset ratio, or the number of image frames that have been currently discarded is not smaller than the first target value.
If the frame loss rate is smaller than the preset rate, the recorded video only needs to discard a small number of image frames, so that the electronic equipment can perform frame loss operation according to another frame loss strategy; if the number of the image frames which are discarded currently is not less than the first target value, the number of the image frames which are discarded currently is larger, so that the requirement of frame rate adjustment can be met, the frame rate of the video after adjustment is not more than the first target frame rate and not less than the second target frame rate, and the electronic equipment can stop the frame-discarding operation.
S304, discarding the current image frame, setting the number of the current reserved image frames to 0, and adjusting the current adjustment parameters according to a second preset mode;
The frame loss ratio is smaller than the preset ratio, which indicates that the video recording needs to discard a small number of image frames, and if the number of the image frames reserved currently is smaller than the second target value, which indicates that the number of the image frames reserved at the moment is smaller, the electronic device can reserve the current image frames.
If the number of the current reserved image frames is not smaller than the second target value, the number of the reserved image frames is larger, and the electronic equipment can discard the current image frames to meet the frame loss requirement. Since continuous frame dropping operation is not performed as much as possible in order to ensure smooth pictures as much as possible, the electronic device may set the number of image frames that have been currently reserved to 0, so as to ensure that continuous frame dropping is not performed.
The number of discarded image frames is increased by 1, and the number of image frames to be discarded is reduced, so that the electronic device can adjust the current adjustment parameters according to the second preset mode to change the frame loss ratio. Next, the electronic device may continue to perform step S305.
S305, taking the next frame of the current image frame as the current image frame, and returning to the step S301 until the current image frame is the last image frame of the video.
After the electronic device finishes the operation of reserving or discarding the current image frame each time, the electronic device may use the next frame of the current image frame as the current image frame and return to step S301, so as to recalculate the frame discarding ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter, and determine whether the current image frame needs to be discarded. And (5) ending the frame loss processing until the current image frame is the last image frame of the video recording.
In this embodiment, the electronic device may determine a frame loss ratio for a current image frame included in the video recording, and based on a size relationship between the frame loss ratio and a preset ratio, the electronic device may perform frame loss processing on the video recording according to two frame loss policies. When the frame loss ratio is large, the electronic equipment can continuously discard the image frames, and when the frame loss ratio is small, the electronic equipment can discard a small number of image frames, and the obtained adjusted video can be encoded and continuously pushed without changing definition. Therefore, the efficiency of the electronic equipment for completing the frame loss operation can be improved, and the picture smoothness of the recorded video can not be influenced. In the live broadcasting process of recording the screen, the frame rate of the video of reducing recording the screen can reduce CPU and occupation of the memory, have reduced the appearance probability of the fuzzy or blue screen of the picture, have improved users' watching experience, users can interact with the anchor better, have improved the data retention effectively.
As an implementation manner of the embodiment of the present invention, the step of adjusting the current adjustment parameter according to the first preset manner may include:
Calculating a difference value overschot based on the preview frame rate, the first target frame rate and the current adjustment parameter; according to the formula overshootModifier = - (PREVIEWFPS% overshoot)/3, the current adjustment parameter overshootModifier is calculated.
In order to adjust the current adjustment parameter and further change the frame loss ratio, the electronic device may calculate, based on the preview frame rate, the first target frame rate and the current adjustment parameter, a difference value overschoot, where the difference value overschoot represents the number of image frames to be discarded in the video recording.
After the electronic device calculates the difference value overschot, the current adjustment parameter overshootModifier can be calculated according to the formula overshootModifier = - (PREVIEWFPS% overschot)/3, and the current adjustment parameter can reduce the frame loss ratio, so that the frame loss strategy and the number of the discarded image frames are changed. Wherein PREVIEWFPS is the preview frame rate.
Correspondingly, the step of adjusting the current adjustment parameter according to the second preset manner may include:
the current adjustment parameter is calculated according to the formula overshootModifier = overshoot% outputFps.
After the electronic device calculates the difference value overschot, the current adjustment parameter may be calculated according to the formula overshootModifier =overschot% outputFps, where outputFps is the first target frame rate. The current adjustment parameter may adjust the frame loss ratio to change the number of discarded image frames until the current image frame is the last image frame of the video recording.
In this embodiment, in different frame loss policies, the electronic device may adjust the current adjustment parameter according to the first preset manner or the second preset manner, and adjust the frame loss ratio, so that the change of the frame loss policy and the number of discarded image frames may be accurately controlled.
As an implementation manner of the embodiment of the present invention, the step of calculating a difference value overschot based on the preview frame rate, the first target frame rate, and a current adjustment parameter may include:
the difference overschot was calculated according to the formula overschot= overshootModifier + (PREVIEWFPS-outputFps).
After the electronic device performs frame loss processing on the current image frame, the current adjusting parameter can be adjusted according to a first preset mode or a second preset mode, and the next frame of the current image frame is taken as the current image frame, and at the moment, the electronic device needs to recalculate the frame loss ratio to determine a frame loss strategy. The electronic device may calculate the difference overschot according to the formula overschot= overshootModifier + (PREVIEWFPS-outputFps).
Different current adjustment parameters can be obtained by different frame loss strategies, and the difference value can also be changed differently. The current adjustment parameters obtained by the frame dropping strategy for dropping a large number of image frames quickly can reduce the difference value and the frame dropping ratio, thereby changing the frame dropping strategy and the number of dropped image frames. The current adjustment parameters obtained by the frame loss strategy capable of maintaining the smoothness of the picture as much as possible can increase the difference value and adjust the frame loss ratio, thereby changing the number of discarded image frames.
In this embodiment, the electronic device may accurately calculate the difference value overschot, so as to determine the number of image frames to be discarded in the video recording, and may accurately calculate the frame loss ratio based on the difference value overschot, and after the operation of discarding the image frames is completed, the electronic device may also accurately adjust the current adjustment parameter, thereby improving the accuracy of the frame loss processing of the video recording.
As an implementation manner of the embodiment of the present invention, the first target value may be a quotient of the difference value and the first target frame rate; the second target value may be a quotient of the preview frame rate and the difference.
In the frame dropping strategy of dropping a large number of image frames rapidly, the difference value is reduced after each dropping of the image frames, the first target frame rate is determined, the first target value is continuously reduced, and the frame dropping rate is also continuously reduced. Then as the number of currently discarded image frames increases, there may occur a case where the frame loss ratio is smaller than the preset ratio or the number of currently discarded image frames is not smaller than the first target value, thereby changing the frame loss policy and the number of discarded image frames.
In the frame loss strategy capable of maintaining the smoothness of the picture as much as possible, the difference value is increased after each image frame is discarded, the preview frame rate is determined, the second target value is continuously reduced, and the electronic device can adjust the frame loss rate by comparing the number of the currently reserved image frames with the second target value, so as to control the number of the reserved image frames.
In this embodiment, the electronic device may accurately calculate the first target value, so as to control the number of discarded image frames in the frame loss processing procedure; the second target value is accurately calculated, the electronic equipment can control the number of the image frames reserved in the frame loss processing process, and accuracy of the video recording video frame loss processing is improved.
As an implementation manner of the embodiment of the present invention, the step of performing frame interpolation processing on the video recording to obtain an adjusted video according to a preset frame interpolation policy may include:
and taking the acquired last frame of screen display picture as a supplementary video frame to obtain an adjusted video.
If the preview frame rate of the video is smaller than the second target frame rate, the preview frame rate of the video is too low, and if the frame rate of the video is not adjusted, no video stream is pushed out in the stream pushing process, so that the problem of current interruption can occur. The electronic device may perform a frame-up process on the video-on-screen when the preview frame rate is less than the second target frame rate.
The electronic device may acquire a last frame of the on-screen display as the supplemental video frame, and in one embodiment, a last image frame included in the recorded video may be used as the last frame of the on-screen display in a time period in which the last preview frame rate is not less than the second target frame rate. The electronic device can acquire the last frame of screen display picture as a supplementary video frame, and further add the supplementary video frame into the video recording video, so that the frame rate of the video recording video is not greater than the first target frame rate and not less than the second target frame rate, and the adjusted video is obtained.
For example, the second target frame rate is 1, the first target frame rate is 30, and if the preview frame rate of the video recording is less than 1, it indicates that the frame rate of the video recording is 0, and during the push process, no video stream is pushed out, so that the current interruption is caused. At this time, the electronic device may acquire the last frame of the screen display, and form the image frames into the video recording according to the number of 15 frames per second, where the frame rate of the video recording is smaller than 30 and larger than 1, so as to perform encoding and continuous push. When the preview frame rate of the video recording is not less than 1, the electronic equipment can stop the frame supplementing process of the video recording, and adjust the frame rate of the video recording based on the size relation between the preview frame rate and the preset frame rate.
In this embodiment, the electronic device performs frame-supplementing processing on the video recorded with the preview frame rate smaller than the second target frame rate, so that the frame rate of the video after adjustment is not greater than the first target frame rate and not smaller than the second target frame rate, and encoding and continuous push can be performed. Therefore, in the live broadcasting process of recording the screen, the pushing end cannot be pushed out without video stream, no current interruption is generated, a user enters the live broadcasting room, and the user can see the screen picture of the host broadcasting when pulling the current, so that the success rate of entering the live broadcasting room is effectively improved.
The following description is made by way of example with reference to fig. 5, where the flow of screen recording and pushing provided in the embodiment of the present invention may include processes of screen data collection, frame rate control, video encoding, and pushing, as shown in fig. 5, by way of example, a screen recording and pushing process under an Android system may specifically include:
S501, screen data are collected.
Wherein IMAGEREADER is an image reader, a function get Surface () can be called to provide a Surface layer Surface, mediaProjection Manager is a media projection manager, mediaProjection Manager can call a function get MediaProjection () to obtain a media projection MediaProjection, the media projection can create a virtual display VirtualDisplay based on the Surface layer and the function createVirtualDisplay (), and further generate data, that is, a video recording, and the virtual display can send the generated data to the image reader through the Surface layer. Thus, the electronic equipment can acquire the video recorded on the screen display picture.
S502, frame rate adjustment.
The electronic equipment can adjust the frame rate of the video recording based on the size relation between the preview frame rate and the preset frame rate of the video recording, and the adjusted video is obtained.
S503, video coding.
The electronic device may encode the adjusted video to obtain an encoded video.
S504, plug flow.
The electronic device may push the encoded video onto the server.
In this embodiment, the electronic device may obtain a video recorded on a screen display through virtual display, and adjust a frame rate of the video recorded on the screen, so as to encode and push the adjusted video to the server. When the screen frame rate is too high, a part of image frames can be lost according to a frame loss strategy, so that the recorded video is ensured not to be blurred, the watching experience of a user is improved, and the occupation of a CPU and a memory can be reduced. When the screen frame rate is too low, a certain number of image frames can be supplemented according to the screen display picture of the last frame, so that the condition of cutting off current is avoided, and the success rate of the user entering the live broadcasting room can be effectively improved.
Corresponding to the screen recording and stream pushing method, the embodiment of the invention also provides a screen recording and stream pushing device, and the screen recording and stream pushing device provided by the embodiment of the invention is introduced.
As shown in fig. 6, a screen recording and pushing device, the device includes:
the frame rate obtaining module 601 is configured to obtain a video recording and a frame rate of a screen display, and take the frame rate as a preview frame rate.
And the frame rate control module 602 is configured to adjust the frame rate of the video recording based on the size relationship between the preview frame rate and the preset frame rate, so as to obtain an adjusted video.
The encoding and pushing module 603 is configured to encode the adjusted video to obtain an encoded video, and push the encoded video.
Therefore, in the scheme provided by the embodiment of the invention, the electronic equipment can acquire the video recorded on the screen of the screen display picture and the frame rate, and takes the frame rate as the preview frame rate; based on the size relation between the preview frame rate and the preset frame rate, carrying out frame rate adjustment on the video recorded on the screen to obtain an adjusted video; and further, the adjusted video is encoded to obtain an encoded video, and the encoded video is subjected to stream pushing. By adopting the scheme, the preview frame rate of the video recording can be obtained, and then the video recording is subjected to frame rate adjustment based on the preview frame rate and the preset frame rate, so that the frame rate of the video recording is matched with the configured code rate, the occurrence of blurring or cutoff of pictures in the live broadcast process is reduced, the clear and smooth live broadcast pictures are ensured, and the live broadcast effect of the video recording is improved.
As one implementation of the embodiment of the present invention, the preset frame rate includes a first target frame rate and a second target frame rate, where the first target frame rate is higher than the second target frame rate;
the frame rate control module 602 may include:
And the frame rate relation determining submodule is used for respectively determining the size relation between the preview frame rate and the first target frame rate and the second target frame rate.
And the frame loss sub-module is used for carrying out frame loss processing on the video recording according to a preset frame loss strategy if the preview frame rate is larger than the first target frame rate, so as to obtain an adjusted video.
And the frame supplementing sub-module is used for carrying out frame supplementing processing on the video recording according to a preset frame supplementing strategy if the preview frame rate is smaller than the second target frame rate, so as to obtain an adjusted video.
As shown in fig. 7, the above-mentioned frame loss sub-module may include:
The frame loss ratio determining unit 701 is configured to determine, for a current image frame included in the video recording, a frame loss ratio based on the preview frame rate, the first target frame rate, and a current adjustment parameter.
And the first frame dropping unit 702 is configured to drop the current image frame and adjust the current adjustment parameter according to a first preset manner if the frame dropping ratio is not less than a preset ratio and the number of the current dropped image frames is less than a first target value.
A first circulation unit 703, configured to take a next frame of the current image frame as a current image frame, and trigger the frame loss ratio determining unit 701 until if the frame loss ratio is smaller than the preset ratio, or if the number of the currently discarded image frames is not smaller than the first target value.
And a second frame loss unit 704, configured to discard the current image frame if the frame loss ratio is smaller than the preset ratio and the number of the current reserved image frames is not smaller than a second target value, set the number of the current reserved image frames to 0, and adjust the current adjustment parameter according to a second preset manner.
The second circulation unit 705 is configured to take a next frame of the current image frame as a current image frame, and trigger the frame loss ratio determining unit 701 until the current image frame is a last image frame of the video recording.
As an implementation manner of the embodiment of the present invention, the first frame dropping unit 702 may include:
And the difference value calculating subunit is used for calculating and obtaining a difference value overlaw based on the preview frame rate, the first target frame rate and the current adjusting parameter.
A first adjustment parameter calculation subunit, configured to calculate the current adjustment parameter overshootModifier according to a formula overshootModifier = - (PREVIEWFPS% overshoot)/3;
Wherein PREVIEWFPS is the preview frame rate.
The second frame dropping unit 704 may include:
A second adjustment parameter calculating subunit, configured to calculate the current adjustment parameter according to a formula overshootModifier =overschoot% outputFps;
wherein outputFps is the first target frame rate.
As an implementation manner of the embodiment of the present invention, the above-mentioned difference value calculation subunit may be used to calculate the difference value overboot according to the formula overboot= overshootModifier + (PREVIEWFPS-outputFps).
As an implementation of the embodiment of the present invention, the first target value is a quotient of the difference value and the first target frame rate; the second target value is a quotient of the preview frame rate and the difference.
As an implementation manner of the embodiment of the present invention, the above-mentioned frame supplementing sub-module may include:
and the frame supplementing unit is used for taking the acquired last frame of screen display picture as a supplemental video frame to obtain an adjusted video.
The embodiment of the present invention further provides an electronic device, as shown in fig. 8, including a processor 801, a communication interface 802, a memory 803, and a communication bus 804, where the processor 801, the communication interface 802, and the memory 803 complete communication with each other through the communication bus 804,
A memory 803 for storing a computer program;
the processor 801 is configured to implement the method steps described in any of the above embodiments when executing the program stored in the memory 803.
The communication bus mentioned above for the electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The memory may include random access memory (Random Access Memory, RAM) or may include non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.
In a further embodiment of the present invention, a computer readable storage medium is also provided, in which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the method according to any of the embodiments described above.
In a further embodiment of the present invention, a computer program product comprising instructions is also provided, which when run on a computer causes the computer to perform the method steps of any of the embodiments described above.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus, electronic device, computer readable storage medium, and computer program product embodiments, the description is relatively simple, as relevant to the method embodiments being referred to in the section of the description of the method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims (14)

1. A screen recording and stream pushing method, characterized in that the method comprises the following steps:
Acquiring a video of a screen display picture and a frame rate, and taking the frame rate as a preview frame rate;
determining a size relationship between the preview frame rate and a first target frame rate and a second target frame rate, respectively, the first target frame rate being higher than the second target frame rate;
if the preview frame rate is larger than the first target frame rate, carrying out frame loss processing on the video recording according to a preset frame loss strategy to obtain an adjusted video;
if the preview frame rate is smaller than the second target frame rate, carrying out frame supplementing processing on the video recording according to a preset frame supplementing strategy to obtain an adjusted video;
And encoding the adjusted video to obtain an encoded video, and pushing the encoded video.
2. The method of claim 1, wherein the step of performing frame loss processing on the video recording according to a preset frame loss policy to obtain an adjusted video comprises:
determining a frame loss ratio based on the preview frame rate, the first target frame rate and current adjustment parameters for a current image frame included in the video;
If the frame loss ratio is not smaller than a preset ratio and the number of the image frames which are discarded currently is smaller than a first target value, discarding the current image frames and adjusting the current adjusting parameters according to a first preset mode;
taking the next frame of the current image frame as the current image frame, and returning to the step of determining a frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter until the frame loss ratio is smaller than the preset ratio or the number of the currently discarded image frames is not smaller than the first target value;
If the frame loss ratio is smaller than the preset ratio and the number of the current reserved image frames is not smaller than a second target value, discarding the current image frames, setting the number of the current reserved image frames to 0, and adjusting the current adjustment parameters according to a second preset mode;
And taking the next frame of the current image frame as the current image frame, and returning to the step of determining the frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter until the current image frame is the last image frame of the video.
3. The method according to claim 2, wherein said step of adjusting said current adjustment parameters in a first preset manner comprises:
Calculating a difference value overschot based on the preview frame rate, the first target frame rate and the current adjustment parameter;
Calculating according to a formula overshootModifier = - (PREVIEWFPS% overshoot)/3 to obtain the current adjustment parameter overshootModifier, wherein PREVIEWFPS is the preview frame rate;
the step of adjusting the current adjustment parameters according to a second preset mode comprises the following steps:
And calculating to obtain the current adjustment parameter according to a formula overshootModifier =overschoot% outputFps, wherein outputFps is the first target frame rate.
4. The method of claim 3, wherein the step of calculating a difference value overschot based on the preview frame rate, the first target frame rate, and a current adjustment parameter comprises:
the difference overschot was calculated according to the formula overschot= overshootModifier + (PREVIEWFPS-outputFps).
5. The method of claim 4, wherein the first target value is a quotient of the difference and the first target frame rate; the second target value is a quotient of the preview frame rate and the difference.
6. The method according to any one of claims 1-5, wherein the step of performing frame-filling processing on the recorded video according to a preset frame-filling policy to obtain an adjusted video includes:
and taking the acquired last frame of screen display picture as a supplementary video frame to obtain an adjusted video.
7. A screen recording and pushing device, characterized in that the device comprises:
the frame rate acquisition module is used for acquiring the video of the screen display picture and the frame rate, and taking the frame rate as a preview frame rate;
The frame rate control module is used for adjusting the frame rate of the video recording based on the size relation between the preview frame rate and the preset frame rate to obtain an adjusted video;
The coding and pushing module is used for coding the adjusted video to obtain coded video and pushing the coded video;
The preset frame rate comprises a first target frame rate and a second target frame rate, and the first target frame rate is higher than the second target frame rate;
The frame rate control module includes:
a frame rate relationship determination submodule for determining a magnitude relationship between the preview frame rate and the first target frame rate and the second target frame rate, respectively;
the frame loss sub-module is used for carrying out frame loss processing on the video recording according to a preset frame loss strategy if the preview frame rate is larger than the first target frame rate, so as to obtain an adjusted video;
and the frame supplementing sub-module is used for carrying out frame supplementing processing on the video recording according to a preset frame supplementing strategy if the preview frame rate is smaller than the second target frame rate, so as to obtain an adjusted video.
8. The apparatus of claim 7, wherein the frame loss submodule comprises:
the frame loss ratio determining unit is used for determining a frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter for the current image frame included in the video recording;
the first frame loss unit is used for discarding the current image frame and adjusting the current adjusting parameter according to a first preset mode if the frame loss ratio is not smaller than a preset ratio and the number of the current discarded image frames is smaller than a first target value;
A first circulation unit configured to take a next frame of the current image frame as a current image frame, and trigger the frame loss ratio determining unit until if the frame loss ratio is smaller than the preset ratio, or if the number of the currently discarded image frames is not smaller than the first target value;
A second frame loss unit, configured to discard the current image frame if the frame loss ratio is smaller than the preset ratio and the number of the current reserved image frames is not smaller than a second target value, set the number of the current reserved image frames to 0, and adjust the current adjustment parameter according to a second preset manner;
And the second circulation unit is used for taking the next frame of the current image frame as the current image frame and triggering the frame loss ratio determining unit until the current image frame is the last image frame of the screen recording video.
9. The apparatus of claim 8, wherein the first frame loss unit comprises:
the difference value calculating subunit is used for calculating a difference value overhop based on the preview frame rate, the first target frame rate and the current adjusting parameter;
A first adjustment parameter calculating subunit, configured to calculate the current adjustment parameter overshootModifier according to a formula overshootModifier = - (PREVIEWFPS% overschot)/3, where PREVIEWFPS is the preview frame rate;
The second frame loss unit includes:
And a second adjustment parameter calculating subunit, configured to calculate the current adjustment parameter according to a formula overshootModifier =overschoot% outputFps, where outputFps is the first target frame rate.
10. The apparatus of claim 9, wherein the difference calculation subunit is configured to calculate the difference value overshot according to the formula overshot= overshootModifier + (PREVIEWFPS-outputFps).
11. The apparatus of claim 10, wherein the first target value is a quotient of the difference and the first target frame rate; the second target value is a quotient of the preview frame rate and the difference.
12. The apparatus according to any of claims 7-11, wherein the frame-filling submodule comprises:
and the frame supplementing unit is used for taking the acquired last frame of screen display picture as a supplemental video frame to obtain an adjusted video.
13. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
A memory for storing a computer program;
a processor for carrying out the method steps of any one of claims 1-6 when executing a program stored on a memory.
14. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-6.
CN202111512331.XA 2021-12-08 2021-12-08 Screen recording and stream pushing method and device, electronic equipment and storage medium Active CN114245196B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111512331.XA CN114245196B (en) 2021-12-08 2021-12-08 Screen recording and stream pushing method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111512331.XA CN114245196B (en) 2021-12-08 2021-12-08 Screen recording and stream pushing method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114245196A CN114245196A (en) 2022-03-25
CN114245196B true CN114245196B (en) 2024-04-19

Family

ID=80754884

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111512331.XA Active CN114245196B (en) 2021-12-08 2021-12-08 Screen recording and stream pushing method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114245196B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115037905B (en) * 2022-06-17 2025-04-08 深圳乐播科技有限公司 Screen recording file processing method, electronic equipment and related products
CN117544740A (en) * 2022-07-28 2024-02-09 北京字跳网络技术有限公司 Video recording method, apparatus, device, storage medium, and program product
CN116996639B (en) * 2023-02-13 2024-08-20 深圳Tcl新技术有限公司 Screen-projection frame rate acquisition method and device, computer equipment and storage medium
CN117714696A (en) * 2023-12-14 2024-03-15 支付宝(杭州)信息技术有限公司 Video code rate calculation method and system
CN119110074B (en) * 2024-08-01 2025-06-24 广州三七极耀网络科技有限公司 Method and device for executing service, storage medium and electronic equipment

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108900919A (en) * 2018-06-29 2018-11-27 乐蜜有限公司 Company's wheat method, apparatus, server and terminal in a kind of live streaming
CN109089131A (en) * 2018-09-21 2018-12-25 广州虎牙信息科技有限公司 A kind of record screen live broadcasting method, device, equipment and storage medium based on IOS system
CN110912922A (en) * 2019-12-03 2020-03-24 锐捷网络股份有限公司 Image transmission method and device, electronic equipment and storage medium
CN111641858A (en) * 2020-04-29 2020-09-08 上海推乐信息技术服务有限公司 Audio and video synchronization method and system
CN111918093A (en) * 2020-08-13 2020-11-10 腾讯科技（深圳）有限公司 Live broadcast data processing method and device, computer equipment and storage medium
CN112218148A (en) * 2020-09-11 2021-01-12 杭州易现先进科技有限公司 Screen recording method and device, computer equipment and computer readable storage medium
CN113115111A (en) * 2021-03-16 2021-07-13 广州朗国电子科技有限公司 Video frame rate control method, system and storage medium for screen recording live broadcast

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7986846B2 (en) * 2004-10-26 2011-07-26 Samsung Electronics Co., Ltd Apparatus and method for processing an image signal in a digital broadcast receiver

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108900919A (en) * 2018-06-29 2018-11-27 乐蜜有限公司 Company's wheat method, apparatus, server and terminal in a kind of live streaming
CN109089131A (en) * 2018-09-21 2018-12-25 广州虎牙信息科技有限公司 A kind of record screen live broadcasting method, device, equipment and storage medium based on IOS system
CN110912922A (en) * 2019-12-03 2020-03-24 锐捷网络股份有限公司 Image transmission method and device, electronic equipment and storage medium
CN111641858A (en) * 2020-04-29 2020-09-08 上海推乐信息技术服务有限公司 Audio and video synchronization method and system
CN111918093A (en) * 2020-08-13 2020-11-10 腾讯科技（深圳）有限公司 Live broadcast data processing method and device, computer equipment and storage medium
CN112218148A (en) * 2020-09-11 2021-01-12 杭州易现先进科技有限公司 Screen recording method and device, computer equipment and computer readable storage medium
CN113115111A (en) * 2021-03-16 2021-07-13 广州朗国电子科技有限公司 Video frame rate control method, system and storage medium for screen recording live broadcast

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Dynamic Frame-Skipping Scheme for Live Video Encoders;Z. Zhang et al.;《2010 International Conference on Multimedia Technology》;1-3 *
基于IP城域网的4K超高清融合媒体服务平台;黄强 等;《广播电视网络》;第28卷(第01期);94-98 *

Also Published As

Publication number Publication date
CN114245196A (en) 2022-03-25

Similar Documents

Publication Publication Date Title
CN114245196B (en) Screen recording and stream pushing method and device, electronic equipment and storage medium
JP7012747B2 (en) Video frame coding method, terminal, and storage medium
CN113766226A (en) Image encoding method, apparatus, device and storage medium
CN108881931B (en) A data buffering method and network device
CN110913245A (en) A method and device for controlling video transcoding rate
CN110267100B (en) Code rate switching method and device for FLV (flash video), electronic equipment and storage medium
CN107872669A (en) Video code rate treating method and apparatus
CN108989832B (en) Image data processing method and equipment, storage medium and terminal thereof
CN108063973A (en) A kind of method for decoding video stream and equipment
CN109729353B (en) Video coding method, device, system and medium
EP3522544A1 (en) Estimating video quality of experience
CN112866746A (en) Multi-path streaming cloud game control method, device, equipment and storage medium
WO2018161867A1 (en) Code rate allocation method, device, and storage medium
CN114501014A (en) Video coding parameter processing method, system, device and storage medium
US20130007206A1 (en) Transmission apparatus, control method for transmission apparatus, and storage medium
JP6089846B2 (en) Video distribution system, decoder, and video distribution method
US20080267284A1 (en) Moving picture compression apparatus and method of controlling operation of same
WO2023071469A1 (en) Video processing method, electronic device and storage medium
CN116962806A (en) A video streaming transmission method, terminal device and storage medium
CN108124155B (en) Code rate control method and device and electronic equipment
CN112040235B (en) Video resource encoding method and device and video resource decoding method and device
US10135896B1 (en) Systems and methods providing metadata for media streaming
CN111510703B (en) Video playing method, device and system
CN115633149A (en) Conference data transmission method and device, electronic equipment and storage medium
CN116962613A (en) Data transmission method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant