[go: up one dir, main page]

CN110933517B - Rate switching method, client and computer-readable storage medium - Google Patents

Rate switching method, client and computer-readable storage medium Download PDF

Info

Publication number
CN110933517B
CN110933517B CN201911209068.XA CN201911209068A CN110933517B CN 110933517 B CN110933517 B CN 110933517B CN 201911209068 A CN201911209068 A CN 201911209068A CN 110933517 B CN110933517 B CN 110933517B
Authority
CN
China
Prior art keywords
key frame
playback
current
label information
download
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911209068.XA
Other languages
Chinese (zh)
Other versions
CN110933517A (en
Inventor
周平雨
陈望都
王�琦
程志鹏
潘兴浩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
MIGU Video Technology Co Ltd
MIGU Culture Technology Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
MIGU Video Technology Co Ltd
MIGU Culture Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, MIGU Video Technology Co Ltd, MIGU Culture Technology Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN201911209068.XA priority Critical patent/CN110933517B/en
Publication of CN110933517A publication Critical patent/CN110933517A/en
Application granted granted Critical
Publication of CN110933517B publication Critical patent/CN110933517B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331Caching operations, e.g. of an advertisement for later insertion during playback
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/437Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/438Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/8543Content authoring using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/8547Content authoring involving timestamps for synchronizing content

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本发明实施例涉及多媒体技术领域,公开了一种码率切换方法、客户端和计算机可读存储介质。本发明中,上述码率切换方法包括:获取当前播放的视频中各关键帧的标签信息;若检测到预设的触发操作,则根据当前播放缓冲区的播放数据和各所述关键帧的标签信息,确定目标关键帧;以所述目标关键帧为切换起始位置,进行码率切换,使得可以实现快速平滑切换的效果,减小切换过程中卡顿的概率和时长,节省切换过程中所需下载流量。

Figure 201911209068

Embodiments of the present invention relate to the field of multimedia technologies, and disclose a code rate switching method, a client and a computer-readable storage medium. In the present invention, the above code rate switching method includes: obtaining the label information of each key frame in the currently playing video; if a preset trigger operation is detected, then according to the playback data of the current playback buffer and the label of each key frame information, determine the target key frame; take the target key frame as the switching starting position, and perform bit rate switching, so that the effect of fast and smooth switching can be achieved, the probability and duration of the freeze during the switching process are reduced, and the switching process is saved. Download traffic is required.

Figure 201911209068

Description

码率切换方法、客户端和计算机可读存储介质Rate switching method, client and computer-readable storage medium

技术领域technical field

本发明实施例涉及多媒体技术领域,特别涉及一种码率切换方法、客户端和计算机可读存储介质。Embodiments of the present invention relate to the field of multimedia technologies, and in particular, to a rate switching method, a client, and a computer-readable storage medium.

背景技术Background technique

在多媒体领域,各大主流客户端,为了满足不同带宽用户的需求,将视频清晰度业务划分为若干等级,常见的有标清、高清、超清等。用户在观看视频的过程中,可以根据自己的网络带宽变化情况,主观的在不同清晰度视频内容之间进行码率切换,以便于可以享受顺畅的观影体验。同时,客户端也可以在用户网络带宽充足的情况下,采用自适应的方式进行码率切换,为用户提供更高品质的视频服务。为了防止花屏,码率切换需要以关键帧作为切换点,相关技术中实现码率切换的方式有:1、根据播放缓冲区内关键帧情况,选取切换时间点进行码率切换;2、基于传输流(Transport Stream,简称:TS)片段首帧是关键帧,选取某个TS片段进行码率切换。In the multimedia field, major mainstream clients, in order to meet the needs of users with different bandwidths, divide video definition services into several levels, such as SD, HD, and UHD. In the process of watching videos, users can subjectively switch the bit rate between video contents of different resolutions according to the changes of their own network bandwidth, so that they can enjoy a smooth viewing experience. At the same time, the client can also perform bit rate switching in an adaptive manner when the user's network bandwidth is sufficient to provide users with higher-quality video services. In order to prevent the screen from blurring, the bit rate switching needs to use the key frame as the switching point. The methods to realize the bit rate switching in the related art are: 1. According to the key frame situation in the playback buffer, select the switching time point to perform the bit rate switching; 2. Based on the transmission The first frame of a stream (Transport Stream, TS for short) segment is a key frame, and a certain TS segment is selected for bit rate switching.

然而,发明人发现相关技术中至少存在如下问题:上述第一种方式中当播放缓冲区内没有关键帧时,需要下载等待直到播放缓冲区中有关键帧,这可能会导致长时间卡顿,且需要的下载流量较多。上述第二种方式中若确定需要在第n个TS片段进行码率切换,则需要等待第n-1个TS片段播放完成,即需要等待下载时机,使得切换时长较长。且如果该TS片段实际首帧为非关键帧则会出现花屏现象。综上,相关技术中的码率切换方案在快速性上都存在着不可避免的缺陷。However, the inventor found that there are at least the following problems in the related art: in the above-mentioned first method, when there is no key frame in the playback buffer, it is necessary to download and wait until there is a key frame in the playback buffer, which may cause a long-term freeze, And more download traffic is required. In the above-mentioned second manner, if it is determined that the rate switching needs to be performed in the nth TS segment, it is necessary to wait for the n-1th TS segment to be played, that is, to wait for the download opportunity, so that the switching time is longer. And if the actual first frame of the TS clip is a non-key frame, the screen will appear blurry. To sum up, the rate switching schemes in the related art all have inevitable defects in terms of rapidity.

发明内容SUMMARY OF THE INVENTION

本发明实施方式的目的在于提供一种码率切换方法、客户端和计算机可读存储介质,使得可以实现快速平滑切换的效果,减小切换过程中卡顿的概率和时长,节省切换过程中所需下载流量。The purpose of the embodiments of the present invention is to provide a code rate switching method, a client and a computer-readable storage medium, so that the effect of fast and smooth switching can be achieved, the probability and duration of jamming during the switching process can be reduced, and the time required for the switching process can be saved. Download traffic is required.

为解决上述技术问题,本发明的实施方式提供了一种码率切换方法,包括以下步骤:获取当前播放的视频中各关键帧的标签信息;若检测到预设的触发操作,则根据当前播放缓冲区的播放数据和各所述关键帧的标签信息,确定目标关键帧;以所述目标关键帧为切换起始位置,进行码率切换。In order to solve the above technical problems, the embodiments of the present invention provide a bit rate switching method, which includes the following steps: obtaining the label information of each key frame in the currently playing video; The playback data in the buffer and the label information of each key frame are used to determine the target key frame; and the code rate switching is performed with the target key frame as the switching starting position.

本发明的实施方式还提供了一种客户端,包括:至少一个处理器;以及,与所述至少一个处理器通信连接的存储器;其中,所述存储器存储有可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行,以使所述至少一个处理器能够执行如上述的码率切换方法。Embodiments of the present invention also provide a client, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores data executable by the at least one processor The instructions are executed by the at least one processor, so that the at least one processor can execute the rate switching method as described above.

本发明的实施方式还提供了一种计算机可读存储介质,存储有计算机程序,所述计算机程序被处理器执行时实现上述的码率切换方法。Embodiments of the present invention further provide a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned code rate switching method is implemented.

本发明实施方式相对于现有技术而言,获取当前播放的视频中各关键帧的标签信息;若检测到预设的触发操作,则根据当前播放缓冲区的播放数据和各所述关键帧的标签信息,确定目标关键帧;以所述目标关键帧为切换起始位置,进行码率切换。基于视频中各关键帧的标签信息和当前播放缓冲区中的播放数据,可以方便的选择出合理的目标关键帧进行无缝码率切换。且直接以目标关键帧为切换起始位置,进行码率切换,无需等待切换时机,有利于实现快速平滑切换的效果,减小切换过程中卡顿的概率和时长,节省切换过程中所需的下载流量。Compared with the prior art, the embodiment of the present invention acquires the label information of each key frame in the video currently being played; if a preset trigger operation is detected, then according to the playback data of the current playback buffer and the label information of each key frame The tag information is used to determine the target key frame; and the target key frame is used as the switching start position to perform code rate switching. Based on the label information of each key frame in the video and the playback data in the current playback buffer, a reasonable target key frame can be easily selected for seamless bit rate switching. And directly use the target key frame as the switching starting position to switch the bit rate without waiting for the switching opportunity, which is conducive to achieving the effect of fast and smooth switching, reducing the probability and duration of the freeze during the switching process, and saving the time required in the switching process. download traffic.

另外,所述获取当前播放的视频中各关键帧的标签信息,包括:获取所述视频的M3U8文件;其中,所述M3U8文件包括各所述关键帧的标签信息的下载地址;根据所述下载地址,下载所述当前播放的视频中各关键帧的标签信息。提供了一种可以快速、方便获取各关键帧的标签信息的方式,通过对现有的M3U8文件进行扩展,使其携带上述下载地址,使得客户端在获取M3U8文件时可以通过解析M3U8文件得到下载地址,进而下载得到各关键帧的标签信息。由于M3U8文件为客户端播放视频时,本身需要下载的文件,因此,通过扩展M3U8文件使其携带上述下载地址,方便了客户端在下载M3U8文件后,快速解析得到上述下载地址,而且下载地址本身所占空间不大,不会对现有的M3U8文件的大小造成太大影响。In addition, the obtaining the label information of each key frame in the currently playing video includes: obtaining the M3U8 file of the video; wherein, the M3U8 file includes the download address of the label information of each key frame; according to the download address to download the label information of each key frame in the currently playing video. Provides a way to quickly and conveniently obtain the label information of each key frame. By extending the existing M3U8 file to carry the above download address, the client can obtain the download by parsing the M3U8 file when obtaining the M3U8 file. address, and then download the label information of each key frame. Since the M3U8 file is a file that needs to be downloaded when the client plays a video, extending the M3U8 file to carry the above download address is convenient for the client to quickly parse and obtain the above download address after downloading the M3U8 file, and the download address itself It does not take up much space and will not cause much impact on the size of existing M3U8 files.

另外,所述根据所述下载地址,下载所述当前播放的视频中各关键帧的标签信息,具体为:当检测到所述视频的首帧渲染完成后,根据所述下载地址,下载所述当前播放的视频中各关键帧的标签信息。提供了一种标签信息的下载时机,考虑到若在首帧渲染完之前下载标签信息,会占用不必要的带宽;若在首帧渲染完之后下载标签信息,可能会影响码率切换的速度,即可能存在需要查询各关键帧的标签信息时,标签信息还没有被下载下来的情况。因此,在渲染完首帧后开始下载,有利于在不占用不必要的带宽的同时达到快速起播的效果。确保主要播放性能指标,即快速起播这一性能指标不受影响,以提高用户体验。In addition, the downloading of the label information of each key frame in the currently playing video according to the download address is specifically: after it is detected that the rendering of the first frame of the video is completed, downloading the key frame according to the download address Label information of each key frame in the currently playing video. Provides a time to download tag information. Considering that downloading the tag information before the first frame is rendered, it will take up unnecessary bandwidth; if the tag information is downloaded after the first frame is rendered, it may affect the speed of bit rate switching. That is, there may be a situation in which the label information has not been downloaded when the label information of each key frame needs to be queried. Therefore, starting the download after rendering the first frame is beneficial to achieve the effect of fast playback without occupying unnecessary bandwidth. Make sure that the main playback performance indicator, that is, the performance indicator of fast playback, is not affected to improve user experience.

另外,所述标签信息包括:各所述关键帧的索引序列号、各所述关键帧所在TS片段序列号、各所述关键帧的时间戳、各所述关键帧所在TS片段的位置偏移量。提出了一种新的关键帧的标签信息的定义格式,这种格式与相关技术中对关键帧的描述信息相比,携带的信息少,降低媒资存储和CDN缓存成本,从而节省用户下载流量,提升播放性能。In addition, the label information includes: the index sequence number of each of the key frames, the sequence number of the TS segment where each of the key frames is located, the timestamp of each of the key frames, and the position offset of the TS segment where each of the key frames is located quantity. A new definition format for the label information of key frames is proposed. Compared with the description information of key frames in related technologies, this format carries less information, reduces the cost of media asset storage and CDN caching, and saves user download traffic. to improve playback performance.

另外,所述根据所述当前播放缓冲区的播放数据和查询的关键帧,确定目标关键帧,包括:若检测到所述当前播放缓冲区的播放数据中没有关键帧,则根据所述查询的关键帧的时间戳,选择当前时间戳之前的时间戳对应的关键帧作为所述目标关键帧;若检测到所述当前播放缓冲区的播放数据中有关键帧,则根据当前下载速率估算从上一个关键帧到当前播放位置所需的下载时长,并估算从所述当前播放位置到下一个关键帧所需的播放时长;其中,所述上一个关键帧为根据所述查询的关键帧的时间戳,选择的当前时间戳之前的时间戳对应的关键帧,所述下一个关键帧为所述当前播放缓冲区中的关键帧;若所述下载时长小于所述播放时长,则将所述上一个关键帧确定为所述目标关键帧;若所述下载时长大于所述播放时长,则将所述下一个关键帧确定为所述目标关键帧。提供了在当前播放缓冲区的播放数据中没有关键帧和有关键帧两种情况下确定目标关键帧的具体实现方式,基于当前播放缓冲区中的播放数据和当前下载速率,查询关键帧索引表来合理动态选择目标关键帧进行切换,有利于减小切换过程中卡顿的概率,节省用户下载流量。In addition, determining the target key frame according to the playback data of the current playback buffer and the queried key frame includes: if it is detected that there is no key frame in the playback data of the current playback buffer, then according to the queried key frame Time stamp of the key frame, select the key frame corresponding to the time stamp before the current time stamp as the target key frame; if it is detected that there is a key frame in the playback data of the current playback buffer, then according to the current download rate estimation from the top The download duration required from a key frame to the current playback position, and the estimated playback duration from the current playback position to the next key frame; wherein, the previous key frame is the time of the key frame according to the query stamp, the key frame corresponding to the timestamp before the selected current timestamp, and the next key frame is the key frame in the current playback buffer; if the download duration is less than the playback duration, the above One key frame is determined as the target key frame; if the download duration is greater than the playback duration, the next key frame is determined as the target key frame. Provides a specific implementation method for determining the target key frame when there are no key frames in the playback data of the current playback buffer and there are key frames. Based on the playback data in the current playback buffer and the current download rate, the key frame index table is queried. To reasonably dynamically select the target key frame for switching, it is beneficial to reduce the probability of freezing during the switching process and save the user's download traffic.

另外,所述以所述目标关键帧为切换起始位置,进行码率切换,包括:查询所述目标关键帧所在TS片段的位置偏移量;根据查询的位置偏移量,下载切换后的码率对应的播放数据;其中,所述切换后的码率根据所述触发操作确定。通过查询关键帧索引表可以方便、快速的得到目标关键帧的位置偏移量,直接根据目标关键帧的位置偏移量去下载播放数据。相比于相关技术,本发明实施方式无需按时间顺序依次下载播放数据,等到检测到下载了包括关键帧的播放数据才去下载切换后的码率对应的播放数据,有利于节省用户的下载流量,减少码率切换时长。In addition, using the target key frame as the switching starting position to perform bit rate switching includes: querying the position offset of the TS segment where the target key frame is located; downloading the switched data according to the queried position offset Playback data corresponding to the bit rate; wherein, the switched bit rate is determined according to the trigger operation. By querying the key frame index table, the position offset of the target key frame can be obtained conveniently and quickly, and the playback data can be downloaded directly according to the position offset of the target key frame. Compared with the related art, the embodiment of the present invention does not need to download the playback data in chronological order, and only downloads the playback data corresponding to the switched bit rate after detecting that the playback data including the key frame has been downloaded, which is beneficial to save the download traffic of the user. , to reduce the bit rate switching time.

附图说明Description of drawings

一个或多个实施例通过与之对应的附图中的图片进行示例性说明,这些示例性说明并不构成对实施例的限定。One or more embodiments are exemplified by the pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments.

图1是根据本发明第一实施方式中的码率切换方法的流程图;1 is a flowchart of a code rate switching method according to a first embodiment of the present invention;

图2是根据本发明第二实施方式中的码率切换方法的流程图;2 is a flowchart of a code rate switching method according to a second embodiment of the present invention;

图3是根据本发明第二实施方式中进行码率切换时服务器与客户端的交互流程图;3 is a flow chart of the interaction between the server and the client when the code rate is switched according to the second embodiment of the present invention;

图4是根据本发明第三实施方式中的客户端的结构示意图。FIG. 4 is a schematic structural diagram of a client according to a third embodiment of the present invention.

具体实施方式Detailed ways

为使本发明实施例的目的、技术方案和优点更加清楚,下面将结合附图对本发明的各实施方式进行详细的阐述。然而,本领域的普通技术人员可以理解,在本发明各实施方式中,为了使读者更好地理解本申请而提出了许多技术细节。但是,即使没有这些技术细节和基于以下各实施方式的种种变化和修改,也可以实现本申请所要求保护的技术方案。以下各个实施例的划分是为了描述方便,不应对本发明的具体实现方式构成任何限定,各个实施例在不矛盾的前提下可以相互结合相互引用。In order to make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art can appreciate that, in the various embodiments of the present invention, many technical details are set forth in order for the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be realized. The following divisions of the various embodiments are for the convenience of description, and should not constitute any limitation on the specific implementation of the present invention, and the various embodiments may be combined with each other and referred to each other on the premise of not contradicting each other.

本发明的第一实施方式涉及一种码率切换方法,应用于客户端。其中,客户端可以为手机、电脑等电子设备,客户端中安装有用于播放视频的应用程序,应用程序可以在连接互联网时在线播放视频。客户端在播放视频时,通常先下载当前码率对应的M3U8文件,M3U8文件存储了若干个TS片段的下载地址,TS片段为包含播放数据的文件。客户端根据M3U8文件中各TS片段的下载地址下载TS片段中的播放数据,将下载的播放数据存入播放缓冲区中。客户端按照视频的播放顺序,播放对应的TS片段。如果网络环境发生变化,客户端可以切换播放的视频的码率,以实现顺畅播放视频的目的。下面对本实施方式的码率切换方法的实现细节进行具体的说明,以下内容仅为方便理解提供的实现细节,并非实施本方案的必须。本实施方式的码率切换方法的流程图可以如图1所示,包括:The first embodiment of the present invention relates to a code rate switching method, which is applied to a client. The client may be an electronic device such as a mobile phone or a computer, and an application for playing videos is installed in the client, and the application can play videos online when connected to the Internet. When the client plays a video, it usually downloads the M3U8 file corresponding to the current bit rate first. The M3U8 file stores the download addresses of several TS segments, and the TS segments are files containing playback data. The client downloads the playback data in the TS segment according to the download address of each TS segment in the M3U8 file, and stores the downloaded playback data in the playback buffer. The client plays the corresponding TS segments according to the playback sequence of the videos. If the network environment changes, the client can switch the bit rate of the video being played to achieve the purpose of playing the video smoothly. The implementation details of the code rate switching method in this embodiment will be specifically described below. The following content is only provided for the convenience of understanding, and is not necessary for implementing this solution. The flowchart of the code rate switching method in this embodiment may be shown in FIG. 1 , including:

步骤101:获取当前播放的视频中各关键帧的标签信息。Step 101: Acquire label information of each key frame in the currently playing video.

其中,各关键帧的标签信息,即视频中所有的关键帧的标签信息,可以由媒资管理系统预先生成。The label information of each key frame, that is, the label information of all key frames in the video, may be pre-generated by the media asset management system.

在一个例子中,各关键帧的标签信息的获取方式可以如下:In an example, the way to obtain the label information of each key frame can be as follows:

首先,客户端获取当前播放的视频的M3U8文件,M3U8文件包括各关键帧的标签信息的下载地址。其中,该下载地址可以通过对HTTP的流媒体网络传输协议(HTTP LiveStreaming,简称:HLS)进行扩展,添加在现有的M3U8文件中。比如,使用预设的标签定义该下载地址,将HLS协议和该下载地址结合起来。其中,预设的标签可以根据实际需要进行设置,本实施方式对此不作具体限定。下面以预设的标签为EXT-X-DEFINE标签为例进行说明,EXT-X-DEFINE标签为官方预留的标签,更容易通过官方的认证。本示例中M3U8文件的范例可以如下:First, the client obtains the M3U8 file of the currently playing video, and the M3U8 file includes the download address of the label information of each key frame. Wherein, the download address can be added to the existing M3U8 file by extending the HTTP streaming media network transmission protocol (HTTP LiveStreaming, HLS for short). For example, use a preset tag to define the download address, and combine the HLS protocol with the download address. The preset label may be set according to actual needs, which is not specifically limited in this embodiment. The following is an example of the preset label being the EXT-X-DEFINE label. The EXT-X-DEFINE label is an officially reserved label, which makes it easier to pass the official certification. An example of an M3U8 file in this example could be as follows:

#EXTM3U#EXTM3U

#EXT-X-VERSION:3#EXT-X-VERSION:3

#EXT-X-TARGETDURATION:10#EXT-X-TARGETDURATION:10

#EXT-X-DEFINE:keyframeurl=http://XXXXXX.xml#EXT-X-DEFINE:keyframeurl=http://XXXXXX.xml

#EXT-X-INF:10.000,ts#EXT-X-INF:10.000,ts

0001.ts0001.ts

............

#ENDLIST#ENDLIST

其中,与现有的M3U8文件相比,本示例中的M3U8文件中还包括各关键帧的标签信息的下载地址,即增加了“#EXT-X-DEFINE:keyframeurl=http://XXXXXX.xml”,表示各关键帧的标签信息的下载地址是http://XXXXXX.xml。Among them, compared with the existing M3U8 file, the M3U8 file in this example also includes the download address of the label information of each key frame, that is, "#EXT-X-DEFINE:keyframeurl=http://XXXXXX.xml is added. ", indicating that the download address of the tag information of each key frame is http://XXXXXX.xml.

接着,客户端可以根据各关键帧的标签信息的下载地址,下载当前播放的视频中各关键帧的标签信息。具体的说,用户在观看视频时,客户端可以向服务器发送播放请求,请求下载当前播放的视频的M3U8文件。服务器在接收到播放请求后,返回M3U8文件给客户端。客户端对M3U8文件进行解析,解析出各关键帧的标签信息的下载地址。客户端使用解析出的下载地址去服务器下载,服务器会返回各关键帧的标签信息给客户端。在具体实现中,服务器也可以在接收到客户端的播放请求后,直接将各关键帧的标签信息发送给客户端,或者将各关键帧的标签信息的下载地址发送至客户端。Next, the client can download the label information of each key frame in the currently playing video according to the download address of the label information of each key frame. Specifically, when a user is watching a video, the client can send a play request to the server, requesting to download the M3U8 file of the currently playing video. After receiving the playback request, the server returns the M3U8 file to the client. The client parses the M3U8 file and parses out the download address of the label information of each key frame. The client uses the parsed download address to download to the server, and the server returns the label information of each key frame to the client. In a specific implementation, the server may also directly send the label information of each key frame to the client after receiving the playback request from the client, or send the download address of the label information of each key frame to the client.

在一个例子中,各关键帧的标签信息可以以关键帧索引表的形式进行存储,关键帧索引表中包括各关键帧的标签信息,即存储有视频中所有关键帧的描述信息。该关键帧索引表可以由媒资管理系统预先生成,且关键帧索引表的存储格式可以为可扩展标记语言(Extensible Markup Language,简称:xml)。基于xml格式的文件轻量级存储及支持多平台和多种编码的特性,本示例中关键帧索引表采用xml格式进行存储和解析。以下将xml格式存储的关键帧索引表称为xml文件。需要说明的是,本示例中关键帧索引表的存储格式只是以xml格式为例,在具体实现中并不以此为限,本领域技术人员可以根据实际需要选择存储格式。In one example, the label information of each key frame may be stored in the form of a key frame index table, and the key frame index table includes label information of each key frame, that is, description information of all key frames in the video is stored. The key frame index table may be pre-generated by the media asset management system, and the storage format of the key frame index table may be Extensible Markup Language (Extensible Markup Language, xml for short). The lightweight storage of files based on xml format and the feature of supporting multiple platforms and multiple encodings, in this example, the key frame index table is stored and parsed in xml format. The key frame index table stored in the xml format is hereinafter referred to as an xml file. It should be noted that the storage format of the key frame index table in this example is only the xml format, which is not limited in the specific implementation, and those skilled in the art can select the storage format according to actual needs.

在一个例子中,为了保证xml文件数据的有效性,本实施方式中提供了一种新的关键帧标签的定义格式,可以有效地减小xml文件大小,降低媒资存储和内容分发网络(Content Delivery Network,简称:CND)的缓存成本,从而节省用户下载流量,提升播放性能。在这种新的格式下:各关键帧的标签信息可以包括:各关键帧的索引序列号id、各关键帧所在TS片段的序列号seq、各关键帧的时间戳pts(单位:毫秒)、各关键帧所在TS片段的位置偏移量offset。在具体实现中,根据实际需要标签信息还可以包括各关键帧的画面信息,即自定义信息字符串info,用于描述关键帧画面的相关信息,如关键帧的图像信息描述等。其中,各关键帧的画面信息为可选项,以便于根据实际需要对关键帧的标签信息进行扩展。为方便理解,下面给出关键帧索引表的范例:In one example, in order to ensure the validity of the xml file data, this embodiment provides a new definition format of the key frame tag, which can effectively reduce the size of the xml file, reduce the storage of media resources and the content distribution network (Content Distribution Network). Delivery Network (CND for short) cache cost, thus saving user download traffic and improving playback performance. In this new format: the label information of each key frame may include: the index sequence number id of each key frame, the sequence number seq of the TS segment where each key frame is located, the timestamp pts of each key frame (unit: milliseconds), The position offset of the TS segment where each key frame is located. In a specific implementation, according to actual needs, the label information may also include picture information of each key frame, that is, a custom information string info, which is used to describe the relevant information of the key frame picture, such as image information description of the key frame. The picture information of each key frame is optional, so as to expand the label information of the key frame according to actual needs. For ease of understanding, an example of a keyframe index table is given below:

<keyframes><keyframes>

<id=“11”seq=“41”pts=“40”offset=“0”info=“XXXXXX”/><id="11" seq="41" pts="40" offset="0" info="XXXXXX"/>

<id=“12”seq=“41”pts=“2040”offset=“1024”info=“XXXXXX”/><id="12" seq="41" pts="2040" offset="1024" info="XXXXXX"/>

<id=“13”seq=“42”pts=“4040”offset=“2048”info=“XXXXXX”/><id="13" seq="42" pts="4040" offset="2048" info="XXXXXX"/>

<id=“14”seq=“42”pts=“6040”offset=“3072”info=“XXXXXX”/><id="14" seq="42" pts="6040" offset="3072" info="XXXXXX"/>

<id=“15”seq=“43”pts=“8040”offset=“4096”info=“XXXXXX”/><id="15" seq="43" pts="8040" offset="4096" info="XXXXXX"/>

<id=“16”seq=“43”pts=“10040”offset=“5100”info=“XXXXXX”/><id="16" seq="43" pts="10040" offset="5100" info="XXXXXX"/>

............

</keyframes></keyframes>

需要说明的是,上述关键帧索引表中只是为方便理解,示例性的列出了视频中第11个至第16个关键帧的标签信息,在具体实现中视频中每一个关键帧的标签信息均会包含在关键帧索引表中。It should be noted that the above key frame index table is only for the convenience of understanding, and the label information of the 11th to 16th key frames in the video is exemplarily listed. In the specific implementation, the label information of each key frame in the video is listed. are included in the keyframe index table.

可以理解的是,若各关键帧的标签信息以关键帧索引表的形式进行存储,那么M3U8文件中包括的各关键帧的标签信息的下载地址,可以具体为关键帧索引表的下载地址。客户端在下载得到关键帧索引表后,可以通过解析得到各关键帧的标签信息。It can be understood that, if the label information of each key frame is stored in the form of a key frame index table, then the download address of the label information of each key frame included in the M3U8 file may specifically be the download address of the key frame index table. After downloading and obtaining the key frame index table, the client can obtain the label information of each key frame by parsing.

步骤102:若检测到预设的触发操作,则根据当前播放缓冲区的播放数据和各所述关键帧的标签信息,确定目标关键帧。Step 102: If a preset trigger operation is detected, the target key frame is determined according to the playback data of the current playback buffer and the label information of each of the key frames.

在一个例子中,预设的触发操作可以为:用户的点击操作,即触发操作为手动触发操作。比如,客户端在播放视频时,播放界面上通常会有用于码率切换的虚拟按键,用户可以根据实际的网络环境选择切换为标清、高清或超清等。若客户端检测到用于码率切换的虚拟按键被点击,则可以确定检测到预设的触发操作。可以理解的是,视频网站往往会提供多种码率的视频流,码率越高,视频清晰度越高,但同样视频的数据量较大,所需的流量较多;反正,码率越低,视频清晰度越低,但同样视频的数据量较小,所需的流量较小。上述的标清、高清或超清可以分别表征三个依次增大的码率,代表视频的清晰度越来越高。In an example, the preset trigger operation may be: a user's click operation, that is, the trigger operation is a manual trigger operation. For example, when the client is playing a video, there are usually virtual buttons on the playback interface for switching the bit rate, and the user can choose to switch to SD, HD, or UHD according to the actual network environment. If the client detects that the virtual key used for rate switching is clicked, it can be determined that a preset trigger operation is detected. It is understandable that video websites often provide video streams with various bit rates. Low, the lower the video resolution, but the same video data volume is smaller, the required traffic is smaller. The above-mentioned standard definition, high definition or ultra-definition can respectively represent three bit rates that increase in sequence, which means that the definition of the video is getting higher and higher.

在另一个例子中,预设的触发操作可以为:当前网络带宽与播放的视频的当前码率不匹配,即触发操作为自适应触发操作。具体的说,若检测到当前网络带宽与当前码率不匹配,则可以认为检测到预设的触发操作。其中,确定当前网络带宽与当前码率是否匹配的方式可以为:判断当前网络带宽与当前码率之差的绝对值是否小于预设值,若小于预设值可以认为当前网络带宽与当前码率匹配,否则认为不匹配;预设值的大小可以根据实际需要进行设置,本实施方式对此不做具体限定。比如,若当前网络带宽1024kbps,当前码率为720kbps,可以确定当前网络带宽与当前码率不匹配;该触发操作可以表明:需要将当前码率切换到比当前码率更高的码率,即与当前网络带宽匹配的码率。再比如,当前网络带宽780kbps,当前码率为1028kbps,可以确定当前网络带宽与当前码率不匹配;该触发操作可以表明:需要将当前码率切换到比当前码率更低的码率,即与当前网络带宽匹配的码率。In another example, the preset trigger operation may be: the current network bandwidth does not match the current bit rate of the video being played, that is, the trigger operation is an adaptive trigger operation. Specifically, if it is detected that the current network bandwidth does not match the current bit rate, it can be considered that a preset trigger operation is detected. Wherein, the method of determining whether the current network bandwidth matches the current code rate may be: judging whether the absolute value of the difference between the current network bandwidth and the current code rate is less than a preset value, if it is less than the preset value, it can be considered that the current network bandwidth and the current code rate match, otherwise, it is considered not to match; the size of the preset value can be set according to actual needs, which is not specifically limited in this embodiment. For example, if the current network bandwidth is 1024kbps and the current bit rate is 720kbps, it can be determined that the current network bandwidth does not match the current bit rate; the trigger operation can indicate that the current bit rate needs to be switched to a bit rate higher than the current bit rate, that is, The bit rate that matches the current network bandwidth. For another example, if the current network bandwidth is 780kbps and the current bit rate is 1028kbps, it can be determined that the current network bandwidth does not match the current bit rate; the trigger operation can indicate that the current bit rate needs to be switched to a bit rate lower than the current bit rate, that is, The bit rate that matches the current network bandwidth.

在一个例子中,根据当前播放缓冲区的播放数据和各关键帧的标签信息,确定目标关键帧的具体实现方式,可以如下:In an example, according to the playback data of the current playback buffer and the label information of each key frame, the specific implementation manner of determining the target key frame may be as follows:

首先,可以确定当前播放的TS片段的序列号。可以理解的是,当前播放的视频包括若干个TS片段,若检测到预设的触发操作,则可以确定当前播放的视频内容所属TS片段的序列号seq。参考上述的关键帧索引表的范例,假设确定的当前播放的TS片段的序列号为41,即表明当前播放的是第41个TS片段。First, the sequence number of the currently playing TS segment can be determined. It can be understood that the currently played video includes several TS segments, and if a preset trigger operation is detected, the sequence number seq of the TS segment to which the currently played video content belongs can be determined. Referring to the above example of the key frame index table, it is assumed that the determined sequence number of the currently playing TS segment is 41, which means that the currently playing TS segment is the 41st TS segment.

接着,根据查询关键帧索引表,即根据各关键帧的标签信息,查询正在播放的TS片段的序列号对应的关键帧,比如可以在关键帧索引表中查询正在播放的TS片段的序列号对应的关键帧。参考上述的关键帧索引表的范例,假设当前播放的是第41个TS片段即seq=“41”,那么查询的对应的关键帧即为索引序列号id=“11”和id=“12”的关键帧,以下分别用关键帧11和关键帧12表示。Next, according to the query key frame index table, that is, according to the label information of each key frame, query the key frame corresponding to the sequence number of the TS clip being played. keyframes. Referring to the example of the key frame index table above, assuming that the 41st TS segment currently being played is seq=“41”, then the corresponding key frame of the query is the index sequence number id=”11” and id=”12” The key frames of , are represented by key frame 11 and key frame 12 respectively below.

然后,根据当前播放缓冲区的播放数据和查询的关键帧,确定目标关键帧。Then, the target key frame is determined according to the playback data of the current playback buffer and the queried key frame.

在一个例子中,若检测到当前播放缓冲区的播放数据中没有关键帧,则根据查询的关键帧的时间戳,选择当前时间戳之前的时间戳对应的关键帧作为目标关键帧。比如,参考上述的关键帧索引表的范例,假设当前时间戳pts=“1040”,通过关键帧索引表可以查询得到关键帧11的时间戳pts=“40”,关键帧12的时间戳pts=“2040”。由此可见,关键帧11的时间戳pts=“40”在当前时间戳pts=“1040”之前,则可以确定关键帧11为目标关键帧。In one example, if it is detected that there is no key frame in the playback data of the current playback buffer, according to the time stamp of the queried key frame, a key frame corresponding to a time stamp before the current time stamp is selected as the target key frame. For example, referring to the example of the key frame index table above, assuming that the current timestamp pts="1040", the timestamp pts="40" of the key frame 11 and the timestamp pts= "40" of the key frame 12 can be obtained by querying the key frame index table. "2040". It can be seen that, if the timestamp pts="40" of the key frame 11 is before the current timestamp pts="1040", the key frame 11 can be determined as the target key frame.

在另一个例子中,若检测到当前播放缓冲区的播放数据中有关键帧,则根据当前下载速率估算从上一个关键帧到当前播放位置所需的下载时长,并估算从当前播放位置到下一个关键帧所需的播放时长;其中,上一个关键帧为根据查询的关键帧的时间戳,选择的当前时间戳之前的时间戳对应的关键帧,下一个关键帧为当前播放缓冲区中的关键帧。若下载时长小于播放时长,则将上一个关键帧确定为目标关键帧;若下载时长大于播放时长,则将下一个关键帧确定为目标关键帧。在具体实现中,若下载时长等于播放时长,则可以将上一个关键帧确定为目标关键帧,也可以将下一个关键帧确定为目标关键帧,本实施方式对此不做具体限定。比如,参考上述的关键帧索引表的范例,假设当前时间戳pts=“1040”,即当前播放位置处pts=“1040”,通过关键帧索引表可以查询得到关键帧11的时间戳pts=“40”,关键帧12的时间戳pts=“2040”。由此可见,关键帧11的时间戳pts=“40”在当前时间戳pts=“1040”之前,则可以确定关键帧11为上一个关键帧,接着根据当前下载速率估算从关键帧11到当前播放位置所需的下载时长。将检测到的当前播放缓冲区的关键帧作为下一个关键帧,预估从当前播放位置到前播放缓冲区的关键帧所需的播放时长。最后,根据下载时长和播放时长的长短确定将哪一个关键帧作为目标关键帧。In another example, if it is detected that there are key frames in the playback data of the current playback buffer, the download time required from the previous key frame to the current playback position is estimated according to the current download rate, and the duration from the current playback position to the next playback position is estimated. The playback duration required for a key frame; among them, the previous key frame is the key frame corresponding to the timestamp before the selected current timestamp according to the timestamp of the queried key frame, and the next key frame is the one in the current playback buffer. Keyframe. If the download duration is shorter than the playback duration, the previous key frame is determined as the target key frame; if the download duration is longer than the playback duration, the next key frame is determined as the target key frame. In a specific implementation, if the download duration is equal to the playback duration, the previous key frame may be determined as the target key frame, and the next key frame may also be determined as the target key frame, which is not specifically limited in this embodiment. For example, referring to the example of the key frame index table above, assuming that the current timestamp pts="1040", that is, the current playback position pts="1040", the timestamp pts="1040" of the key frame 11 can be obtained by querying the key frame index table 40", the timestamp of key frame 12 pts="2040". It can be seen that the timestamp pts="40" of the key frame 11 is before the current timestamp pts="1040", then it can be determined that the key frame 11 is the previous key frame, and then the current download rate is estimated from the key frame 11 to the current The download time required for the playback location. Use the detected key frame of the current playback buffer as the next key frame, and estimate the playback duration from the current playback position to the key frame of the previous playback buffer. Finally, determine which keyframe to use as the target keyframe according to the download duration and playback duration.

为进一步方便理解,可以参考以下示例:假设当前播放位置为2秒处,上一个关键帧(通过关键帧索引表查询到的)的位置为0秒处,下一个关键帧(当前缓冲区中存在的)的位置为5秒处。若当前下载速度比较快,使得下载时长较短,则可以将0秒处的关键帧作为目标关键帧,那么客户端不会继续播放当前缓冲区内2秒至5秒之间播放数据,直接去下载切换后的码率对应的0秒处的播放数据。若当前下载速度比较慢,使得下载时长较长,则可以将5秒处的关键帧作为目标关键帧,那么客户端会继续播放当前缓冲区内2秒至5秒之间播放数据,然后再去下载切换后的码率对应的5秒处的播放数据。For further understanding, you can refer to the following example: Assuming that the current playback position is 2 seconds, the position of the previous key frame (queried through the key frame index table) is 0 seconds, and the next key frame (existing in the current buffer ) is at 5 seconds. If the current download speed is fast and the download time is short, the key frame at 0 seconds can be used as the target key frame, then the client will not continue to play the data between 2 seconds and 5 seconds in the current buffer, and go directly to the Download the playback data at 0 seconds corresponding to the switched bit rate. If the current download speed is relatively slow and the download time is long, the key frame at 5 seconds can be used as the target key frame, then the client will continue to play the data in the current buffer from 2 seconds to 5 seconds, and then go to Download the playback data at 5 seconds corresponding to the switched bit rate.

在具体实现中,若关键帧索引表中包含自定义信息字符串info,由于info可用于描述某个关键帧得画面信息,因此还可以将info作为确定目标关键帧的参考依据,例如指定某个动作画面时进行码率切换等等。In the specific implementation, if the key frame index table contains the custom information string info, since info can be used to describe the picture information of a certain key frame, info can also be used as a reference for determining the target key frame, for example, specifying a certain key frame Bit rate switching and so on during the action screen.

步骤103:以目标关键帧为切换起始位置,进行码率切换。Step 103: Use the target key frame as the starting position for switching, and perform bit rate switching.

在一个例子中,首先可以根据各关键帧的标签信息,查询目标关键帧所在TS片段的位置偏移量。参考上述关键帧索引表的范例,假设目标关键帧为关键帧11,则查询到的目标关键帧所在TS片段的位置偏移量offset=“0”。然后,可以根据查询的位置偏移量,下载切换后的码率对应的播放数据;即对于切换后的码率对应的播放数据,从查询的位置偏移量开始下载并播放。In an example, first, the position offset of the TS segment where the target key frame is located may be queried according to the label information of each key frame. Referring to the example of the key frame index table above, assuming that the target key frame is key frame 11, the position offset of the TS segment where the queried target key frame is located is offset="0". Then, the playback data corresponding to the switched bit rate can be downloaded according to the queried position offset; that is, the playback data corresponding to the switched bit rate can be downloaded and played from the queried position offset.

其中,切换后的码率根据触发操作确定。比如,预设的触发操作为:用户的点击操作;视频的当前码率为标清这一等级对应的码率,若用户点击了切换为高清,则切换后的码率为高清这一等级对应的码率。再比如,预设的触发操作为:自适应触发操作;若当前网络带宽与播放的视频的当前码率不匹配,则切换后的码率为与当前网络带宽匹配的码率。The code rate after switching is determined according to the trigger operation. For example, the preset trigger operation is: the user's click operation; the current bit rate of the video is the bit rate corresponding to the standard definition level. If the user clicks to switch to high definition, the switched bit rate corresponds to the high definition level. code rate. For another example, the preset trigger operation is: an adaptive trigger operation; if the current network bandwidth does not match the current bit rate of the video being played, the switched bit rate is the bit rate that matches the current network bandwidth.

需要说明的是,本实施方式中的上述各示例均为为方便理解进行的举例说明,并不对本发明的技术方案构成限定。It should be noted that, the above examples in this embodiment are all examples for the convenience of understanding, and do not limit the technical solutions of the present invention.

与现有技术相比,本实施方式,基于视频中各关键帧的标签信息和当前播放缓冲区中的播放数据,使得可以方便的选择合理的目标关键帧进行无缝码率切换。且无需等待切换时机,直接以目标关键帧为切换起始位置,进行码率切换,有利于实现快速平滑切换的效果,减小切换过程中卡顿的概率和时长,节省切换过程中所需的下载流量。而且,提出了一种新的关键帧的标签信息的定义格式,这种格式与相关技术中对关键帧的描述信息相比,携带的信息少,可以有效的减小xml格式下关键帧索引表所占的空间大小,降低媒资存储和CDN缓存成本,从而节省用户下载流量,提升播放性能。Compared with the prior art, in this embodiment, based on the label information of each key frame in the video and the playback data in the current playback buffer, it is possible to conveniently select a reasonable target key frame for seamless bit rate switching. And there is no need to wait for the switching opportunity, directly use the target key frame as the starting position for switching, and perform bit rate switching, which is conducive to the effect of fast and smooth switching, reduces the probability and duration of the freeze during the switching process, and saves the switching process. download traffic. Moreover, a new definition format of key frame label information is proposed. Compared with the description information of key frames in the related art, this format carries less information and can effectively reduce the key frame index table in xml format. The size of the space occupied reduces the cost of media asset storage and CDN caching, thereby saving user download traffic and improving playback performance.

本发明的第二实施方式涉及一种码率切换方法。下面对本实施方式的码率切换方法的实现细节进行具体的说明,以下内容仅为方便理解提供的实现细节,并非实施本方案的必须。本实施方式的码率切换方法的流程图可以如图2所示,包括:The second embodiment of the present invention relates to a code rate switching method. The implementation details of the code rate switching method in this embodiment will be specifically described below. The following content is only provided for the convenience of understanding, and is not necessary for implementing this solution. The flowchart of the code rate switching method of this embodiment may be shown in FIG. 2 , including:

步骤201:获取当前播放的视频的M3U8文件。Step 201: Obtain the M3U8 file of the currently playing video.

步骤202:解析M3U8文件,获取各关键帧的标签信息的下载地址。Step 202: Parse the M3U8 file to obtain the download address of the label information of each key frame.

步骤203:当检测到视频的首帧渲染完成后,根据下载地址,下载当前播放的视频中各关键帧的标签信息。Step 203 : when it is detected that the rendering of the first frame of the video is completed, download the label information of each key frame in the currently playing video according to the download address.

具体的说,客户端在下载播放数据存入播放缓冲区后,会启动解码器对播放数据进行解码,然后渲染播放。当检测到首帧画面被渲染播放后,客户端使用解析出的下载地址去下载各关键帧的标签信息。比如,客户端在一开始解析出下载地址后,可以先将下载地址进行缓存,在检测到首帧画面渲染完成后,再开始根据下载地址下载得到各关键帧的标签信息。Specifically, after the client downloads the playback data and stores it in the playback buffer, it starts the decoder to decode the playback data, and then renders it for playback. After detecting that the first frame is rendered and played, the client uses the parsed download address to download the label information of each key frame. For example, after the client first parses out the download address, it can cache the download address first, and after detecting that the rendering of the first frame is completed, it starts to download and obtain the label information of each key frame according to the download address.

步骤204:若检测到预设的触发操作,则根据当前播放缓冲区的播放数据和各关键帧的标签信息,确定目标关键帧。Step 204: If a preset trigger operation is detected, the target key frame is determined according to the playback data of the current playback buffer and the label information of each key frame.

步骤205:以目标关键帧为切换起始位置,进行码率切换。Step 205: Using the target key frame as the switching start position, perform bit rate switching.

其中,上述步骤201至步骤205中,除步骤203外,均在第一实施方式中详细介绍过,可以参考第一实施方式中的相关描述,为避免重复,本实施方式不再一一赘述。Wherein, the above steps 201 to 205, except for step 203, have been described in detail in the first embodiment, and the relevant descriptions in the first embodiment can be referred to. To avoid repetition, this embodiment will not describe them one by one.

为进一步方便理解本实施方式中的码率切换方法,下面以客户端与服务器的具体交互示意图进行说明,其中,各关键帧的标签信息以存储在关键帧索引表中为例。参考图3,码率切换方法的交互过程包括:In order to further facilitate the understanding of the rate switching method in this embodiment, a schematic diagram of a specific interaction between a client and a server is described below, wherein the label information of each key frame is stored in a key frame index table as an example. Referring to Figure 3, the interaction process of the rate switching method includes:

S301,客户端向服务器发送播放请求。即,用户在观看视频时,客户端检测到用户点击了开始播放的控件,向服务器发送播放请求,请求去下载M3U8文件。S301, the client sends a play request to the server. That is, when the user is watching the video, the client detects that the user has clicked the control to start playing, and sends a play request to the server, requesting to download the M3U8 file.

S302,服务器向客户端返回M3U8文件。即,服务器接收到客户端发送的播放请求后,返回视频的M3U8文件给客户端。S302, the server returns the M3U8 file to the client. That is, after receiving the playback request sent by the client, the server returns the M3U8 file of the video to the client.

S303,客户端解析M3U8文件,得到关键帧索引表的下载地址。另外,通过解析M3U8文件客户端还可以得到播放数据的下载地址,从而可以下载播放数据存入播放缓冲区,并启动解码器解码后渲染播放。S303, the client parses the M3U8 file to obtain the download address of the key frame index table. In addition, by parsing the M3U8 file, the client can also obtain the download address of the playback data, so that the playback data can be downloaded and stored in the playback buffer, and the decoder can be started to render and play after decoding.

S304,客户端检测到首帧渲染完成后,请求下载关键帧索引表。S304, after detecting that the rendering of the first frame is completed, the client requests to download the key frame index table.

S305,服务器向客户端返回关键帧索引表。其中,客户端可以根据关键帧索引表的定义格式,解析各关键帧的标签信息。S305, the server returns the key frame index table to the client. The client can parse the label information of each key frame according to the definition format of the key frame index table.

S306,客户端检测到预设的触发操作,选取目标关键帧作为切换起始点请求播放数据。比如,用户点击进行无缝码率切换操作时,客户端基于当前播放缓冲区中有无关键帧,再结合当前下载速率,选择目标关键帧来进行无缝码率切换。客户端可以将目标关键帧作为切换起始点,向服务器请求切换后的码率对应的播放数据。S306, the client detects the preset trigger operation, and selects the target key frame as the switching starting point to request playback data. For example, when the user clicks to perform a seamless bit rate switching operation, the client selects the target key frame to perform seamless bit rate switching based on whether there are key frames in the current playback buffer and combined with the current download rate. The client can use the target key frame as the switching starting point, and request the server for playback data corresponding to the switched bit rate.

与现有技术相比,本实施方式中,提供了一种各关键帧的标签信息的下载时机,考虑到若在首帧渲染完之前下载标签信息,会占用不必要的带宽;若在首帧渲染完之后下载标签信息,可能会影响码率切换的速度,即可能存在需要查询标签信息时,标签信息还没有被下载下来的情况。因此,在渲染完首帧后开始下载,有利于在不占用不必要的带宽的同时达到快速起播的效果。确保主要播放性能指标,即快速起播这一性能指标不受影响,以提高用户体验。Compared with the prior art, in this embodiment, a download timing of the label information of each key frame is provided, considering that if the label information is downloaded before the first frame is rendered, unnecessary bandwidth will be occupied; Downloading the tag information after rendering may affect the speed of bit rate switching, that is, there may be a situation where the tag information has not been downloaded when the tag information needs to be queried. Therefore, starting the download after rendering the first frame is beneficial to achieve the effect of fast playback without occupying unnecessary bandwidth. Make sure that the main playback performance indicator, that is, the performance indicator of fast playback, is not affected to improve user experience.

上面各种方法的步骤划分,只是为了描述清楚,实现时可以合并为一个步骤或者对某些步骤进行拆分,分解为多个步骤,只要包括相同的逻辑关系,都在本专利的保护范围内;对算法中或者流程中添加无关紧要的修改或者引入无关紧要的设计,但不改变其算法和流程的核心设计都在该专利的保护范围内。The steps of the above various methods are divided only for the purpose of describing clearly. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as the same logical relationship is included, they are all within the protection scope of this patent. ;Adding insignificant modifications to the algorithm or process or introducing insignificant designs, but not changing the core design of the algorithm and process are all within the scope of protection of this patent.

本发明第三实施方式涉及一种客户端,如图4所示,包括:至少一个处理器401;以及,与至少一个处理器401通信连接的存储器402;其中,存储器402存储有可被至少一个处理器401执行的指令,指令被至少一个处理器401执行,以使至少一个处理器401能够执行第一、或第二实施方式中的码率切换方法。The third embodiment of the present invention relates to a client, as shown in FIG. 4 , comprising: at least one processor 401; and a memory 402 connected in communication with the at least one processor 401; Instructions executed by the processor 401, the instructions are executed by at least one processor 401, so that the at least one processor 401 can execute the code rate switching method in the first or second embodiment.

其中,存储器402和处理器401采用总线方式连接,总线可以包括任意数量的互联的总线和桥,总线将一个或多个处理器401和存储器402的各种电路连接在一起。总线还可以将诸如外围设备、稳压器和功率管理电路等之类的各种其他电路连接在一起,这些都是本领域所公知的,因此,本文不再对其进行进一步描述。总线接口在总线和收发机之间提供接口。收发机可以是一个元件,也可以是多个元件,比如多个接收器和发送器,提供用于在传输介质上与各种其他装置通信的单元。经处理器401处理的数据通过天线在无线介质上进行传输,进一步,天线还接收数据并将数据传送给处理器401。The memory 402 and the processor 401 are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors 401 and various circuits of the memory 402 together. The bus may also connect together various other circuits, such as peripherals, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides the interface between the bus and the transceiver. A transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a means for communicating with various other devices over a transmission medium. The data processed by the processor 401 is transmitted on the wireless medium through the antenna, and further, the antenna also receives the data and transmits the data to the processor 401 .

处理器401负责管理总线和通常的处理,还可以提供各种功能,包括定时,外围接口,电压调节、电源管理以及其他控制功能。而存储器402可以被用于存储处理器401在执行操作时所使用的数据。Processor 401 is responsible for managing the bus and general processing, and may also provide various functions including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory 402 may be used to store data used by the processor 401 when performing operations.

本发明第四实施方式涉及一种计算机可读存储介质,存储有计算机程序。计算机程序被处理器执行时实现上述方法实施例。A fourth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The above method embodiments are implemented when the computer program is executed by the processor.

即,本领域技术人员可以理解,实现上述实施例方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成,该程序存储在一个存储介质中,包括若干指令用以使得一个设备(可以是单片机,芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(ROM,Read-OnlyMemory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。That is, those skilled in the art can understand that all or part of the steps in the method of implementing the above embodiments can be completed by instructing relevant hardware through a program, and the program is stored in a storage medium and includes several instructions to make a device ( It may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, removable hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes.

本领域的普通技术人员可以理解,上述各实施方式是实现本发明的具体实施例,而在实际应用中,可以在形式上和细节上对其作各种改变,而不偏离本发明的精神和范围。Those skilled in the art can understand that the above-mentioned embodiments are specific examples for realizing the present invention, and in practical applications, various changes in form and details can be made without departing from the spirit and the spirit of the present invention. scope.

Claims (8)

1.一种码率切换方法,其特征在于,包括:1. a code rate switching method, is characterized in that, comprises: 获取当前播放的视频中各关键帧的标签信息;Get the label information of each key frame in the currently playing video; 若检测到预设的触发操作,则根据当前播放缓冲区的播放数据和各所述关键帧的标签信息,确定目标关键帧;If a preset trigger operation is detected, then according to the playback data of the current playback buffer and the label information of each of the key frames, the target key frame is determined; 以所述目标关键帧为切换起始位置,进行码率切换;Taking the target key frame as the switching starting position, the code rate switching is performed; 其中,所述标签信息包括:各所述关键帧的索引序列号、各所述关键帧所在TS片段的序列号、各所述关键帧的时间戳、各所述关键帧所在TS片段的位置偏移量;The label information includes: the index sequence number of each key frame, the sequence number of the TS segment where each key frame is located, the time stamp of each key frame, and the position offset of the TS segment where each key frame is located. shift; 其中,所述根据当前播放缓冲区的播放数据和各所述关键帧的标签信息,确定目标关键帧,包括:Wherein, determining the target key frame according to the playback data of the current playback buffer and the label information of each key frame includes: 确定当前播放的TS片段的序列号;Determine the sequence number of the currently playing TS segment; 根据各所述关键帧的标签信息,查询所述当前播放的TS片段的序列号对应的关键帧;According to the label information of each described key frame, query the key frame corresponding to the sequence number of the currently playing TS segment; 根据所述当前播放缓冲区的播放数据和查询的关键帧,确定目标关键帧。The target key frame is determined according to the play data of the current play buffer and the queried key frame. 2.根据权利要求1所述的码率切换方法,其特征在于,所述获取当前播放的视频中各关键帧的标签信息,包括:2. bit rate switching method according to claim 1, is characterized in that, described obtaining the label information of each key frame in the video that is currently playing, comprises: 获取所述视频的M3U8文件;其中,所述M3U8文件包括各所述关键帧的标签信息的下载地址;Obtain the M3U8 file of the video; Wherein, the M3U8 file includes the download address of the label information of each described key frame; 根据所述下载地址,下载所述当前播放的视频中各关键帧的标签信息。According to the download address, the tag information of each key frame in the currently playing video is downloaded. 3.根据权利要求2所述的码率切换方法,其特征在于,所述根据所述下载地址,下载所述当前播放的视频中各关键帧的标签信息,具体为:3. The code rate switching method according to claim 2, wherein, according to the download address, the label information of each key frame in the currently played video is downloaded, specifically: 当检测到所述视频的首帧渲染完成后,根据所述下载地址,下载所述当前播放的视频中各关键帧的标签信息。After it is detected that the rendering of the first frame of the video is completed, the label information of each key frame in the currently playing video is downloaded according to the download address. 4.根据权利要求1所述的码率切换方法,其特征在于,所述根据所述当前播放缓冲区的播放数据和查询的关键帧,确定目标关键帧,包括:4. The code rate switching method according to claim 1, wherein, determining the target key frame according to the play data of the current play buffer and the key frame of the query, comprising: 若检测到所述当前播放缓冲区的播放数据中没有关键帧,则根据所述查询的关键帧的时间戳,选择当前时间戳之前的时间戳对应的关键帧作为所述目标关键帧;If it is detected that there is no key frame in the playback data of the current playback buffer, then according to the time stamp of the key frame of the query, the key frame corresponding to the time stamp before the current time stamp is selected as the target key frame; 若检测到所述当前播放缓冲区的播放数据中有关键帧,则根据当前下载速率估算从上一个关键帧到当前播放位置所需的下载时长,并估算从所述当前播放位置到下一个关键帧所需的播放时长;其中,所述上一个关键帧为根据所述查询的关键帧的时间戳,选择的当前时间戳之前的时间戳对应的关键帧,所述下一个关键帧为所述当前播放缓冲区中的关键帧;If it is detected that there is a key frame in the playback data of the current playback buffer, estimate the download time required from the previous key frame to the current playback position according to the current download rate, and estimate the duration from the current playback position to the next key frame The playback duration required for the frame; wherein, the last key frame is the key frame corresponding to the timestamp before the selected current timestamp according to the timestamp of the key frame of the query, and the next key frame is the keyframes in the current playback buffer; 若所述下载时长小于所述播放时长,则将所述上一个关键帧确定为所述目标关键帧;If the download duration is less than the playback duration, determining the last key frame as the target key frame; 若所述下载时长大于所述播放时长,则将所述下一个关键帧确定为所述目标关键帧。If the download duration is greater than the playback duration, the next key frame is determined as the target key frame. 5.根据权利要求1所述的码率切换方法,其特征在于,所述以所述目标关键帧为切换起始位置,进行码率切换,包括:5. The code rate switching method according to claim 1, wherein the performing code rate switching with the target key frame as a switching starting position, comprising: 查询所述目标关键帧所在TS片段的位置偏移量;query the position offset of the TS segment where the target key frame is located; 根据查询的位置偏移量,下载切换后的码率对应的播放数据;其中,所述切换后的码率根据所述触发操作确定。According to the queried position offset, the playback data corresponding to the switched bit rate is downloaded; wherein the switched bit rate is determined according to the trigger operation. 6.根据权利要求1所述的码率切换方法,其特征在于,所述标签信息还包括:各所述关键帧的画面信息。6 . The rate switching method according to claim 1 , wherein the tag information further comprises: picture information of each of the key frames. 7 . 7.一种客户端,其特征在于,包括:7. A client, characterized in that, comprising: 至少一个处理器;以及,at least one processor; and, 与所述至少一个处理器通信连接的存储器;其中,a memory communicatively coupled to the at least one processor; wherein, 所述存储器存储有可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行,以使所述至少一个处理器能够执行如权利要求1至6中任一所述的码率切换方法。the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the execution of any one of claims 1 to 6 rate switching method. 8.一种计算机可读存储介质,存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现权利要求1至6中任一项所述的码率切换方法。8 . A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the code rate switching method according to any one of claims 1 to 6 is implemented. 9 .
CN201911209068.XA 2019-11-30 2019-11-30 Rate switching method, client and computer-readable storage medium Active CN110933517B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911209068.XA CN110933517B (en) 2019-11-30 2019-11-30 Rate switching method, client and computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911209068.XA CN110933517B (en) 2019-11-30 2019-11-30 Rate switching method, client and computer-readable storage medium

Publications (2)

Publication Number Publication Date
CN110933517A CN110933517A (en) 2020-03-27
CN110933517B true CN110933517B (en) 2022-04-29

Family

ID=69848407

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911209068.XA Active CN110933517B (en) 2019-11-30 2019-11-30 Rate switching method, client and computer-readable storage medium

Country Status (1)

Country Link
CN (1) CN110933517B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111526387B (en) * 2020-03-30 2022-09-23 北京达佳互联信息技术有限公司 Video processing method and device, electronic equipment and storage medium
CN112165636B (en) * 2020-09-10 2021-12-14 东北大学 D2D network multimedia stream code rate adjusting method and multimedia stream real-time transmission system
CN112637661B (en) * 2020-12-22 2022-08-05 广州博冠信息科技有限公司 Video stream switching method and device, computer storage medium and electronic equipment
CN113438513B (en) * 2021-06-25 2023-05-02 深圳Tcl新技术有限公司 Video resolution switching method, device, equipment and storage medium
CN116150428B (en) * 2021-11-16 2024-06-07 腾讯科技(深圳)有限公司 Video tag acquisition method and device, electronic equipment and storage medium
CN114125576A (en) * 2021-11-29 2022-03-01 广州繁星互娱信息科技有限公司 Synchronization method and device for multimedia resources, storage medium and electronic device
CN114339291B (en) * 2021-12-31 2024-05-28 杭州当虹科技股份有限公司 Method for scaling and playing streaming media video
CN115002544A (en) * 2022-06-17 2022-09-02 中国电信股份有限公司 Video playing method and device, nonvolatile storage medium and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101778104A (en) * 2009-12-29 2010-07-14 常州中流电子科技有限公司 System and method for playing stream media by using self-adaption bandwidth
CN102957672A (en) * 2011-08-25 2013-03-06 中国电信股份有限公司 Method, client side and system for self-adaptive playing of FLV (flash video) media streams
CN108737862A (en) * 2018-05-23 2018-11-02 浙江大华技术股份有限公司 A kind of method and apparatus generating video code flow segment
CN109862403A (en) * 2019-02-19 2019-06-07 未来电视有限公司 Self-adaption code rate switching method, device, electronic equipment and storage medium

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102256173A (en) * 2011-07-30 2011-11-23 冠捷显示科技(厦门)有限公司 PVR (Personal Video Record)-based television program playback marking method, playback mark using method and playback mark deletion method
CN105979404A (en) * 2015-12-01 2016-09-28 乐视致新电子科技(天津)有限公司 Streaming media processing method and device
CN107438202A (en) * 2016-05-26 2017-12-05 中兴通讯股份有限公司 A kind of method and device for realizing code check switching
CN107071529B (en) * 2017-03-29 2018-10-26 咪咕视讯科技有限公司 A kind of HLS video broadcasting methods, terminal and server
US10659514B2 (en) * 2018-03-28 2020-05-19 Arlo Technologies, Inc. System for video monitoring with adaptive bitrate to sustain image quality
CN109168078B (en) * 2018-09-10 2022-07-12 苏宁智能终端有限公司 Video definition switching method and device
CN110493324A (en) * 2019-07-29 2019-11-22 咪咕视讯科技有限公司 Method for down loading, downloader and computer readable storage medium
CN110446072B (en) * 2019-08-14 2021-11-23 咪咕视讯科技有限公司 Video stream switching method, electronic device and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101778104A (en) * 2009-12-29 2010-07-14 常州中流电子科技有限公司 System and method for playing stream media by using self-adaption bandwidth
CN102957672A (en) * 2011-08-25 2013-03-06 中国电信股份有限公司 Method, client side and system for self-adaptive playing of FLV (flash video) media streams
CN108737862A (en) * 2018-05-23 2018-11-02 浙江大华技术股份有限公司 A kind of method and apparatus generating video code flow segment
CN109862403A (en) * 2019-02-19 2019-06-07 未来电视有限公司 Self-adaption code rate switching method, device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN110933517A (en) 2020-03-27

Similar Documents

Publication Publication Date Title
CN110933517B (en) Rate switching method, client and computer-readable storage medium
US8510375B2 (en) Apparatus and methods for time mapping media segments in streaming media files
TWI470983B (en) Method and apparatus for updating http content descriptions
US20170149860A1 (en) Partial prefetching of indexed content
CN102647634A (en) A method and device for playing multi-slice video based on HTML5video
CN110636346A (en) Code rate self-adaptive switching method and device, electronic equipment and storage medium
CN113141522B (en) Resource transmission method, device, computer equipment and storage medium
KR20040098119A (en) Http-based video streaming apparatus and method for a mobile communication system
EP3047627B1 (en) Dash representations adaptations in network
WO2020155959A1 (en) Definition switching method and apparatus, computer device, and readable storage medium
US20210320957A1 (en) Method of requesting video, computing device, and computer-program product
WO2011054319A1 (en) Method, device and system for realizing hierarchically requesting content in http streaming system
WO2021017958A1 (en) Video transcoding method and apparatus
US20150268808A1 (en) Method, Device and System for Multi-Speed Playing
KR102635342B1 (en) Session-based information for dynamic adaptive streaming over HTTP
CN107690093B (en) Video playing method and device
CN108702542A (en) Client operation method for streaming service
CN115834925B (en) Video transcoding method, device, equipment and medium
KR102212973B1 (en) Method for providing a content part of a multimedia content to a client terminal, corresponding cache
TW201501526A (en) Method for providing a content part of a multimedia content to a client terminal, corresponding cache
KR102237900B1 (en) Method for retrieving, by a client terminal, a content part of a multimedia content
CN116527992A (en) Video playing resolution switching method and device
CN106936808A (en) HTTP flow-medium transmission methods and device
CN114501166A (en) DASH on-demand fast-forward and fast-backward method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant