CN105812922A - Multimedia file data processing method, system, player and client - Google Patents

Abstract

The present invention provides a method and system for processing multimedia file data, a player and a client, wherein the method includes: when the player builds a graphics filter, calling the first filter and the second filter preset by the system according to preset rules Two filters; the first filter transparently transmits the received media file data sent by the source filter to the second filter; the second filter sends the media file data to the client. The present invention solves the problems of unsmooth and unclear video playback and high CPU usage of the virtual machine during video playback in the related art. In addition, the player does not decode and render media files in the present invention. , saving CPU resources.

Description

Multimedia file data processing method and system, player and client

technical field

The present invention relates to the communication field, in particular to a multimedia file data processing method and system, a player and a client.

Background technique

When using the traditional in-band video solution to play video in the cloud desktop, the video file is decoded in the virtual machine, and then the decoded audio and video data is sent to the The client renders. Decoding on the virtual machine side will additionally occupy the CPU usage of the virtual machine, and also reduce the number of virtual machines that can be run on the same host; at the same time, the amount of decoded data is dozens or even hundreds of times the amount of undecoded data, which occupies a large amount of network Bandwidth affects the efficiency of data transmission between virtual machines and desktops.

Figure 1 is a schematic diagram of the Windows player architecture in the cloud desktop in the related art, using multimedia players in the cloud desktop, such as WindowsMediaPlayer, Baofengyingyin, to play multimedia files; as shown in Figure 1, in the Windows virtual machine, the media files are read Fetching, decoding, rendering and playing, the decoded multimedia data is transmitted to the client through the Spice channel, and finally presented on the client. At present, the core components of the player on the Windows platform are based on DirectShow, and Windows versions after Vista are based on the MediaFoundation architecture by default, and different plug-ins complete data reading, decompression and presentation. On this basis, different player applications realize personalized UI and integrate different video portals.

FIG. 2 is a schematic diagram of the architecture of the streaming media development kit in the related art. As shown in FIG. 2 , the architecture of the streaming media development kit DirectShow is mainly divided into a graphic filter management device FilterGraphManager and three types of filters. FilterGraphManager is used to manage the processing of the entire data stream, and manage functions such as the start, pause, and stop of the playback process. Filters are divided into three categories according to their functions: SourceFilters are used to obtain data, and the data source can be files, digital cameras of acquisition cards in Internet computers, etc.; TransformFilters are used for data format conversion, such as data stream separation/synthesis, decoding/encoding, etc.; RenderingFilters The final destination of the language data is to transfer the data to the graphics card, the sound card for multimedia presentation, or output to a file for storage.

It can be seen that in the Spice video solution in the related art, in order to save network bandwidth, the SpiceServer needs to perform H.264 compression on the video played in the virtual machine, and decoding and compression will cause high CPU usage of the server, affecting the number of virtual machines supported by the server. Moreover, video files are decoded in the virtual machine, and now various video encodings have a high compression rate, but at the same time the compression process and decompression are becoming more and more complicated, resulting in a large amount of CPU resources being occupied when the video file is decoded in the virtual machine. In addition, the amount of multimedia data after decoding is very large. When these data are transmitted in the Spice channel, a large amount of bandwidth is taken up, which seriously affects the communication of other Spice channels; The video playback is not smooth and clear, and often freezes. At the same time, the CPU usage of the virtual machine is too high, which directly causes the entire desktop to freeze.

Aiming at the problems that the video playback is not smooth and clear and the CPU usage rate of the virtual machine is too high when the video is played in the related technology, no effective solution has been proposed yet.

Contents of the invention

The main purpose of the present invention is to provide a processing method, system, player and client of multimedia file data, so as to at least solve the problem of unsmooth and unclear video playback and virtual machine CPU usage during video playback in related technologies. The problem of high rates.

According to one aspect of the present invention, a method for processing multimedia file data is provided, including: calling the first filter and the second filter preset by the system according to preset rules when the player builds a graphic filter; The first filter transparently transmits the received media file data sent by the source filter to the second filter; the second filter sends the media file data to the client.

Further, when the system is Windowsxp, the first filter is an empty decoding filter, the second filter is a remote rendering filter, and the first filter and the second filter preset by the system are invoked according to preset rules. The second filter includes: receiving an instruction triggered by the user, calling the empty decoding filter and the remote rendering filter according to the instruction; or calling the empty decoding filter and the remote rendering filter according to the priority preset by the system A remote rendering filter, wherein the null decoding filter and the remote rendering filter have the highest priority.

Further, when the system is Windows7, both the first filter and the second filter are remote rendering filters, and the first filter and the second filter preset by the system are invoked according to preset rules, including : Obtain the function information of the application programming interface; judge whether the function information indicates to call the default filter of the system, and switch from the current default filter of the system to the remote rendering filter when the judgment result is yes .

Further, the second filter sends the media file data to the client through an IP link.

According to another aspect of the present invention, a kind of processing method of multimedia file data is provided, and client receives the media file data that player sends; Described client decodes and renders described media file data; Described client according to After decoding and rendering the media file data, the video corresponding to the media file data is played.

According to still another aspect of the present invention, a player is provided, including: a calling module, configured to call the first filter and the second filter preset by the system according to preset rules when the player constructs a graphic filter; The first filter is configured to transparently transmit the received media file data sent by the source filter to the second filter; the second filter is configured to send the media file data to the client.

Further, when the system is Windowsxp, the first filter is an empty decoding filter, and the second filter is a remote rendering filter; the calling module is used to receive an instruction triggered by a user, and according to the The above instruction calls the empty decoding filter and the remote rendering filter; or, calls the empty decoding filter and the remote rendering filter according to the priority order preset by the system, wherein the empty decoding The filter and the remote rendering filter have the highest priority.

Further, when the system is Windows7, both the first filter and the second filter are remote rendering filters; the calling module includes: an acquisition unit, configured to acquire function information of an application programming interface ; Judging unit, used to judge whether the function information indicates to call the default filter of the system; switching unit, used to switch from the current default filter of the system to the remote rendering filter when the judgment result is yes device.

According to still another aspect of the present invention, a client is provided, including: a receiving module, configured to receive media file data sent by the player; a processing module, configured to decode and render the media file data; a playing module , for playing the video corresponding to the media file data according to the decoded and rendered media file data.

According to yet another aspect of the present invention, a multimedia file data processing system is provided, and the system includes: the above-mentioned player and the client.

Through the present invention, when the player constructs the image filter, the first filter and the second filter are called, and the first filter and the second filter do not convert and render the format of the media file data, but only obtain The received media file data is transparently transmitted to the client, and the format conversion and rendering of the media file are performed on the client. In the above method, the media file is not decoded and rendered in the player, which saves CPU resources.

Description of drawings

The accompanying drawings described here are used to provide a further understanding of the present invention and constitute a part of the application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations to the present invention. In the attached picture:

Fig. 1 is a schematic diagram of the Windows player architecture in the cloud desktop in the related art;

Fig. 2 is a schematic diagram of the architecture of the streaming media development kit in the related art;

FIG. 3 is a first flow chart of a method for processing multimedia file data according to an embodiment of the present invention;

4 is a second flow chart of a method for processing multimedia file data according to an embodiment of the present invention;

Fig. 5 is a structural block diagram of a player according to an embodiment of the present invention;

Fig. 6 is a structural block diagram of a client according to an embodiment of the present invention;

Fig. 7 is a structural block diagram of a processing system for multimedia file data according to an embodiment of the present invention;

FIG. 8 is a block diagram of the overall structure of the system according to an optional embodiment of the present invention;

Fig. 9 is a schematic diagram of video redirection under the Windowsxp system according to an embodiment of the present invention;

Fig. 10 is a schematic diagram of video redirection under Windows 7 according to an embodiment of the present invention.

detailed description

Hereinafter, the present invention will be described in detail with reference to the drawings and examples. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

In this embodiment, a method for processing multimedia file data is provided. FIG. 3 is a flowchart one of a method for processing multimedia file data according to an embodiment of the present invention. As shown in FIG. 3 , the process includes the following steps:

Step S302: When the player builds a graphics filter, call the first filter and the second filter preset by the system according to preset rules;

Step S304: the first filter transparently transmits the received media file data sent by the source filter to the second filter;

Step S306: the second filter sends the media file data to the client.

Through this embodiment, when the player builds an image filter, the first filter and the second filter are called, but the first filter and the second filter do not perform format conversion and rendering on the media file data, but only convert the The obtained media file data is transparently transmitted to the client, and the format conversion and rendering of the media file is performed on the client. Through this embodiment, the media file is not decoded and rendered in the player, which saves CPU resources.

In this embodiment, different systems call the first filter and the second filter in different ways, and the first filter and the second filter are also different in different systems; Take Windows7 as an example for corresponding description;

Application scenario 1: When the system is Windowsxp, the first filter is an empty decoding filter, and the second filter is a remote rendering filter. In the above step S102, the first filter and the second filter preset by the system are called according to the preset rules. Filters can be implemented in the following ways:

Method 1: Receive an instruction triggered by the user, and call the empty decoding filter and the remote rendering filter according to the instruction.

Method 2: call the empty decoding filter and the remote rendering filter according to the priority preset by the system, wherein the empty decoding filter and the remote rendering filter have the highest priority.

Application Scenario 2: When the system is Windows7, both the first filter and the second filter are remote rendering filters, the method of invoking the first filter and the second filter preset by the system according to the preset rules in the above step S102 This can be achieved by:

Step S11: Obtain function information of the application programming interface;

Step S12: Judging whether the function information indicates calling the system default filter, and switching from the current system default filter to the remote rendering filter if the judgment result is yes.

It should be noted that the above two systems are only used for illustration, and other systems that can realize the corresponding functions of the first filter and the second filter are also within the protection scope of the present invention.

Optionally, in an optional implementation manner of this embodiment, the second filter sends the media file data to the client through an IP link, and the IP link is optionally a Socket channel. However, the Socket channel is out-of-band and does not use the Simple Protocol for Independent Computing Environment (Spice for short), thereby effectively saving bandwidth.

Fig. 4 is the second flowchart of the method for processing multimedia file data according to an embodiment of the present invention. As shown in Fig. 4, the method includes:

Step S402: the client receives the media file data sent by the player;

Step S404: the client decodes and renders the media file data;

Step S406: The client plays the video corresponding to the media file data according to the decoded and rendered media file data.

Through this embodiment, the media file data to be played is decoded and rendered on the client side, so that the media file data does not need to be decoded and rendered in the cloud desktop player, saving CPU resources.

In this embodiment, a player and a client are also provided, and the player and the client are used to implement the above embodiments and preferred implementation manners, and those that have already been described will not be repeated. As used below, the term "module" may be a combination of software and/or hardware that realizes a predetermined function.

Fig. 5 is a structural block diagram of a player according to an embodiment of the present invention. As shown in Fig. 5, the player includes:

The calling module 52 is used to call the first filter 54 and the second filter 56 preset by the system according to preset rules when the player builds a graphic filter; the first filter 54 is coupled to the calling module 52 for Transparently transmit the received media file data sent by the source filter to the second filter; the second filter 56 is coupled to the first filter 54 and used for sending the media file data to the client.

In this embodiment, different systems call the first filter and the second filter in different ways, and the first filter and the second filter are also different in different systems; Take Windows7 as an example for corresponding description;

Application scenario 1: When the system is Windowsxp, the first filter is an empty decoding filter, and the second filter is a remote rendering filter;

Optionally, the calling module 52 is configured to receive an instruction triggered by the user, and call the empty decoding filter and the remote rendering filter according to the instruction; or, call the empty decoding filter and the remote rendering filter according to the priority sequence preset by the system, Among them, the empty decoding filter and the remote rendering filter have the highest priority.

Application Scenario 2: When the system is Windows7, both the first filter and the second filter are remote rendering filters;

The calling module 52 may also include: an acquisition unit, used to acquire the function information of the application programming interface; the judgment unit is coupled to the acquisition unit, and used to judge whether the function information indicates the default filter of the calling system; the switching unit is coupled to the judgment unit , used to switch from the current system default filter to the remote rendering filter when the judgment result is yes.

Fig. 6 is a structural block diagram of a client according to an embodiment of the present invention. As shown in Fig. 6, the client includes: a receiving module 62 for receiving the media file data sent by the player; a processing module 64 coupled with the receiving module 62, It is used to decode and render the media file data; the playing module 66 is coupled to the processing module 64, and is used to play the video corresponding to the media file data according to the decoded and rendered media file data.

Fig. 7 is a structural block diagram of a multimedia file data processing system according to an embodiment of the present invention. As shown in Fig. 7, the system includes: a player and a client in the embodiment of the present invention.

The present invention is illustrated below in conjunction with optional embodiments of the present invention;

In this optional embodiment, Detour and remote injection technology are used. The player takes WindowsMediaPlayer as an example. The optional embodiment of the present invention is described in detail in conjunction with FIG. 8, FIG. 9 and FIG. 10. FIG. 8 is an optional implementation according to the present invention As shown in FIG. 8 , the player in the Windows virtual machine in the device includes media files, and the client includes components for decoding and playing multimedia data.

Fig. 9 is the schematic diagram of video redirection under Windowsxp system according to the embodiment of the present invention, as shown in Fig. 9, the video redirection scheme of WindowsMediaPlayer under Windowsxp: register NullFilter and RemoteRendererFilter that we write in system, player is building FilterGraph will call our Filter, thus realizing video redirection. In order to avoid soft decoding in the virtual machine and support various players, the DecoderFilter and RendererFilter in the FilterGraph created by the player must be replaced, as shown in Figure 5, NULLDecoder is a Filter that does not perform actual encoding. It just transparently transmits the data sent by SourceFilter to RemoteRender, and RemoteRender establishes a connection with the client, and sends the undecoded data to the client, which decodes and presents it.

For players using smart connections, you can set the priority Merit value of NULLDecoder and RemoteRendererFilter to a high priority, so that the player automatically selects NULLDecoder and RemoteRendererFilter when constructing the FilterGraph, thereby realizing media data redirection.

Fig. 10 is a schematic diagram of video redirection under Windows 7 according to an embodiment of the present invention, as shown in Fig. 10,

Firstly, in the process of constructing the FilterGraph by the media player, modify the Filter selected by the Windows 7 system according to the needs of the user. By intercepting the API of WindowsDirectShow, judge in the hook function: if it is Decoder/Render, replace it with RemoteRender, and the RemoteRender is in the hook function. After receiving the media data, the video will not be decoded and rendered, but the media data will be directly sent to the client through the Socket.

Through this optional embodiment, when playing video on the desktop of the virtual machine, the video encoding of the virtual machine is extracted and separated from the desktop protocol. The video is not decoded in the virtual machine but is directly transmitted to the client for decoding and playback, which reduces the CPU load. usage rate. Moreover, the transmission is through the Socket, that is, it is out-of-band, and does not need to pass through the Spice channel, which effectively saves bandwidth. It should be noted that the player after video redirection can support multiple formats such as AVI, FLV, RM, RMVB.

The English abbreviations involved in the embodiments of the present invention are explained in the following table:

The above are only optional embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. A processing method for multimedia file data, characterized in that, comprising: When the player constructs a graphics filter, it invokes the first filter and the second filter preset by the system according to the preset rules; The first filter transparently transmits the received media file data sent by the source filter to the second filter; The second filter sends the media file data to the client. 2. The method according to claim 1, wherein, when the system is Windowsxp, the first filter is an empty decoding filter, and the second filter is a remote rendering filter, according to preset The first filter and second filter preset by the rule call system include: receiving an instruction triggered by a user, calling the empty decoding filter and the remote rendering filter according to the instruction; or calling the empty decoding filter and the remote rendering filter according to the priority preset by the system, Among them, the empty decoding filter and the remote rendering filter have the highest priority. 3. The method according to claim 1, wherein, when the system is Windows7, both the first filter and the second filter are remote rendering filters, and the system presets are invoked according to preset rules. The set first filter and second filter include: Obtain function information of the application programming interface; Judging whether the function information indicates calling the system default filter, and switching from the current system default filter to the remote rendering filter when the judgment result is yes. 4. The method according to claim 1, wherein the second filter sends the media file data to the client through an IP link. 5. A processing method for multimedia file data, characterized in that, The client receives the media file data sent by the player; The client decodes and renders the media file data; The client plays the video corresponding to the media file data according to the decoded and rendered media file data. 6. A player, characterized in that, comprising: The calling module is used to call the first filter and the second filter preset by the system according to the preset rules when the player constructs the graphic filter; The first filter is configured to transparently transmit the received media file data sent by the source filter to the second filter; The second filter is configured to send the media file data to the client. 7. The player according to claim 6, wherein, when the system is Windowsxp, the first filter is an empty decoding filter, and the second filter is a remote rendering filter; The calling module is configured to receive an instruction triggered by a user, and call the empty decoding filter and the remote rendering filter according to the instruction; or call the empty decoding filter according to the priority sequence preset by the system decoder and the remote rendering filter, wherein the null decoding filter and the remote rendering filter have the highest priority. 8. The player according to claim 6, wherein, when the system is Windows7, the first filter and the second filter are all remote rendering filters; the calling module includes: an acquisition unit, configured to acquire function information of the application programming interface; a judging unit, configured to judge whether the function information indicates calling a default filter of the system; The switching unit is configured to switch from the current system default filter to the remote rendering filter when the judging result is yes. 9. A client, characterized in that, comprising: The receiving module is used to receive the media file data sent by the player; A processing module, configured to decode and render the media file data; The playing module is used to play the video corresponding to the media file data according to the decoded and rendered media file data. 10. A multimedia file data processing system, characterized in that the system comprises: the player according to any one of claims 6 to 8 and the client according to claim 9.