CN102045312B - Self-adaptative transcoding method and system - Google Patents

Abstract

The invention provides an adaptive transcoding method and system. The adaptive transcoding method includes: when the media player does not support the format of the local multimedia file, triggering the media server; the media server determines whether the transcoding module supports transcoding, and when determining that the transcoding module supports transcoding, notifies the media The player uploads the multimedia file; when the media player uploads the multimedia file to the media server, the media server starts the transcoding module to read the uploaded multimedia file, so that the transcoding module performs the multimedia file with a predetermined transcoding strategy Transcoding; the media server receives the transcoded data from the transcoding module, and sends the received transcoded data to the media player for playback; the media server detects the network performance parameters of the media player during playback and the player own performance parameters, and adjust the transcoding strategy in real time according to the performance parameters.

Description

Adaptive transcoding method and system

technical field

The present invention relates to an adaptive transcoding method and system, more specifically, to an automatic adaptive transcoding method and system based on a digital home network media player.

Background technique

The digital home network extends the public network functions and applications to the home through the home gateway, and connects various information terminals (such as household appliances, PCs, etc.) through wired or wireless networks to provide functions such as voice, data, multimedia, control and management. , to achieve full circulation and sharing of information between internal terminals in the home and with external public networks. From the birth of the digital home concept in 1994 to the official launch of the digital home market, from the launch of the "Venus" project to the establishment of the UPnP forum and the DLNA digital home organization, it has been more than ten years of development, and the digital life has come a long way. into real development. As the dream of a digital home network becomes clearer, the question of standards comes along.

In June 2003, 17 consumer electronics industry giants including Intel, Philips, IBM, HP, Microsoft and Fujitsu established the "Digital Home Work Group" (DHWG: Digital Home Work Group) to jointly develop and establish a A platform with the ability to work together enables manufacturers to develop digital home products that can share content through wired or wireless Internet networks. The IGRS standard was officially approved as a national recommended industry standard on June 29, 2005, becoming China's first "3C collaborative industry technology standard". Generally speaking, the scope of DLNA is limited to the interior of the home, and its purpose is to realize the intelligent connection of home appliances, computers, and communication equipment. The main difference between IGRS and DLNA is: the IGRS standard not only realizes the intelligent interconnection between devices, but more importantly, realizes the resource sharing and collaborative services of the devices; in terms of the scope of work, the scope of the IGRS standard is not limited to the family, including the application in the enterprise And the collaborative application of relevant resources in the society.

For the above standards, it can be divided into three parts according to the level: device detection and connection, media content management, and media format and transmission. The transcoding function falls under the category of "media format and transmission". IGRS is implemented by a gateway, and DLNA is implemented by an MIU (Media Interoperation Unit). Both the gateway and MIU can realize real-time transcoding, and can automatically adjust the transcoding scheme according to the configuration information. However, no matter IGRS or DLNA, what is implemented is the transcoding of the media stream in the process of being transmitted from the media server to the media player. By default, the multimedia files are stored in the media server. Therefore, the existing transcoding scheme does not take into account the problem that the media player needs to transcode the multimedia file to play the multimedia file when the multimedia file exists on the media player side. In addition, the prior art only provides several fixed transcoding schemes, but there is no transcoding scheme that can be adjusted in real time according to the performance of the media player and network conditions.

Contents of the invention

Additional aspects and/or advantages of the invention will be set forth in part in the description which follows, and will be apparent from the description, or may be learned by practice of the invention.

According to one aspect of the present invention, a method for adaptive transcoding is provided, comprising the following steps: when the media player does not support the format of the local multimedia file, trigger the media server; the media server determines whether the transcoding module supports transcoding, and when When it is determined that the transcoding module supports transcoding, notify the media player to upload the multimedia file; when the media player uploads the multimedia file to the media server, the media server starts the transcoding module to read the uploaded multimedia file, thereby transcoding The module transcodes multimedia files with a predetermined transcoding strategy; the media server receives the transcoded data from the transcoding module, and sends the received transcoded data to the media player for playback; the media server detects the media During the playback process of the player, the network performance parameters and the performance parameters of the player itself, and adjust the transcoding strategy in real time according to the performance parameters.

According to another aspect of the present invention, an adaptive transcoding system is provided, including: a media player, triggering the media server when the format of the local multimedia file is not supported, and uploading the multimedia file according to the notification of the media server; the media server, Determine whether the transcoding module supports transcoding, and when it is determined that the transcoding module supports transcoding, notify the media player to upload the multimedia file; the transcoding module, when the media player uploads the multimedia file to the media server, Read the uploaded multimedia file, transcode the multimedia file with a predetermined transcoding strategy, and send the transcoded data to the media server, wherein the media player receives the transcoded data from the media server for playback, The media server detects the network performance parameters during the playback of the media player and the performance parameters of the player itself, and adjusts the transcoding strategy in real time according to the performance parameters.

Description of drawings

These and/or other aspects and advantages of the present invention will become clearer and easier to understand through the following description of embodiments in conjunction with the accompanying drawings, wherein:

FIG. 1 shows a block diagram of an adaptive transcoding system according to an embodiment of the present invention;

Fig. 2 shows a flowchart of an adaptive transcoding method according to an embodiment of the present invention.

Detailed ways

Embodiments of the invention will now be described in detail, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like parts throughout. The embodiments are described below in order to explain the present invention by referring to the figures.

Fig. 1 shows a block diagram of an adaptive transcoding system according to an embodiment of the present invention. Referring to FIG. 1 , the adaptive transcoding system includes a media server, a media player, and a transcoding module, wherein the transcoding module can be set in the media server, or set separately from the media server. The media server, media player, and transcoding module also contain multiple functional modules to cooperate to complete specific functions, such as uploading, transcoding, and downloading.

Specifically, the media server includes: device detection and connection module, content directory service module, upload control module, streaming media service module, XML text analysis module, transcoding dynamic configuration module, network and player performance detection module, FIFO (advanced first out) control module. Among them, the device detection and connection module, the content directory service module, the upload control module, the streaming media service module and the XML text analysis module are already defined in the digital home network protocol.

The device detection and connection module detects the connection status of the media server and the devices in the home network, and realizes the physical connection with the devices in the home network. The content catalog service module provides the catalog service of playable content (video, audio, picture, etc.) to the media player. The XML text parsing module is used for parsing the configuration files of the transcoder side and the media player side, and provides the information of the parsed configuration files to the transcoding dynamic configuration module. The functions implemented by the above three modules are well known to those skilled in the art, so the above three modules are omitted in FIG. 1 .

The upload control module controls the multimedia file data uploaded by the media player, and is used to control the rate to match the transcoding speed of the transcoding module.

The FIFO control module provides a first-in-first-out mechanism of data flow, so as to facilitate flow control of streaming media data. The FIFO control module includes two FIFO files, one is used to store the FIFO input file in the original format received from the media player (that is, the multimedia file data uploaded by the media player), and the other reads the FIFO output file from the transcoder , that is, the multimedia data transcoded by the transcoder. According to an embodiment of the present invention, the FIFO control module can set a capacity limit to control flow. For example, when the FIFO input file reaches the capacity limit, the upload control module requests the media player to stop uploading multimedia file data according to the error message sent by the FIFO control module. When the FIFO input file is idle after the FIFO input file has been read by the transcoding module, the upload control module requests the media player to continue uploading multimedia file data according to the idle message sent by the FIFO control module.

The streaming media service module receives the FIFO output file (that is, transcoded multimedia data) from the FIFO control module, and transmits the transcoded multimedia data to the media player.

The network and player performance detection module detects network performance parameters and player performance parameters in real time. Network performance parameters include current bandwidth, delivery rate of IP packets, and so on. The performance parameters of the player itself include the current CPU usage, the current frame rate, and so on. The network performance parameters and the performance parameters of the player itself are updated at regular intervals (for example, every 5 seconds). The network and player performance detection module obtains the performance parameters of the network and the player itself in real time, and transmits these performance parameters to the transcoding dynamic configuration module.

The transcoding dynamic configuration module can obtain the following information: the performance parameters of the network and the player itself, the function description file of the transcoder and the configuration file of the media player. Wherein, the configuration file at the media player side includes a player hardware function description file and a player performance description file, which will be described later in conjunction with the media player. The transcoding dynamic configuration module uses the transcoder function description file and the media player configuration file to roughly configure the transcoding strategy of the transcoding module, and uses the performance parameters of the network and the player itself to adjust the transcoding strategy in real time, so that the transcoding encoding and playback performance to achieve the best.

The media player includes an upload module, a media control module, a media player module, a player hardware function description file and a player performance description file. Among them, the upload module, media control module and media player module are already defined in the digital home network protocol, and the player hardware function description file and the player performance description file form the above media player configuration file. The media player side configuration file may be stored in the local memory of the media player. In addition, multimedia files placed in a media player are shown in FIG. 1 .

The media player module is used to play multimedia files. Specifically, the media player module is a multimedia player with multiple input sources, supporting the playback of audio, video, and pictures. Under normal circumstances, if the local hardware decoder of the media player can decode the multimedia file, the playback module directly plays the multimedia file; if the local hardware decoder does not support the decoding of the multimedia file and the transcoding module supports transcoding, the media The playback module turns the data source to the input of the media server, and receives the data (that is, transcoded multimedia files) provided by the streaming service module in the media server, so as to realize real-time streaming media playback.

The media control module controls the upload module and the media player module, and monitors player performance information. The media control module monitors the format of the multimedia file played by the media player module, and triggers the upload module when it finds that the format of the multimedia file is not supported by the media player module. When the upload module is triggered by the media control module, the multimedia file is uploaded to the upload control module in the media server. The upload module can upload multimedia files according to the HTTP protocol under the control of the media control module. In addition, the media control module can also send the media player configuration file to the transcoding dynamic configuration module in the media server. On the other hand, during the transcoding by the transcoding module, the media control module sends the player performance information to the network and player performance detection module in real time according to the request of the network and player performance detection module in the media server.

The player hardware function description file includes two aspects, one is the function description of the hardware decoder, and the other is the hardware description of the player, such as screen size, resolution, memory size, number of channels, etc. The player performance description file includes the capability of the processor, the capability of the hardware decoder, and the capacity of the display memory allocated to the decoder, etc.

The transcoding module includes the transcoder provided by any third party and the function description file of the transcoder. The most common transcoder is ffmpeg. The transcoder function description file defines the transcoding function supported by the transcoder, for example, the transcoding function of converting a wmv file into an mpeg4 file. During operation, the transcoding module can be an independent process, or an independent hardware transcoding device.

Specifically, the transcoding module may be a third-party transcoding software library, or a hardware device on the network. If the transcoding module is a hardware device on the network, a network communication module needs to be set between the transcoding module and the media server to realize the transmission of data and control information between the two. The transcoding module itself is dynamically configurable, involving bit rate conversion, output resolution conversion, color space conversion, etc.

According to the embodiment of the present invention, the transcoding dynamic configuration module in the media server first extracts the transcoder function description file, determines the range of formats that can be transcoded, and establishes a transcoding rule base. Then, the transcoding dynamic configuration module extracts the configuration file of the media player, matches in sequence and inquires whether the transcoding module supports converting the original format to the format supported by the player, and selects according to the nearest matching principle. Due to the instability of network and player performance, the transcoding dynamic configuration module can use the information provided by the network and player performance detection module to adjust the transcoding action in real time, so as to achieve the optimal playback effect on the player side.

The adaptive transcoding method performed by the adaptive transcoding system is described below with reference to FIG. 2 .

Fig. 2 shows a flowchart of an adaptive transcoding method according to an embodiment of the present invention. Assume that the media player and the media server have established a connection at this point.

With reference to Fig. 2, in step S201, when the media player module in the media player is playing local multimedia file, if the media control module finds that the format of this multimedia file is not supported by the media player module, then read the media in the local memory Player-side configuration file, and send the read media player-side configuration file to the media server side. At the same time, the media control module can trigger the transcoding dynamic configuration module in the media server.

An example of the player hardware function description file in the media player configuration file is described by XML text as follows:

<profile name="supported file format">

<mimetype>audio/x-wav</mimetype>

<mimetype>audio/midi</mimetype>

<mimetype>audio/mp3</mimetype>

<mimetype>video/mp4</mimetype>

<mimetype>video/3gp</mimetype>

<mimetype>video/x-mp4</mimetype>

<mimetype>image/jpeg</mimetype>

<mimetype>image/gif</mimetype>

<mimetype>image/png</mimetype>

</profile>

<profile name="player hardware infomation">

<pixel>720x576</pixel>

<memory>128</memory>

<channel>2</channel>

</profile>

According to the player hardware function description file shown above, the audio, video, and picture formats supported by the media player can be determined, as well as the hardware information of the media player, such as resolution, memory size, and number of channels.

In step S202, after the transcoding dynamic configuration module in the media server receives the configuration file of the media player, it invokes the XML text analysis module to resolve the configuration file of the media player into metadata (for example, in the player hardware function description file defined audio/x-wav, video/mp4, image/png, etc.), and analyze the transcoder function description file at the same time, and use the parsed information to determine whether the transcoding module supports transcoding. Here, the transcoding dynamic configuration module first determines the supported transcoding types defined in the transcoder function description file, and then determines whether the formats not supported by the current media player can be converted to the ones listed in the player configuration file. any kind of file. If it is determined that the transcoding module supports transcoding, the transcoding dynamic configuration module notifies the media player to start uploading the multimedia file, and at the same time redirects the data source of the media player from the local multimedia file to the streaming media service of the media server.

An example of the text of the transcoder function description file is described by the XML text as follows:

<profile name="transcode function">

<transcode from="audio/x-ms-wma"to="audio/mp3"/>

<transcode from="video/x-ms-wmv"to="video/mp4"/>

<transcode from=”image/tif”to=”image/jpeg”/>

<codec command="ffmpeg" arguments="-d%format-f%out-f%in">

</profile>

According to the transcoder function description file shown above, a series of transcoding rules and default transcoding parameters are defined. Here, the transcoding rule refers to conversion from one encoding format to another encoding format, and the transcoding parameter refers to a parameter to be transmitted to the transcoding module for adjusting the running state of the transcoding module. Specifically, the transcoding parameters may include resolution, frame rate, video bit rate, audio bit rate, and the like. According to the embodiment of the present invention, the transcoding module supports dynamic adjustment of its operating parameters.

In step S203, the upload module in the media player transmits the multimedia file data to the media server. Specifically, when the transcoding dynamic configuration module determines that the transcoding module supports transcoding, the upload control module sends an HTTP-Get request to the media player, and the media player that receives the request controls the upload module to upload the multimedia file through the media control module. The data is transmitted to the media server, and the upload control module of the media server transmits the received multimedia file data to the FIFO input file of the FIFO control module.

In step S204, the transcoding dynamic configuration module in the media server activates the transcoding module, so that the transcoding module reads the uploaded multimedia file and decodes the multimedia file data in real time with a predetermined transcoding strategy. Here, the transcoding module can be initialized from the parsed transcoder function description file and media player configuration file in the mode of a separate process or in the mode of a separate hardware module (ie, configure the transcoding strategy of the transcoding module).

In step S205, after the transcoding module performs real-time transcoding, the transcoded data is sent to the FIFO output file of the media server.

In step S206, the streaming media service module in the media server reads the data in the FIFO output file, and then sends the read data to the media player in the form of HTTP protocol for playing.

In step S207, the playing module in the media player receives the data transmitted by the streaming media service module, calls the local hardware decoder to decode the data, and displays the decoded data on the display screen.

In step S208, the network and player performance detection module in the media server detects the performance parameters of the network and the player itself during the playback of the media player with a fixed frequency (for example, 5 seconds/time), and the detected parameters The transcoding dynamic configuration module is transmitted to the transcoding dynamic configuration module, so that the transcoding dynamic configuration module can adjust the transcoding strategy of the transcoding module in real time according to the performance parameters. That is, the transcoding dynamic configuration module can adjust the transcoding parameters in real time according to the performance parameters, so as to realize dynamic configuration of the transcoding module.

Although in the above description, the transmission of the multimedia file data is based on the HTTP protocol, the present invention is not limited thereto, and the multimedia file data can also be transmitted based on the FTP protocol.

The adaptive transcoding method and system according to the present invention utilize the existing functions provided by the digital home network, and have the advantages of low cost, high flexibility, configurability and the like. In addition, the adaptive transcoding method and system according to the present invention provide a concept of media format independence for digital devices in the digital home network environment, so that only one powerful transcoding server needs to be provided in the entire digital home network environment, Therefore, the problem of weak decoding ability of multiple digital devices can be solved.

While the present invention has been shown and described with reference to embodiments thereof, it will be understood by those skilled in the art that changes in form and details may be made without departing from the spirit and scope of the invention as defined by the claims and their equivalents. Make various changes.

Claims (17)

1. A method for adaptive transcoding, comprising the following steps: When the media player does not support the format of the local multimedia file, trigger the media server; The media server determines whether the transcoding module supports transcoding, and when it is determined that the transcoding module supports transcoding, notifies the media player to upload the multimedia file; When the media player uploads the multimedia file to the media server, the media server starts the transcoding module to read the uploaded multimedia file, so that the transcoding module transcodes the multimedia file with a predetermined transcoding strategy; The media server receives the transcoded data from the transcoding module, and sends the received transcoded data to the media player for playback; The media server detects the network performance parameters and the performance parameters of the player itself during the playback of the media player, and adjusts the transcoding strategy in real time according to the network performance parameters and the performance parameters of the player itself. 2. The adaptive transcoding method according to claim 1, wherein the step of triggering the media server comprises: When the media player does not support the format of the local multimedia file, read the media player configuration file in the local storage; Send the read media player configuration file to the media server; Trigger the media server. 3. The adaptive transcoding method according to claim 2, wherein the media server analyzes the received media player configuration file and the transcoder function description file provided by the transcoding module to determine whether the transcoding module supports transcoding code. 4. The adaptive transcoding method according to claim 2, wherein the media server determines the predetermined transcoding strategy according to the parsed media player configuration file and the transcoder function description file. 5. The adaptive transcoding method according to claim 1, wherein the media server detects the network performance parameters and the performance parameters of the media player itself during the playback of the media player at a fixed frequency. 6. The adaptive transcoding method according to claim 2, wherein the media player configuration file includes a player hardware function description file and a player performance description file, and the player hardware function description file includes a hardware decoder A function description and a hardware description of the player. The player performance description file includes the capability of the processor, the capability of the hardware decoder, and the capacity of the display memory allocated to the decoder. 7. The adaptive transcoding method according to claim 3, wherein the transcoder function description file defines the transcoding functions supported by the transcoding module. 8. The adaptive transcoding method according to claim 1, wherein the network performance parameters include the current bandwidth and the delivery rate of IP packets, and the performance parameters of the player itself include the current CPU occupancy rate and the current frame rate. 9. An adaptive transcoding system, comprising: The media player triggers the media server when the format of the local multimedia file is not supported, and uploads the multimedia file according to the notification of the media server; The media server determines whether the transcoding module supports transcoding, and when it is determined that the transcoding module supports transcoding, notifies the media player to upload the multimedia file; A transcoding module, when the media player uploads the multimedia file to the media server, reads the uploaded multimedia file, transcodes the multimedia file with a predetermined transcoding strategy, and sends the transcoded data to the media server , Wherein, the media player receives the transcoded data from the media server to play, and the media server detects the network performance parameters and the performance parameters of the player itself during the playback of the media player, and according to the network performance parameters and the player itself Real-time adjustments to the transcoding strategy based on performance parameters. 10. The adaptive transcoding system according to claim 9, wherein the media player comprises an upload module, a media control module, a media player module and a media player end configuration file, Wherein, when the media playing module does not support the format of the local multimedia file, the media control module sends the configuration file of the media player to the media server, and controls the uploading module to upload the multimedia file according to the notification of the media server. 11. The adaptive transcoding system according to claim 9, wherein the media server comprises: The device detection and connection module detects the connection status between the media server and the media player, and realizes the physical connection with the media player; The content directory service module provides the media player with the directory service of the played content; The upload control module controls the multimedia files uploaded by the media player; The XML text parsing module parses the media player configuration file provided by the media server and the transcoder function description file provided by the transcoding module; The transcoding dynamic configuration module determines the predetermined transcoding strategy according to the parsed media player configuration file and the transcoder function description file; The first-in-first-out control module is used to store the multimedia files uploaded by the media player and the data transcoded by the transcoding module; The streaming media service module receives the transcoded data and sends it to the media player; Network and player performance detection module detects network performance parameters and the performance parameters of the player itself during the playback of the media player, and provides the network performance parameters and the performance parameters of the player itself to the transcoding dynamic configuration module The transcoding strategy is adjusted in real time. 12. The adaptive transcoding system according to claim 9, wherein the transcoding module comprises: a transcoder for performing a transcoding function; a transcoder function description file defining the transcoding functions supported by the transcoder . 13. The adaptive transcoding system according to claim 11, wherein the network and player performance detection module detects the network performance parameters and the performance parameters of the player itself during the playback of the media player at a fixed frequency. 14. The adaptive transcoding system according to claim 10 or 11, wherein the media player configuration file includes a player hardware function description file and a player performance description file, and the player hardware function description file includes a hardware decoding The player's function description and the player's hardware description, the player performance description file includes the capability of the processor, the capability of the hardware decoder and the capacity of the display memory allocated to the decoder. 15. The adaptive transcoding system according to claim 9, wherein the network performance parameters include current bandwidth and delivery rate of IP packets, and the performance parameters of the player itself include current CPU usage and current frame rate. 16. The adaptive transcoding system according to claim 9, wherein the transcoding module is set in the media server. 17. The adaptive transcoding system according to claim 11, wherein the first-in-first-out control module is provided with a capacity limit for controlling flow, Wherein, when the uploaded multimedia file reaches the capacity limit, the upload control module requests the media player to stop uploading the multimedia file according to the error message sent by the first-in first-out control module, and the first-in first-out When the control module is idle, the upload control module requests the media player to continue uploading multimedia files according to the idle message sent by the first-in-first-out control module.

Images