HK1169767A - Method and apparatus for distributing a multimedia content - Google Patents

Description

Method and apparatus for distributing multimedia content

Technical Field

The present invention relates to the field of data communication, and more particularly to a method and apparatus for distributing multimedia content to at least two rendering devices.

Background

Multimedia is a medium that contains a set of content that uses different content forms including text, audio, still images, animation, video, and so on. Rich Media (RM) is a synonym for interactive multimedia, i.e. rich media is a dynamic, interactive set of multimedia data.

Currently, the standards associated with rich media include the MPEG-4 part 20: light application rendering/simple aggregation format (LASeR/SAF), 3GPP SA4 dynamic and exchange multimedia scene (DIMS), Open Mobile Alliance (OMA) BCAST rich media environment architecture, and so on. Other standards, such as Scalable Vector Graphics (SVG) and Synchronized Multimedia Integration Language (SMIL), defined by W3C (world wide web consortium), can support the editing of rich media presentations that integrate audio and video streams with images, text, and other types of material.

Further, industrial solutions supporting rich media services include Adobe's flash, microsoft's Silverlight, Sun's JavaFX, and Nokia's mobile open rich media environment.

In the current RM application, RM contents containing a plurality of elementary contents (elementary contents) are transmitted to one client device, and all the elementary contents are presented on a presentation space of the client device. Here, the action of "presenting" has a different meaning depending on the content to be presented. For example, it may mean playing audio content; it may mean displaying video content; and it may mean displaying text or still images, etc.

However, a user may wish to present one RM content on multiple presentation spaces (i.e., multiple devices) simultaneously. For example, in a home environment, a user wishes to watch video on a TV screen and simultaneously display information related to the content on a second screen; a TV service provider wants to push interactive contents to a user's mobile phone in addition to a conventional TV program, thereby making interactive services such as voting possible.

Disclosure of Invention

According to an aspect of the present invention, a method for distributing multimedia content to at least two rendering devices is provided, wherein the multimedia content comprises at least two elementary contents. The method comprises the following steps: distributing synchronization information of the multimedia content based on a relationship of the at least two rendering devices and the at least two elementary contents, wherein the synchronization information indicates a synchronization relationship between the at least two elementary contents; receiving a multimedia data stream carrying multimedia content; and distributing the multimedia data stream based on the relationship of the at least two rendering devices and the at least two elementary contents.

According to one aspect of the present invention, a method is provided for enabling the rendering of multimedia content on two or more rendering devices.

It is to be understood that further aspects and advantages of the present invention will be found in the following detailed description of the invention.

Drawings

The accompanying drawings are incorporated in and constitute a part of this application and serve to provide a further understanding of the invention. Which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention. Therefore, the present invention should not be limited to these specific embodiments. In the drawings:

FIG. 1 is a diagram that schematically illustrates a system for rendering RM content on different devices, in accordance with an embodiment of the present invention;

FIG. 2 is a block diagram illustrating a dispensing device according to the present embodiment;

FIG. 3 illustrates one example of a configuration file that exists in XML format according to the present embodiment;

FIG. 4 is a flow chart illustrating a method for rendering a multimedia content on different rendering devices in accordance with the present embodiment;

fig. 5 illustrates a configuration file for the device 1 according to the present embodiment;

FIG. 6 illustrates a configuration file for the device 2 according to the present embodiment;

fig. 7 is a flow chart illustrating distribution of a multimedia data stream carrying multimedia content according to the present embodiment;

FIG. 8 is a flow chart illustrating a method for altering distribution of multimedia content in accordance with the present embodiment;

FIG. 9 illustrates an updated configuration file for multimedia content when a move action is performed, in accordance with the present embodiments;

FIG. 10 shows an updated configuration file for the device 1 when a move action is performed, according to the present embodiment;

FIG. 11 illustrates an updated configuration file for the device 2 when a move action is performed, according to the present embodiment;

FIG. 12 illustrates an updated configuration file for multimedia content when a pause action is performed, in accordance with the present embodiments;

FIG. 13 shows an updated configuration file for the device 1 when a pause action is performed, according to the present embodiment;

FIG. 14 shows an updated configuration file for the device 2 when a pause action is performed, according to the present embodiment;

figure 15 is a diagram illustrating one application of the method of the present invention according to this embodiment.

Detailed Description

Embodiments of the present invention will be described below with reference to the accompanying drawings. In the following description, a detailed description of some known functions and configurations will be omitted for clarity and conciseness.

FIG. 1 is a diagram that schematically illustrates a system for rendering RM content on different devices, in accordance with an embodiment of the present invention. The system of this example includes an interaction server 101, an RM server 102, a distribution device 103, and two client devices 104 and 105.

RM server 102 is used to provide digital multimedia content comprising two or more elementary contents. The multimedia content may exist in one of a group of content forms including: audio, video, animation, still images, text, and the like. An example of digital multimedia content includes a set of interrelated data including video, audio, subtitles in different languages for movies, and related advertisements. In addition, RM server 102 can send the configuration file in conjunction with or before the digital multimedia content at the beginning of the transmission of the digital multimedia content. The profile is used to indicate (1) synchronization information of the base content; and (2) a correspondence between the primary content of the multimedia content and the RM client device, i.e. indicating which primary content is to be sent to which destination device (i.e. RM client device).

The interaction server 101 is arranged to receive the instructions and to control the RM server in accordance with the instructions. The instructions may include instructions for changing a destination device for certain primary content, instructions for playing, continuing to play, or pausing a multimedia content presentation. Further, the functionality of the interaction server and the RM server can be integrated into one single device.

The distributing device 103 is adapted to (1) receive the configuration file from the RM server 102, generate a configuration file for each destination device based on the received configuration file, and send the generated configuration file to the corresponding destination device. Here, the generated profile of one destination device may include synchronization information and control information corresponding to the primary content to be transmitted to the destination device; (2) receiving a multimedia data stream carrying digital multimedia content from RM server 102, the digital multimedia content comprising a plurality of mutually synchronized elementary content, obtaining the plurality of mutually synchronized elementary content from the received digital multimedia content, and transmitting the obtained elementary content to a corresponding destination device via different data streams (or referred to as elementary data streams) based on a profile received from RM server 102.

-an RM client device (104 or 105) for receiving one or more base content and a profile for the one or more base content from the distribution server 103 and rendering the received one or more base content based on the profile. Furthermore, the RM client device can also send instructions to the interaction server 101 via the distribution device 103 to control the multimedia content, e.g., change the destination device of one or more primary content, change the state of the digital multimedia content, such as play, pause, and resume play. Further, the RM client device can send instructions directly to the interaction server 101.

Fig. 2 is a block diagram showing the distribution apparatus 103 of fig. 1 according to the present embodiment. It comprises a receiving module 201, a sending module 203, a processing module 202 and an adaptation module 204.

The receiving module 201 and the sending module 203 are used for receiving and sending data, respectively.

The processing module 202 is adapted to demultiplex the received multimedia content to generate two or more elementary contents and to generate one profile for each destination RM client device based on the profiles received from the RM server.

The adaptation module 204 is adapted to perform content adaptation by transforming the base content from one dataform to another, by which adaptation it is made more convenient to transmit or to be supported by the destination device. Content adaptation includes format conversion between different formats (e.g., AVI to RMVB), data rate adaptation, and quality adaptation performed by reducing the quality of the base content in order to adapt to the current bandwidth.

Furthermore, the distribution device 103 comprises a storage device for storing the profile of the requested multimedia content received from the RM server.

Fig. 3 shows an example of a configuration file in XML format sent by the RM server. In this example, the data stream of the multimedia content comprises three elementary data streams, each carrying one elementary content, namely video 1 (video), animation 1 (animation) and audio 1 (audio). The multimedia content is arranged to be presented on two client devices, specifically video 1 and animation 1 will be presented on the RM client device with identifier device 1 and audio 1 will be presented on the RM client device with identifier device 2. As can be seen from fig. 3, the control information corresponding to each of the primary contents is contained therein with information indicating which primary content is to be presented on which presentation device, information when presentation of the primary content is started, and information of a presentation time period, and the like. Video 1 and audio 1 start at time 0 and last 500 seconds, and animation 1 starts at time 100 seconds and lasts 100 seconds. Selectable options for video 1 and audio 1 include "play", "pause", "resume play", and "stop", and options for animation 1 include simply "move" from device 1 to device 2.

FIG. 4 is a flow chart describing a method for rendering multimedia content on different rendering devices according to the present embodiment.

In step 401, a user has two or more rendering devices that send a request for multimedia content. For example, the rendering devices are connected to the distribution device through different connections, and the rendering devices communicate directly with each other differently. The device used by the user for sending the request may be one of the rendering devices or may be a control device dedicated to sending commands. Here, the multimedia content includes two or more pieces of elementary contents synchronized with each other therebetween. The synchronization referred to herein includes not only the meaning of synchronization between video and audio but also their correspondence on the time axis, for example, the relationship of the start time between the primary contents, and in the above example, the start time of the text advertisement is 100 seconds later on the time axis than the start time of the video.

In step 402, the distribution device receives the profile of the requested multimedia content. The profile may be pre-configured or dynamically generated based on a user's request. Here, the configuration file shown in fig. 3 is used as an example to explain the present embodiment. The profile of the requested multimedia content indicates control information and synchronization information of the primary content. The three elementary contents, animation 1, audio 1 and video 1, are synchronized by using the parameter start and a synchronization mechanism (not shown in fig. 3). The parameter "start" indicates the moment in time at which presentation of the primary content is started on a time axis associated with the multimedia content. The synchronization mechanism is a mechanism for synchronizing subsequent data of the base content in the course of rendering the base content. For example, time stamps associated with video and audio data packets are used to synchronize subsequent data of the video and audio. Here, animation 1 is removed from the tag of "synchronization" (synchronization) since animation 1 does not need to synchronize its playback data with video 1 and audio 1. Further, the timestamp synchronization mechanism is set as a default synchronization mechanism. Unless otherwise indicated in the configuration file, we consider the timestamp synchronization mechanism to be used for the primary content contained in the "synchronized" tag. In addition, other time codes may be employed as a means of synchronizing file, video and audio applications. These time codes are characterized by a sequence of digital code words that are generated by a time system at regular time intervals. The control information in this example indicates a destination presentation device and action for each primary content. Further, a profile of the requested multimedia content is stored in the distribution device.

In step 403, the distributing device generates a configuration file for the rendering devices based on the configuration file received in step 402 and sends the generated configuration file to its corresponding rendering device, respectively. Fig. 5 and 6 show configuration files for device 1 and device 2, respectively.

In step 404, the distribution device receives a multimedia data stream carrying the requested multimedia content.

In step 405, the distribution device distributes the multimedia data stream according to a correspondence between the base content and the rendering device, wherein the correspondence is obtained from a configuration file of the multimedia content.

Fig. 7 is a flow chart illustrating the distribution of multimedia data streams carrying multimedia content according to the present embodiment.

In step 701, the distribution apparatus demultiplexes the multimedia data stream carrying the multimedia content into two or more elementary data streams, each carrying a elementary content; and

in step 702, the distribution device sends two or more elementary data streams according to their correspondence with the presentation device. In addition, the distribution device may content adapt some of the all of the base content before it is sent.

According to a variant of this embodiment, all elementary data streams can be divided into two or more groups according to the correspondence of elementary content and presentation device, and then the two or more groups are transmitted. For example, a multimedia data stream is demultiplexed to generate two or more groups of elementary data streams corresponding to two or more rendering devices. In the example of fig. 3, the multimedia data stream is divided into two basic data stream groups, for example, group 1 containing video 1 and animation 1 corresponding to device 1, and group 2 containing audio 1 corresponding to device 2.

According to a variant of this embodiment, the configuration file for the rendering device can be transmitted together with the base content.

According to a variant of this embodiment, the profile contains only information indicating the destination rendering device of the primary content.

According to a variant of this embodiment, the configuration file is not received from the RM server, but is statically configured in the distribution device or received from a dedicated control device for saving the configuration file of the multimedia content. Furthermore, the distribution device may periodically generate and transmit the profile to the respective rendering device, thereby making the synchronization of the playback of all the elementary content more accurate.

According to a variant of this embodiment, the multimedia content is stored in a rendering device. If the rendering device does not have the capability to stream data, the distribution device with the newly added data stream module will retrieve the multimedia content and stream it.

According to a variant of this embodiment, the control information and the synchronization information may be sent in different profiles or determined in different ways. For example, control information is pre-stored in the distribution device and synchronization information is present in the configuration file. Further, it is also possible that the distribution device is used for generating control information and/or synchronization information.

During playback of the primary content on the different rendering devices, one of the rendering devices or other dedicated control device can send an interactive message or a control message. In case of an interactive message, such as a voting message and a message exchanged for another multimedia content, the message will be sent to the interactive server for processing, which works similarly to the current RM system. However, if it is a control message that requires a change in multimedia distribution, the message is sent directly to or relayed via the interaction server to the distribution device for processing. Here, the concept of multimedia distribution includes not only control information such as a change in destination device of a piece of primary content but also synchronization information such as a change in start time of a piece of primary content and a change in synchronization mechanism, and the like. In the case where the message is sent directly to the distributing device, if the server needs the information for some purpose, the distributing device will notify the server of the changes.

Fig. 8 is a flow chart illustrating a method for changing distribution of multimedia content according to the present embodiment.

In step 801, the distribution device receives a request for changing the distribution of the multimedia content;

in step 802, the distribution device updates the profile of the multimedia content based on the request;

in step 803, the distributing device generates a configuration file for each rendering device and sends them to the corresponding rendering device; and

in step 804, the distribution device distributes the multimedia data stream carrying the multimedia content based on the relation of the base content and the rendering device, which is obtained from the updated configuration file of the multimedia content.

According to a variant of this embodiment, a request to change the distribution of the media is sent to the RM server, which then generates and sends the profile of the updated multimedia content to the distribution device.

Still using the example in fig. 3, changes to the destination device are described as follows. When the "move" action of the animation 1 on the device 1 is triggered, the device 1 sends a corresponding instruction to the dispensing device. The distribution device then updates the configuration file accordingly, as shown in fig. 9. After the update, the distribution device generates configuration files for device 1 and device 2 (as shown in fig. 10 and 11), and then transmits the configuration files to device 1 and device 2, respectively.

The following contains a description of changing the synchronization information. When the "pause" action of video 1 is triggered, video 1 sends a corresponding command to the distribution device, thereby enabling audio 1 to synchronize to the same state of video 1, i.e., the pause state. Fig. 12, 13, and 14 describe the updated configuration file in the distribution apparatus, the configuration file for the apparatus 1, and the configuration file for the apparatus 2, respectively, according to the present embodiment.

The system diagram of fig. 15 describes one application of the above-introduced method according to the present embodiment. In the system, a music box capable of playing songs is connected to a home gateway through a UPnP interface; the mobile phone is connected to the wireless access point via a wireless interface. Songs, lyrics, and synchronization and control information are sent from the rich media server. The home gateway works like the distribution device described above, i.e. it sends the song and its associated synchronization and control information to the music box for playing and the lyrics of the song and its associated synchronization and control information to the mobile phone for display. The lyrics and the song are synchronized by using the synchronization and control information. In addition, the mobile phone has a function of controlling playback of songs and lyrics, such as play, pause, resume play, by sending instructions to the home gateway.

According to a variant of this embodiment, the multimedia content comes from a client device, instead of the server. Which means that the multimedia content is stored at a first client device and the distribution device retrieves the multimedia content from the first client device and distributes the multimedia content to a plurality of client devices (possibly including or not including the first client device) according to a profile associated with the multimedia content.

A number of implementations have been described. Of course, it should be understood that modifications can also be made thereto. For example, elements of different implementations can be combined, supplemented, modified, or removed to form other implementations. Moreover, those of ordinary skill in the art will appreciate that structures and processes described above can be substituted for other structures or processes and that the resulting implementations perform at least substantially the same function in substantially the same way to achieve at least substantially the same result as described above. Accordingly, these and other implementations are within the scope of the invention.

Claims (11)

1. Method for distributing multimedia content to at least two presentation devices, characterized in that said multimedia content comprises at least two elementary contents, said method comprising the steps of:

distributing synchronization information of the multimedia content based on a relationship between the at least two presentation devices and the at least two elementary contents, wherein the synchronization information indicates a synchronization relationship between the at least two elementary contents;

receiving a multimedia data stream carrying the multimedia content; and

distributing the multimedia data stream based on the relationship of the at least two rendering devices and the at least two elementary contents.

2. The method of claim 1, wherein the playback of the at least two base content on the at least two rendering devices is synchronized on each rendering device based on the distributed synchronization information.

3. Method according to claim 1 or 2, wherein the step of distributing synchronization information comprises

Generating at least two sets of synchronization information for the at least two base contents, each set of synchronization information corresponding to one of the at least two presentation devices and including synchronization information of at least one base content associated with the corresponding presentation device; and

the at least two sets of synchronization information are transmitted to the corresponding rendering devices.

4. Method according to claim 1 or 2, wherein the step of distributing the multimedia data stream comprises

Demultiplexing the multimedia data stream into at least two sets of elementary data streams, each carrying a primary content of the multimedia content, wherein each set of elementary data streams corresponds to one of the at least two rendering devices; and

the at least two sets of elementary data streams are sent to the corresponding rendering devices.

5. A method according to any one of claims 1 to 4, wherein the relationship of said at least two rendering devices and said at least two elementary contents and/or the synchronization information of said multimedia contents is preconfigured, received from a control device or generated in response to an instruction requesting said multimedia contents.

6. The method of claim 1, wherein the multimedia content is stored in a particular one of the rendering devices, the method further comprising

Receiving a multimedia data stream of the multimedia content from the particular rendering device if the particular rendering device storing the multimedia content has a data streaming function; or

If not, the multimedia content is received and data streamed.

7. The method of claim 1, wherein in response to an instruction to change distribution of the multimedia content, the method further comprises

Determining updated relationships between the at least two rendering devices and the at least two base contents and/or updated synchronization information of the multimedia contents; and

distributing the updated synchronization information of the multimedia content based on the updated relationship between the at least two rendering devices and the at least two base contents.

8. The method of claim 7, wherein the step of determining updated relationships and/or updated synchronization information further comprises

Receiving an instruction to change distribution of the multimedia content; and

updating the relationship of the at least two presentation devices and the at least two elementary contents and/or the synchronization information of the multimedia content based on the received instructions;

or the steps comprise

Receiving updated relations between the at least two presentation devices and the at least two elementary contents and/or updated synchronization information of the multimedia contents from a control device.

9. The method of any one of claims 1 to 8, further comprising

Periodically distributing current synchronization information of the multimedia content based on a current relationship of the at least two rendering devices and the at least two base content.

10. A method as claimed in any one of claims 1 to 9, wherein the relationship of the at least two presentation devices and the at least two base content indicates a destination presentation device for the base content.

11. Device for distributing multimedia content to at least two presentation devices, characterized in that said multimedia content comprises at least two elementary contents, said device comprising

A storage module for storing a relationship between the at least two presentation devices and the at least two base content;

a receiving module, configured to receive synchronization information of the multimedia content indicating a synchronization relationship between the at least two pieces of elementary content, and to receive a multimedia data stream carrying the multimedia content; and

a processing module for distributing the synchronization information to the at least two rendering devices based on the relationship of the at least two rendering devices and the at least two elementary contents, and distributing the multimedia data stream received by the receiving module based on the relationship of the at least two rendering devices and the at least two elementary contents.