HK1146760B - Method and system for customising live media content - Google Patents

Description

Method and system for customizing live media content

Technical Field

The present invention relates to a method and system for customizing live media content.

Background

In the information age, the coverage of telecommunications, broadcasting, and information technology has obscured the dichotomy of content and carriers. In particular, traditionally the telecommunication infrastructure was used solely for voice applications, while now the telecommunication infrastructure is being used for broadcasting multimedia content. Also, technology has allowed a large number of applications on the internet, such as VoIP and video streaming. As such, there are now a variety of ways in which the public can access for use in broadcasting to the public. However, more can be done to make these content more appealing to the end user, especially since each user typically has preferences and tastes that may fluctuate with trends and trends.

It is an object of the present invention to provide a method and apparatus for customizing live media content that provides the public with a useful choice.

Disclosure of Invention

In a first aspect, a method of customising live media comprising at least one dewarped panoramic view of a monitored area is provided. The panoramic view may have a plurality of portions for selection by a user, the method comprising: receiving live media having audio and video, the live media being interleaved and/or compressed into single stream data; receiving a selection of one of a plurality of portions of a panoramic view; and customizing the live media for display based on the selection of the one portion of the panoramic view. The user may be a designated content selector/arranger. The user may also edit/modify/change the content of the live media.

Preferably, the at least one panoramic view is a 180 ° or 360 ° monitored area.

Advantageously, the single stream data comprises an access control that restricts the content of the live media, and the method further comprises: authenticating to determine access rights; and releasing the content of the live media if the access rights are granted.

It is also advantageous that when the selected portion of the panoramic view includes a moving object, the method further comprises configuring the selected portion to track the moving object within the monitored area.

Preferably, the method further comprises adding/replacing and playing the selected audio input with an audio input, such as music, sound effects, mood toasting music, voice commentary, and the like.

Preferably, the method further comprises adding a visual index to the selected portion of the panoramic view, such as text in multiple fonts, animation effects, video effects or icons, and displaying the visual index with the selected portion of the panoramic view. It may be advantageous for the visual index to include a hyperlink that directs the user to another portion of the stream.

The method may advantageously comprise storing/re-transmitting the customised live media as a digital file capable of playback on the device.

A system for customizing live media including at least one dewarped panoramic view of a monitored area is also provided. The panoramic view may have a plurality of portions for selection by a user, and the system may include: a receiver for receiving live media having audio and video, the live media being interleaved and/or compressed into single stream data; a processor to receive a selection of one of a plurality of portions of a panoramic view; and the processor is configured to customize the live media for display based on a selection of a portion of the panoramic view. Preferably, the system may be incorporated into a portable media player, which is another aspect of the present invention.

Drawings

Embodiments of the invention will now be described, by way of example, with reference to the accompanying drawings, in which:

fig. 1 is a block diagram illustrating a preferred embodiment of the present invention.

FIG. 2 is a schematic diagram illustrating one aspect of the present invention.

Detailed Description

Fig. 1 shows an application overview of a preferred embodiment of the present invention, in which a camera 100 with a high-resolution fish-eye lens (fish-eye lens) or a wide-angle lens is used to capture a panoramic video image of a monitored area. An example of such a camera 100 is available from www.grandeye.com, and the web site also has exemplary images of 180 ° and 360 ° panoramic images. Preferably, the camera 100 is used for the capture of live images.

Interaction with such panoramic video images is known, for example as described in US6,243,099 and US6,271,853, and certain parts of the panoramic image may be selected as close-up views (close-up views). These close-up views may include motion tracking so that objects subject to the close-up views may be tracked, if not stationary. However, the original panoramic video image captured is warped (skewed), especially around the edges of the image, and therefore a rectification of the panoramic video image is required in order for the user to resolve the content of the panoramic video image, so that the user can resolve which parts of the image are of interest to the user. The rectification may include performing a linear transformation (rectilinear transform) on the warped panoramic video image.

However, for some applications, capturing video images alone is not sufficient, and it is also necessary to receive sounds associated with the monitored area. For example, if the camera 100 is used to capture images of a live audience, it may be desirable to impart an effect on the captured images with the sound emitted by the audience. This is the case, for example, where a camera is used to capture a bird's eye view of a stadium playing a football game, and the audience's voice will enhance the entertainment value of the captured image, which may give the user a more immersive experience related to the user's perception of the captured image. Thus, in a preferred embodiment, one or more microphones 102 are placed at strategic locations to receive sound or audio from the monitored area.

The captured sound and video is transmitted to the processing unit 104 by a wireless channel, but wired means may also be used and may be preferred depending on the application.

The processing unit 104 processes the live sound/audio and live panoramic video images by interleaving (interleave) and/or compressing the audio and video data in a single stream for live transmission by the transmitter 106 to the receiving device. The video data may be in a conventional resolution or high definition format. The processing unit 104 will need to process more data for high definition format video.

It should be appreciated that the video data may only be captured in a high definition format and that if the receiving device is unable to support video in a high definition format, the video data is reduced to a conventional resolution at the processing unit 104. Processing unit 104 may determine the capabilities of each receiving device by sending a query to each receiving device and then receiving a response from each receiving device. The video data subsequently transmitted to each receiving device is streamed to each receiving device according to the response from each receiving device received at processing unit 104.

The transmission to the receiving device may be via any suitable communication network and, in this embodiment, a mobile communication network 108 is used and the receiving device is a portable media player 110 communicatively coupled to the network 108. The mobile communication network 108 may include any network that facilitates/enables data transfer. However, it should be noted that the receiving device is not limited to the portable media player 110 and may include other computing devices, such as a mobile phone, a laptop, a PC, and so forth. The transmission to the receiving device may also be via a peer-to-peer network, as described in US6,901,604.

Each portable media player 110 includes a receiver for receiving single stream live media and a processor for converting the data stream into multimedia content for playback by each media player 110. Traditionally, the user of the portable media player has been limited according to what the source is sending, however, the processor of the portable media player 110 of the preferred embodiment allows the user to customize or interact with the multimedia content being played.

The customization/interaction of multimedia content takes the form:

1) selection of parts within a video image

Using a football game as an example of a monitored area, a user may choose to view a portion of the court instead of a 360 ° bird's eye view. The user may be a content selector/arranger designated for a predetermined group of users and may individually determine content being viewed by other users in the group of users. The user group may join voluntarily by payment or other means. The status of the user (e.g., all content selection/placement rights, limited content selection/placement rights, no content selection/placement rights, etc.) may be based either on a payment level or may be based on the type of receiving device (e.g., a device with little processing power may not be able to select/place content).

Since the panoramic video image shows an entire view of the monitored area, if each user of the plurality of portable media players 110 has full content selection/placement rights, each user is able to view a particular portion of the court according to each user's preferences. Each user with full or limited content selection/arrangement rights may, for example, choose to view panoramic video images, continuously monitor a particular portion of the court, switch to view a different portion of the court, and so forth. The (rectified) panoramic video images transmitted in a single media stream can support live viewing of an unlimited number of different portions of the court by multiple users having the portable media player 110. This is because each frame of the panoramic video image can include sufficient resolution to allow each of the plurality of users having the portable media player 110 to zoom in on a portion of the court without having to do a solid stop-motion animation (navigation) of the zoomed-in portion of the court. In this way, the user may be able to pan and zoom different portions of the panoramic video image. Split screen views of different portions of the monitored area may also be displayed on the portable media player 110. It should be appreciated that the split view is produced at the processing unit 104 and sent to the portable media player 110 as a single media stream. This is advantageous because the portable media player 110 may not have sufficient processing power to receive and playback multiple media streams.

Access to a particular portion of the monitored area may be limited by the provider of the panoramic video image. Certain portions with restricted access may be covered by, for example, a black box, a mosaic view of certain portions, a box displaying some form of advertising, etc. Only the user of the portable media player 110 having "keys" or access privileges may be granted access to these particular portions of the monitored area. The "key" or access privilege may be associated with the user or may reside in the portable media player 110. The "key" may be used in an authentication process (virtual handshake process or may include a more complex encryption/biometric identification process). These "keys" and/or access privileges may be provided to users who have provided consideration (money or otherwise) to the provider of the panoramic video image to access a particular portion of the monitored area.

Referring to fig. 2, a representative image 200 of a soccer game being sent to the portable media player 110 is shown. The first portion 202 of the image 200 is in a team rest area of a team. The second portion 204 of the image is in a goal region of a team.

2) Motion tracking

Using a beach volleyball game as another example of a monitored area, a user with full content selection/placement rights may be interested in monitoring one of the players, and may not be interested in just the volleyball game itself. This functionality allows the user to select which player to track so that the processing unit 104 tracks and displays the player's movements for display on the portable media player 110. Since the panoramic video image shows an entire view of the monitored area, each user of the plurality of portable media players 110 is able to view a particular player or even a ball according to each user's preferences. Each user may choose to continuously monitor a moving object, switch views to view different moving objects, or even split picture views with different aspects of the monitored area. It should be appreciated that the split view is produced at the processing unit 104 and sent to the portable media player 110 as a single media stream. This is advantageous because the portable media player 110 may not have sufficient processing power to receive and playback multiple media streams.

The panoramic video images transmitted in a single media stream can support live viewing of an unlimited number of different moving objects by multiple users with the portable media player 110. This is because each frame of the panoramic video image can include sufficient resolution to allow each of the plurality of users having the portable media player 110 to zoom in on the moving object and track the motion of the moving object without the need for a physical freeze animation of the zoomed in moving object.

Monitoring of moving objects may also be limited by the provider of the panoramic video image. A mobile object with limited access may be covered by, for example, a black box, a mosaic view of the mobile object, a mobile box displaying some form of advertisement, etc. Only the user of the portable media player 110 having "keys" or access privileges may monitor these moving objects. The "key" may be associated with the user or may reside in the portable media player 110. The "key" or access privileges may be used in an authentication process (virtual handshake process or may include a more complex encryption/biometric identification process). These "keys" or access privileges may be provided to users who have provided consideration (money or otherwise) to the provider of the panoramic video image to gain access to monitor these moving objects.

Referring to fig. 2, the portion 206 is on a player and the user can track the player's movements.

3) Customization of audio

The portable media player 110 also allows the user to incorporate a selected audio input (track) while viewing the video images. For example, if the user does not wish to hear the original sound picked up from the monitored area or wishes to include additional audio input in addition to the original sound, the user may choose to use, for example, music, sound effects, mood toasting music, voice commentary, and so forth. The portable media player 110 can include at least one database of music, sound effects, atmosphere-baked music, etc. having different file formats. The user may incorporate audio input at any time while viewing the video images. In connection with voice commentary, the user may utilize a microphone included in the portable media player 110 to record the voice commentary (perhaps as a star wav file) and simultaneously merge the commentary with the video image.

It should be appreciated that only users with full content selection/placement rights can merge commentary with video images and send to other users. The voice commentary may be packaged with the video images at the processing unit 104 and sent to the other portable media players 110 for use by the respective users of the portable media players 110. The selected audio input may be tagged to the video image in relation to a point in time (junctureof time). Thus, the user is able to view the video image with at least one custom selected audio input.

4) The visual index includes

The portable media player 110 also allows the user to incorporate a selected visual index when viewing the video images. For example, if a user wishes to change the visual appearance of a portion of a monitored area, the user may select a visual index (e.g., text, animation effects, video effects, icons, etc.) that includes a selection about him. The portable media player 110 may include a database of at least one text font, animation effects, video effects, icons, and the like. The user may incorporate the visual index at any time while viewing the video image. The user may utilize at least one input device included in the portable media player 110 to compose/select a visual index and simultaneously merge the visual index with the video images. Thus, the user may view the video image with at least one custom selected visual index. The visual index may also include hyperlinks that may direct the user to another portion of the stream.

Referring to fig. 2, portion 208 shows an example where text 210 and icons 212 are included in a video image.

The portable media player 110 also has a storage device that enables customized/interacted with received images to be stored for playback or retransmission. The customized/interacted with received image may be stored or retransmitted as a digital file that may also be played back on a device other than the portable media player 110 (e.g., an older media player with less processing power).

According to the preferred embodiment, the user has complete control over what he wishes to see and hear, allowing entertainment to reach new heights.

The described embodiments should not be construed as limiting. For example, the monitored area may be any area depending on the application. For example, if the embodiment is used for real TV, the monitored area may be a scene of a program.

While the invention has been fully described, those skilled in the art will recognize that many modifications may be made without departing from the scope of the claims.

Claims (6)

1. A method of customizing multimedia content, the multimedia content comprising at least one rectified panoramic view of a monitored area of interest to a plurality of users associated with respective media players for viewing the multimedia content and comprising at least one designated content arranger, the plurality of users actively grouped into at least one user group associated with the designated content arranger, the panoramic view having a plurality of portions for selection by the designated content arranger, the designated content arranger determining multimedia content being viewed by the associated user group, the method comprising:

i) receiving the multimedia content having audio and video, the multimedia content being interleaved and/or compressed into single stream data;

ii) receiving a selection of one of a plurality of portions of the panoramic view by the specified content arranger; and

iii) customizing multimedia for display by a media player of at least one user group associated with the designated content arranger based on selection of the portion of the panoramic view,

wherein at least one of the plurality of portions of the panoramic view includes a moving object of interest to the specified content arranger, and the method further comprises configuring the selected portion to track moving objects within the monitored area.

2. The method of claim 1, wherein at least one panoramic view is a 180 ° or 360 ° monitored area.

3. The method of claim 1, wherein the single stream data includes an access control that restricts the multimedia content, and the method further comprises:

authenticating to determine access rights; and

releasing the multimedia content if access rights are granted.

4. A system for customizing multimedia content including at least one rectified panoramic view of a monitored area of interest to a plurality of users associated with respective media players for viewing the multimedia content and including at least one designated content arranger, the plurality of users actively grouped into at least one user group associated with the designated content arranger, the panoramic view having a plurality of portions for selection by the designated content arranger, the designated content arranger determining multimedia content being viewed by the associated user group, the system comprising:

means for receiving the multimedia content having audio and video, the multimedia content being interleaved and/or compressed into single stream data;

means for receiving a selection of one of a plurality of portions of the panoramic view by the designated content arranger; and

means for customizing multimedia for display by a media player of at least one user group associated with the designated content arranger based on selection of the portion of the panoramic view,

wherein at least one of the plurality of portions of the panoramic view includes a moving object of interest to the specified content arranger, and the system further comprises means for configuring the selected portion to track the moving object within the monitored area.

5. The system of claim 4, wherein at least one panoramic view is a 180 ° or 360 ° monitored area.

6. The system of claim 4 or 5, wherein the single stream data includes an access control that restricts the multimedia content, and the system further comprises:

means for authenticating to determine access rights; and

means for releasing the multimedia content if access rights are granted.