CN1428045A - System and method for providing multi-perspective instant replay - Google Patents

Abstract

A method for processing broadcasts comprises receiving a broadcast of the program containing a plurality of perspectives of the program and presenting at least one of the plurality of perspectives to a viewer. The method further includes automatically recording the plurality of perspectives in a storage device and playing alternate recorded perspectives for the viewer without interrupting the recording of the broadcast. A system for recording a broadcast containing a plurality of perspectives of a program is also disclosed.

Description

System and method for providing instant replay of multi-perspective pictures

Cross reference to related applications

This application claims priority to the following U.S. Provisional Patent Application: SYSTEM AND METHOD FOR PROVIDING MULT, filed September 26, 2000, hereby incorporated by reference - PERSPECTIVE INSTANT REPLAY)" U.S. Provisional Patent Application No. 60/235,529 (Attorney Docket OPTVP014+); and hereby incorporated by reference, filed August 2, 2000, entitled "Inserting previously broadcast (SYSTEM AND METHOD FORINCORPORATING PREVIOUSLY BROADCAST CONTENT)" U.S. Provisional Patent Application No. 09/630,646 (Attorney Docket No. OPTVP013).

U.S. Provisional Patent Application No. 60/162,490, filed October 29, 1999, and entitled "RECORDING OF PUSH CONTENT," which is hereby incorporated by reference (Client Docket No. OTV0033+).

Technical field

The present invention relates generally to interactive video delivery media, such as interactive television, and more particularly to systems and methods for providing instant playback of multi-perspective views of broadcast content.

Background technique

The broadcast service provider sends the audio-video stream to the viewer's television. Interactive television systems are capable of displaying text and graphic images in addition to typical audio-video programming. They can also provide viewers with many services such as TV shopping, and other interactive applications. In addition to the audio-video part, the interactive television signal may also include an interactive part consisting of application code, data and signaling information. The broadcast service provider may combine any or all of this information into a single signal and several signals to receivers connected to the viewer's television, or the provider may include only a subset of the information, perhaps also with a resource locator. Such a resource locator may be used to indicate an alternate source of interactive and/or audio-visual information. For example, a resource locator may take the form of a World Wide Web Universal Resource Locator (URL).

Generally, the TV signal is compressed before being sent out over a typical broadcast medium such as a closed circuit television (CATV) line or a direct satellite transmission system. Information referenced by a resource locator may be available on a different medium, for example, through an always-on return channel such as a DOCSIS modem.

The set-top box connected to the TV controls the interactive functions of the TV. The set-top box receives the signal sent by the broadcast service provider, separates the interactive part from the audio-video part, and decompresses each part of the signal. The set-top box sends the audio-video signal to the TV and uses the interactive information to execute an application. Set-top boxes typically only include limited storage capacity. While this memory is sufficient to execute interactive applications, it is generally not sufficient to store applications for infinitely long intervals of time. Also, the memory of a set-top box is often too small to accommodate programs that include large amounts of audio or video data, application code, or other information. An external storage device can be coupled with the set top box to provide additional storage for storage of video and audio broadcast content.

Interactive content, such as application codes or information related to television programs, is often broadcast in a repetitive format. The pieces of information broadcast in this form form a so-called "carousel". Repeatedly sending the objects in the "spinning horse" allows the receiver to receive those objects without a return path from the receiver to the server. If a receiver needs a particular piece of information, it can simply wait until the next time that piece of information is broadcast, and then extract that information from the broadcast stream. If the information is not cyclically broadcast, the receiver would have to send a request to the server for that information, requiring a return path. If the user is initially not interested in "Spin Your Horse" content, but later expresses interest, the information can be obtained at the next time that "Spin Your Horse" is broadcast. Because broadcast networks use only limited bandwidth, audio-visual content is not broadcast as a "spinning horse". There is also not enough bandwidth and server resources to process the massive amounts of data required for video and audio in real time to handle the near-simultaneous requests made by large numbers of TV viewers for previously broadcast content.

In a broadcast over a television network, such as a broadcast of a sporting event, a content provider may generate, for example, multiple video feeds from various angles of the game. The network may select one or more feeds from a plurality of video feeds and broadcast the selected video feed to viewers at any given point in time. That is, for example, the network could simultaneously broadcast video tracks representing the same scene but from different perspectives, or, if the film is played in different languages, send different audio tracks or subtitles . Viewers can use an interactive application running on their set-top boxes to select different perspectives. When a viewer requests a change of perspective, the interactive application utilizes the metadata to determine which data packet contains the selected perspective. Then, the transmission of the data packet containing the newly selected perspective picture is started.

As noted above, viewers cannot request previously broadcast audio or video material due to the limited bandwidth available to broadcast networks. Furthermore, the data accompanying the interactive application often corresponds to the audio and video currently being broadcast and therefore changes frequently. In these cases, those values broadcast as part of the "spinning horse" often also change, and the "spinning horse" no longer carries those old values. Viewers cannot replay scenes from different perspectives or sports events unless the viewer has recorded a video stream from an alternate perspective.

Invention Summary

The invention discloses a method and a system for providing multi-perspective instant playback. A method of processing broadcast content generally includes receiving a broadcast of a program including a plurality of see-through views of a program, and presenting at least one of the plurality of see-through views to a viewer. The method may also include automatically recording a plurality of perspective frames in the storage device, and playing alternately recorded perspective frames for the viewer without interrupting the broadcast recording.

A television program may include many types of associated video streams, audio streams, executable code, and data. When appropriate, a picture-in-picture (PIP) window in the TV screen can be used to display several see-through images to the viewer at the same time.

A system for recording a broadcast comprising multiple perspective views of a program generally includes a receiver operable to receive the broadcast, a storage device coupled to the receiver, and a processor for at least one role of presenting the multiple perspective views to a viewer. The receiver may also function to automatically record multiple perspective frames in the storage device, and to play that recorded perspective frame to the viewer without interrupting the recording of multiple perspective frames.

The receiver may be a set-top box, and the storage device may be contained in or coupled to the set-top box. Storage devices may include, for example, magnetic or optical disks, or flash memory, among others. A receiver box may include one or more tuners.

Other features, advantages and embodiments of the present invention will be apparent to those skilled in the art from the following description and drawings, as well as from the appended claims.

Brief description of the attached drawings

Figure 1 is a diagram showing the distribution of television programs and signaling information from broadcasting stations to receiving stations;

FIG. 2 is a block diagram of a system of the present invention for recording programs received from the broadcast station shown in FIG. 1;

Figure 3 is a block diagram representing data transfer to a storage device coupled to the set-top box shown in Figure 2;

Figure 4 is a diagram showing that three video streams and two audio streams are simultaneously sent to the receiving station, and one of the audio streams and one of the video streams are sent to the television set, and the two streams are combined with the other two video streams one of the streams together to the storage device;

Fig. 5 is similar to the graph of Fig. 4, except that this moment, the second video stream is displayed in the PIP window together with the first audio and video stream shown in the main picture of the TV;

Fig. 6 is a graph similar to the graph of Fig. 5, except that this moment, the second video stream is displayed in the center of the TV screen together with the first video stream displayed in the PIP window;

Figure 6a is a graph similar to that of Figure 6, except that the structure shown does not require or use a PIP;

Figure 7 is a graphic similar to that of Figure 6, except that the live broadcast of the second video stream is replaced by a previously broadcast version of the same perspective;

FIG. 7a is a graph similar to that of FIG. 7, except that the structure shown does not require or use PIP, and a recorded audio stream is played instead of the live audio stream as shown in FIG.

Figure 8 is a diagram illustrating a first video stream and an audio stream displayed on a television set and recorded together with a second audio stream;

Figure 9 is a graph similar to that of Figure 8, except that the first audio stream is replaced by the second audio stream;

Figure 10 is a graph similar to that of Figure 9, except that the first video stream and the second audio stream are replaced by earlier broadcast versions;

Figure 11 shows examples of files and data structures on a storage device, and the text accompanying Figure 11 describes how these data structures can be used to more easily view instant replays from different perspectives; and

Fig. 12 is a flowchart of the method according to the present invention.

Like reference numerals indicate corresponding parts throughout the drawings.

Detailed ways

The following description is given to enable any person of ordinary skill in the art to make and use the present invention. Descriptions of specific embodiments and applications are provided by way of example only, and various modifications will be apparent to those skilled in the art. The general principles described here can be applied to other embodiments and applications without departing from the scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein. Those of ordinary skill in the art will appreciate that many embodiments are possible, for example, using a computer system and display to implement the functions and features described herein. For the sake of clarity, in the application of the present invention, the present invention is described as a set-top box for use with a television set, and details related to technical matters known in the technical field of the present invention are not included.

Referring now to the drawings and initially to FIG. 1, there is shown and generally indicated at 10 a diagram of a television broadcasting and receiving system. The system 10 includes a broadcast station 12 where audio-visual and control information is assembled in the form of digital data and mapped into a digital signal sent via satellite to a receiving station. Control information such as conditional access information and signaling information (e.g., a list of services available to the user, event names, and event schedules (start time/date and duration), and program specific information) can be added to the video , audio, and interactive applications for use with interactive television systems. The control information can describe the relationship between streams, for example, it can be considered which streams carry programs with different perspective images than which other streams. The control information is converted by the broadcast station into a format suitable for transmission on the broadcast medium. Data can be formatted, for example, into data packets that can be sent over a digital satellite network. Data packets may be multiplexed with other data packets for transmission. Typically, the signal is compressed prior to transmission and it may then be transmitted over a broadcast channel such as a closed circuit television line or direct satellite broadcast system 22 (shown in FIG. 1 ). The Internet, telephone lines, cellular networks, fiber optics, or other terrestrial transmission media can also be used in place of closed circuit or satellite systems to transmit broadcast signals. As described below, broadcasters can embed service information in broadcast transport streams. The service information can list each elementary stream identifier and associate each identifier with a description of the type of stream (e.g., whether it contains audio or video). ) is associated with a textual description of the stream that can be understood by the user and that can be used by the user to choose between different perspective views.

The receiving station includes a set top box 16 connected to a storage device 18, and a television 20 for presenting programs to viewers. The set top box 16 functions to decompress the digital data and display the programs to the viewer. The decompressed video signal can be converted to an analog signal such as an NTSC (National Television Standards Committee (US)) format signal for display on a television. The signals sent to the set-top box 16 are filtered, and of those satisfying the filtering requirements, some are used directly by the processor 30 and others may be stored in a local memory such as RAM (Random Access Memory). Examples of requirements required for filtering include specific values in locations reserved for elementary stream identifiers or originating network identifiers. The set top box 16 may be used to overlay or combine the different signals to form the desired display on the viewer's television 20 .

As further described below, set-top box 16 is configured to simultaneously record one or more video and/or audio streams, enabling viewers to replay scenes they have recently seen or heard from different perspectives. Broadcast station 12 simultaneously broadcasts several see-through views for viewing by viewers owning set-top boxes 16 executing interactive television applications. For example, several cameras may be used to record a sporting event, and a station may simultaneously broadcast content from several cameras, enabling viewers to select between different camera perspectives using an interactive application executing on their set-top boxes 16. The broadcaster can also, for example, send the soundtrack of several see-through pictures in different languages. The several video and audio see-throughs are just a few examples of the many types of see-throughs that can be included in a broadcast. Other examples include multiple teletext streams, perhaps in different languages; multiple executable program streams, each perhaps implying a different proficiency level; or multiple data streams. The present invention enables viewers to replay the same scene from different perspectives, while ensuring that viewers are still able to simultaneously or later watch the portion of the program that was broadcast simultaneously with the replayed content they were watching. Viewers may request replay of any combination of audio, video, executables, and data from the same or a different perspective than a previously played perspective.

It should be understood that the term "program" as used herein refers to any broadcast material, or a piece of material, including a television show, sporting event, news program, movie, or any other type of broadcast material. Material may consist of audio only, video, data, or any combination thereof. A program may be just a portion of a television show or broadcast (eg, no commercials or lack an opening or ending) or more than one performance, or may include, for example, commercials. Also, it should be understood that the term "watching" as used herein is defined as viewing a program as soon as the tuner begins filtering data corresponding to the program. If the viewer has tuned to a particular frequency before the program is broadcast, then the beginning of viewing preferably corresponds to the beginning of the program. Viewing preferably ends when the program has run out or when the tuner is no longer filtering the frequencies corresponding to the program. Thus, the recording of the program coincides with the "viewing" of the program, and the program is recorded only when the tuner is tuned to the station broadcasting the program. In the event that the viewer turns off the television display after the viewer has begun recording the program, it is considered to be still active as long as the tuner is tuned to the station on which the program was broadcast and the recording of the information broadcast on the same frequency as that used to begin viewing is in progress. Keep watching. The audio-visual signals and program control signals received by set top box 16 correspond to television programs and menu selections that the viewer can access through the user interface. The viewer can control the set-top box 16 through, for example, an infrared remote control unit, a control panel on the set-top box, or menus displayed on a television screen.

It should be understood that system 10 as described above and shown in FIG. 1 is only an example of a system for transmitting signals to television 20 . The broadcast network system may differ from the broadcast system described herein without departing from the scope of the present invention.

The set top box 16 may be used with a receiver or with an integrated decoder receiver capable of decoding video, audio and data, for example, a satellite receiver or with a satellite integrated decoder receiver capable of decoding MPEG video, audio and data set-top box. Set-top box 16 may be configured, for example, to receive digital video channels supporting wideband communications using quadrature amplitude modulation (QAM), and to control channels for two-way signaling and messaging. A digital QAM channel carries a compressed and encoded MPEG (Moving Picture Experts Group) transport stream. The transport system extracts the desired programs from the transport stream, separates them into audio, video, and data components, and routes them to devices that process them, such as one or more audio decoders, one or more video decoders, and Optionally routed to RAM (or other form of memory) or hard disk (optional). It should be appreciated that set top box 16 and storage device 18 may be analog, digital, or both analog and digital.

As shown in FIGS. 1 and 2 , storage device 18 is coupled to set top box 16 . Storage device 18 is used to provide sufficient storage space to record programs that would not be suitable for recording in the limited capacity of main memory (eg, RAM) typically found in set top boxes. Storage device 18 may include any suitable storage device, such as a hard disk drive, a recordable DVD (Digital Versatile Disk) drive, magnetic tape, optical disk, magneto-optical disk, flash or solid-state memory, and the like. The memory device 18 may be internal to the set-top box 16, or may be connected externally (for example, via IEEE (Institute of Electrical and Electronics Engineers) 1394-1995) to a permanent connection of a replaceable connection. There may be more than one storage device 18 attached to the set top box 16 . Set-top box 16 and/or storage device 18 may also be contained in a single package with television 20 .

FIG. 2 shows one embodiment of the system of the present invention for recording programs received from broadcast station 12. As shown in FIG. Set-top box 16 typically includes a control unit (eg, a microprocessor), main memory (eg, RAM), and other components needed to select and decode received interactive television signals. As shown in FIG. 2 , the set top box 16 includes a front end 26 that functions to receive audio, video, and other data from the broadcast station 12 . The broadcast source signal is fed to a front end 26 of the set top box 16, which includes an analog-to-digital (A/D) converter and a tuner/demodulator (not shown). The front end 26 filters out a specific frequency band, demodulates it, and converts it to digital format. The digitized output is then sent to the transmission stage 28 . Transport stage 28 further processes the data, sending a portion of the data to audiovisual (AV) stage 34 for display and another portion to control processor 30, and filtering out the remainder of the data.

The control information can also be recorded with the audio-video data at the time of broadcast, or it can be managed by software in the set top box 16 first. For example, broadcast conditional access (CA) information can be used to decrypt broadcasted video. Optionally, the original broadcast streams, or altered versions of these streams, may be re-encrypted using set-top box keys or algorithms prior to recording. Encrypted video can also be stored with broadcast CA information upon receipt. Additionally, clock information can be translated into a virtual time system prior to recording. MPEG-2 elementary streams can be demultiplexed from MPEG-2 transport streams and then sealed into program streams for recording.

FIG. 3 shows the transfer of data from transfer stage 28 to storage device 18 . Storage device 18 typically contains a number of programs that have been recorded by viewers. Each perspective picture record is associated with identification information which may have been copied or modified from the original signaling information. This identification information may contain bookkeeping information similar to that typically stored in audio/video file systems or hierarchical computer file systems. The identification information can be of a wide variety of formats and contents, so long as it provides enough information to enable a viewer (perhaps interacting with the system) to uniquely retrieve a specially recorded see-through. Programs can be identified by ID numbers and start and end times. As described below, the memory can be periodically fragmented so that programs are stored contiguously. Direct memory access (DMA) is preferably used to send data from transfer stage 28 to storage device 18 . The data sent to the control processor 30 may include metadata describing the content of the audio-visual data stream, and may also include applications and corresponding data that may be executed on the control processor to provide interactive television.

A copy of the data sent from transport stage 28 to AV stage 34 is sent to storage device 18 at the start of viewing. The CPU (Central Processing Unit) in the control processor 30 configures the DMA controller to ensure that data is written into buffers allocated in the memory device 18 . The number of minutes of viewing the data to be recorded in the buffer is preferably selected by the viewer; however, it is also possible to preset a default value for the set-top box 16, say, 15 minutes. The CPU, which controls the processor, calculates the size of the buffer to allocate based on the number of minutes and the maximum speed at which it can send some bits (bits) of the Transport Stream the viewer is watching. This maximum speed can be obtained from the metadata sent with the audio-video stream. When the bottom of the buffer is reached, an interrupt takes control of the CPU in the processor, at which point it reconfigures the DMA controller to start writing from the beginning of the buffer. This design is called a circular buffer.

The buffer is preferably circular to allow contiguous recording and to overwrite previously recorded content. The CUP of the control processor will be interrupted when the viewer changes channels or when a TV event occurs (for example, the TV program ends). At this point, the CPU can either allocate a new buffer or mark the beginning of the new event in the old buffer. Automatic recording of the program and all associated video, audio, and data streams in the storage device at the beginning of the program without any action by the viewer enables the viewer to replay portions of the program from different perspectives.

As previously mentioned, the control processor 30 records a stream of multiple perspective frames at the beginning of the program and stores these perspective frames in the storage device 18 . These see-through pictures are recorded and stored in the storage device 18 for a predetermined time interval (eg, 15 minutes). If the viewer decides to record the entire viewing portion after the program has started, he selects the record option and processor 30 will allocate space in storage device 18 . All perspective footage will be recorded along with the program being watched. See, for example, U.S. Provisional Patent, entitled "System and Method for Incorporating Previously Broadcast Content," filed August 2, 2000, which is hereby incorporated by reference in its entirety Application No. 09/630,646 (Attorney Docket OPTVP013).

The joining of the first and second recorded portions of any given perspective frame in a common storage area may be accomplished physically or virtually. Physical implementation may include copying the first recorded portion to where the second portion has already been recorded. Virtualization may include modifying the structure of data stored in a storage device. In either case, the viewer watching the replay of any see-through picture should not be aware that the two parts of the see-through picture were originally stored separately. Accordingly, the various parts of the see-through view may be physically connected, or the various parts of the see-through view may be stored separately in a non-connected format, so long as the entire recorded program can be played back in a continuous fashion (i.e., the viewer does not experience the replay of the see-through view). jump between the first and second parts of the perspective view).

It should be appreciated that recording an entire program, including multiple perspective views, in storage device 18 may occur without any action by the viewer. For example, if a viewer wants to replay a scene by reversing a portion of one of several recorded perspectives (or similarly on a different type of storage medium), then, since the viewer has expressed his interest in the program, Interestingly, therefore, the entire program, and all of its multiple perspectives, can be recorded on the storage device.

Control information broadcast with the program preferably indicates which streams are associated with the viewed stream. By filtering for the appropriate identifier in the broadcast MPEG-2 (or DSS or other encoded) packets, the set-top box 16 can locate all relevant elementary streams. It sends the streams being viewed by the viewer to the television set 20 and records the contents of these streams, as well as other related streams, including related video, audio, executable programs and data, in the storage device 18 . Metadata indicating the maximum bit rate of the stream may be accompanied by either the elementary stream or the transport stream. The format of the recorded stream may depend on hardware support. For example, dedicated hardware inside set-top box 16 may support re-multiplexing of streams or simultaneous reading and writing to storage device 18, as is well known to those of ordinary skill in the art.

Broadcast data such as audio and video data, application code, control signals, and other types of information may be sent as data objects. If the program is to be consumed (ie presented to the viewer), the broadcast data must be analyzed in order to extract data objects from the stream. When the necessary data objects have been extracted, the program is played. For example, launching any application that needs to be executed, and playing any audio or video data that needs to be presented to an audience. If the program is stored, then the data objects are extracted in the same way and stored instead of being used directly to render the program. The recorded program is played back using the stored data object. Data objects may include "live" data that will become obsolete if not used immediately. If this data is stored and used when the program is played back, the program will be at least partially out of date. Thus, while most data objects can be stored as files, live data objects are stored as program references. When the program is played back, new live data corresponding to the reference object can be obtained instead of the data that was live when the program was recorded. Therefore, only temporarily corrected data is used by interactive applications for subsequent executions (see, for example, the RECORDING OF PUSH CONTENT filed on October 29, 1999, which is hereby incorporated by reference) )” in U.S. Provisional Patent Application No. 60/162,490 (Client Docket No. OTV0033+).

4-10 show the set top box 16 receiving three video streams and two audio streams from the broadcast station 12 . A tuner in the headend 26 receives signals from the broadcast station 12 and sends the associated stream to the demultiplexer and processor 100 . The video streams V1, V2 and the audio stream A1 are related to each other (for example, the video streams are from different cameras shooting the same sporting event and A1 is the announcer's soundtrack) and they can be provided in a single transport stream. If all relevant streams are provided in a single transport stream, then only one tuner 50 is required. Set-top box 16 may include multiple tuners 50 for recording and displaying associated streams broadcast in separate transport streams. The relevant streams are preferably broadcast on a small number of frequencies so that a large number of tuners inside or attached to the set top box 16 are not required. For example, a large number (eg, 5) of video streams may be multiplexed on a single frequency along with various audio streams, executable programs, data and control information.

Figures 4-7 show a viewer requesting playback from different perspectives using picture-in-picture (PIP) mode. If the viewer wants to watch a replay of the picture from a different perspective, then, without the need for multiple tuners in the set-top box 16 or television 20, it can be viewed in PIP mode. Since one of the previously recorded video or audio streams is from storage device 18, no additional tuner is required. All streams shown are preferably multiplexed on the same frequency. Video or audio can be delivered directly to the AV stage 34 contained in 100 itself inside the set-top box 16 and can be multiplexed with the transport stream sent through the tuner 50 . Note that 100 represents three components: (i) a demultiplexer; (ii) a processor that directs certain parts of the broadcast information to other components; and (iii) a time) modulated AV level. Alternatively, as described below with reference to FIGS. 8-10, the viewer may choose to watch only the replay, with the set-top box 16 buffering the live broadcast in storage device 18 for later delivery.

In FIG. 4, the broadcasting station 12 transmits video streams V1 and V2 containing two different perspective pictures and an audio stream A1. The two video streams could be, for example, two different camera positions on a baseball field. The viewer is currently viewing video stream V1 and listening to audio stream A1. The first and second video streams V1 and V2 and audio stream A1 are automatically recorded. Thus, if a viewer wants to replay, for example, the last scene of a match, previously broadcast information is useful. In particular, by virtue of the present invention, viewers can replay this information from any of the previously broadcast see-through pictures. The viewer can set the set-top box to PIP mode, so that the viewer can watch the first perspective picture (video stream V1) displayed in the large central area of the TV screen and the video stream displayed in the upper right corner (or some other area) of the TV screen. The second perspective frame (video stream V2) in the small frame window (Fig. 5). After an important scene in the game (eg, a double-tap in baseball), the viewer may want to watch the replay, this time from a different perspective than the one showing V1. At this point, the viewer can selectively switch to the windows in which the video streams V1 and V2 are displayed, as shown in FIG. 6 . Now, video stream V1 is sent to the PIP window, and video stream V2 is sent to the central window. The viewer can then issue a command (ie, press a button on the remote control) to rewind the video in the main window while the PIP window continues to display the "live" V1 in the PIP container.

As shown in Figure 7, a recorded video stream V2' from the same perspective as V2, but broadcast and recorded earlier, is sent from the storage device 18 to the demultiplexer in 100, which converts the previously recorded video stream V2 ' and the current video stream V1 are sent to the TV for display. Viewers can rewind or search through the recording until they reach the beginning of the recording. The viewer can also rewind and display the first video stream V1. In the meantime, the broadcast of the remainder of the program can be sent to the storage device 18 since the viewer has indicated his interest in recording. This may be automatic (ie, the program stream is sent to storage device 18 upon request for playback by the viewer), or it may only occur upon receipt of a request from the viewer to record the entire program.

Alternatively, the viewer may not like being distracted by a live broadcast as shown in the PIP of FIG. 7 . Therefore, as shown in Fig. 6a, the viewer can simply switch the perspective picture from V1 to V2 first. Thereafter, the viewer can "rewind" to an earlier event, viewing the previous scene from the perspective picture carried in the video stream V2. This situation is shown in Figure 7a where only a copy of the live video stream V1 is sent to the storage device together with the live video stream V2 and the live audio stream A1. Recorded streams V2' and A1' are sent to the TV set once (possibly after modulation). The script presented in Figures 6a and 7a may also be a script that the viewer uses to switch between a live video perspective and a different, recorded, video perspective when there is no PIP functionality associated with the viewer's television.

Figures 8-10 show the broadcasting of programs with audio streams in different perspectives. For example, a viewer may watch an Italian movie broadcast with audio stream A1 in Italian and audio stream A2 in English. As shown in FIG. 8, video stream V1 and audio stream A1 are presented to the viewer and recorded in storage device 18, while audio stream A2 is also recorded in storage device 18 but not presented to the viewer. The viewer initially listens to the broadcast in Italian (audio stream A1); however, after playing for a while, the viewer cannot understand Italian, so he selects the option "Switch to English" from the menu and the viewer now hears it in English Broadcast (audio stream A2) (Fig. 9). If the viewer wants to listen to the sound track accompanying the previous scene in English, he can rewind the tape of video stream V1 and audio stream A2 and watch the scene again in English (Fig. 10). The video and audio streams V1, A2 and A2 will be recorded forever so that viewers can watch the rest of the movie in delayed mode without missing the part of the movie that is being broadcast as the viewer rewinds and replays previous scenes.

Figure 11 shows an example of a metadata file that may be stored with each record perspective. The present invention does not require the format as shown, but rather, this format is only used as an example of how metadata can facilitate the playback of instant replays from different perspectives. Each record of the metadata file shown contains, among other possible fields, a time and an offset. In this example, program clock references are frequently, though not regularly, broadcast with the video. The value of some of these clock references when they were received by the set-top box, and the offset into the record of the most recent I-frame (I-frame) (one of the 3 types of MPEG-2 frame encoding that can be used for video) , can be recorded as metadata. Also, this is just an example, actual implementations can take advantage of P-frame and B-frame (P-frame and B-frame) (other types of MPEG-2 encoding, these two types are usually more compressed than I-frame ). The offset is measured in bytes from the beginning of the file containing the record for this perspective picture.

In this example, the viewer has watched a live broadcast containing video see-through frame _V1 . When the viewer watches, the video perspective picture _V1 is recorded in the file. Furthermore, other video perspective frames including video perspective frame _V2 are recorded in different files because they represent different perspectives of the same information. Of course, _V2 can be recorded in the same file, as long as other information to distinguish _V1 and _V2 is recorded somewhere. The viewer happens to see something of interest on the screen, and he enters the appropriate command to have V ₁ rewind to the beginning of the scene of interest. When the MPEG-2 I-frame _1,t is being used to display the contents of the screen, the viewer stops V ₁ . (Again, this is just an example. It is also possible to record the P-frame and B-frame in the file containing the I-frame from V ₁ , and use them in positioning scenarios, but, not in this example They are used. Also, MPEG-2 is used only as an example; other formats of media and/or data can be used equally). The viewer then issues a command telling the set-top box to start playing forward, but from _V2 instead of _V1 . The set-top box must determine which I-frame of the V ₂ it should display first. When both perspective frames are sent at the same constant rate, the simple solution of choosing the I-frame whose offset is closest to that of I-frame _1,t in the file containing V ₂ is reasonably correct , although such an approximation may only be useful when the two perspective frames are sent at nearly the same non-constant rate. Now for a better solution for variable rate streams or streams with different constant rates. This solution uses linear interpolation, although in some cases other well-known classical interpolation methods readily found in the open literature may provide better approximations.

The actual time corresponding to the time at which I-frame _1,t was originally intended to be played is first approximated. The offset into the file containing V ₁ where I-frame _1,t resides is used for this approximation. To approximate this time, two consecutive time values d _1,i and d _1,i+1 are retrieved in the metadata file such that d 1, _i ≦d _1,t < d _1,i+1 . (Those of ordinary skill in the art know that if the records are of fixed length and each element is stored in the order shown, then binary search (binary search) may find the two consecutive elements most rapidly .If a different storage format is used, then a different retrieval method would be more appropriate. Moreover, there are well-known techniques that are widely cited in the computer science literature). Once they are located, t _1,i and t _1,i+1 are also known. These values are then used to approximate t. This example uses the linear interpolation formula:

t=((t _1,i +1-t _1,i )(d _1,t -d _1,i )/(d _1,i +1-d _1,i ))+t _1,i

After an approximation to t has been found, it is necessary to find the position of the I-frame closest to that time in the record of the perspective frame _V2 . The first step here is to locate t _2,k and t _2,k+1 such that t _2,k ≦t<t _2,k+1 . Moreover, the best retrieval in any given situation depends on the format of the document, which is a well-studied question. Let these values be used to calculate an approximation of d2 _,t . This example again uses the linear interpolation formula:

d _2,t = ((d _2,j+1 -d _2,j )(tt _2,j )/(t _2,j+1 -t _2,j ))+d _2,j

Now, given an approximation of d2 _,t , the I-frame closest to the beginning of the file containing the recording of _{V2 , d2,t} bytes, is used as the starting frame for the viewer to play back the recording.

Figure 12 shows a process flow according to embodiments described herein. For the sake of clarity, the process is shown with a specific flow, but it should be understood that other orders are possible, and some of them can be performed in parallel, without departing from the spirit of the present invention. At step 200, the system receives a broadcast including multiple perspective views of a program. The system presents one of a plurality of see-through views to the viewer at step 210, and stores all of the see-through views in a memory device at step 220. In the disclosed embodiment, the system stores all see-throughs, however, the system may also be configured to selectively store those see-throughs based on criteria provided by the viewer (eg, an indication of which see-throughs are of interest to the viewer). In step 260, each perspective frame is stored in a circular buffer. At step 230, another see-through picture is presented to the viewer, and at step 250, the presentation of this see-through picture and the first see-through picture includes preparing an audio/visual signal for the television. In step 240, the presentation of the other perspective views in step 230 may include retrieving those perspective views stored, and the perspective view presented may be one of those stored perspective views.

Methods and systems for processing various broadcast contents have been disclosed. Software written in accordance with the present invention may be stored on some form of computer readable medium, such as a memory or CD-ROM (Compact Disc Read Only Memory), or transmitted over a network, and executed by a processor. In addition, where methods are disclosed, various other orders of steps are possible, and such steps may be performed concurrently, without departing from the scope of the invention.

Although the invention has been described in terms of the illustrated embodiments, it will be readily appreciated by those skilled in the art that various changes may be made to these embodiments without departing from the scope of the invention. Accordingly, all matter contained in the foregoing description and shown in the accompanying drawings should be regarded as exemplary and not restrictive.

Claims (54)

1. method of handling broadcasted content comprises:

Program receiving broadcasting, this broadcasting comprises a plurality of perspectives of program;

Present first of a plurality of perspectives to spectators;

Store at least one of a plurality of perspectives; With

Present at least one of a plurality of perspectives to spectators.

2. method according to claim 1 wherein presents first perspective and storage is carried out simultaneously.

3. method according to claim 1, at least one of wherein storing a plurality of perspectives carried out automatically.

4. method according to claim 1, at least one that wherein presents a plurality of perspectives comprise at least one of those perspectives of presenting storage.

5. method according to claim 4, at least one and the storage that wherein present a plurality of perspectives are carried out simultaneously.

6. method according to claim 5, at least one of wherein storing a plurality of perspectives comprise all perspectives of storage.

7. method according to claim 1 wherein receives broadcasting and comprises that receiving many kinds of associated videos flows, and every kind of stream comprises one of these perspectives.

8. method according to claim 1 wherein receives broadcasting and comprises that receiving many kinds of related audios flows, and every kind of stream comprises one of these perspectives.

9. method according to claim 1, wherein present second perspective that presents in the different windows that at least one that present first perspective in the window that first perspective is included in display and present a plurality of perspectives be included in display from a plurality of perspectives of storage.

10. method according to claim 1, at least one of wherein storing a plurality of perspectives comprise those perspectives are stored at least one cyclic buffer.

11. method according to claim 1, vision signal is prepared at least one demonstration that is included as on video screen that wherein presents a plurality of perspectives.

12. method according to claim 1, at least one that wherein presents a plurality of perspectives is included as the step that loud speaker is prepared audio signal.

13. method according to claim 4, at least one that wherein presents a plurality of perspectives comprise by demultiplexer at least one of those perspectives of storage sent to display from memory device.

14. method according to claim 4, at least one that wherein presents a plurality of perspectives comprises one of those perspectives of retrieve stored.

15. a method of playing the multi-perspective program comprises:

Program receiving broadcasting, this broadcasting comprises a plurality of perspectives of program;

At least one of a plurality of perspectives is stored in the memory device;

First of a plurality of perspectives sent to display, so that at least a portion of broadcast program; With

By second of a plurality of perspectives sent to display from memory device, program for playback is from the part of different perspectives.

16. method according to claim 15, wherein first perspective is a perspective of record.

17. method according to claim 15, also be included in when first perspective sent to window in the display, second perspective sent to different windows in the display, so that broadcast program is from the part of second perspective.

Receive to less a kind of video flowing 18. method according to claim 15, reception broadcasting comprise, one of every kind of video flowing and these perspectives interrelate.

Receive to less a kind of audio stream 19. method according to claim 15, reception broadcasting comprise, one of every kind of audio stream and these perspectives interrelate.

20. a record comprises the system of broadcasting of a plurality of perspectives of program, comprising:

Be used to receive the receiver of broadcasting;

Memory device with the receiver coupling; With

Be used for presenting at least one of a plurality of perspectives, be recorded at least one of a plurality of perspectives in the memory device and present at least one processor of those perspectives of record to spectators to spectators.

21. system according to claim 20 also is configured to write down automatically each perspective.

22. system according to claim 20 also is configured to not interrupt broadcast recoding ground presents those perspectives from record to spectators.

23. system according to claim 20, wherein receiver is a set-top box.

24. system according to claim 22, wherein memory device is included in the set-top box.

25. system according to claim 22, wherein memory device and set-top box removably are coupled.

26. system according to claim 20, wherein memory device comprises disk.

27. system according to claim 20, wherein memory device comprises CD.

28. system according to claim 20, wherein memory device comprises flash memory.

29. system according to claim 20, wherein receiver comprises at least one tuner.

30. system according to claim 29, wherein receiver comprises demultiplexer and processor.

31. a system that presents broadcasted content comprises:

Be configured to receive the receiver of the broadcasting of a plurality of perspectives that comprise program;

Store at least one memory device of a plurality of perspectives; With

Be configured to present first perspective, present the processor of the perspective of second storage simultaneously to spectators at least to spectators.

32. system according to claim 31, wherein first perspective is a perspective of record.

33. system according to claim 31, wherein memory device is configured to store automatically at least one perspective.

34. system according to claim 31 also is configured to and presents first perspective side by side store at least one perspective.

35. system according to claim 31 also is configured to and presents second perspective side by side store at least one perspective.

36. system according to claim 31 also comprises and the display of processor coupling, is used to present at least one perspective.

37. system according to claim 36, wherein processor is configured to present in first perspective and second window on display in first window on display and presents second perspective.

38. according to the described system of claim 37, wherein one of first and second windows are nested in another inside of first and second windows.

39. system according to claim 31, wherein receiver is configured to receive many kinds of video flowings that interrelate with a plurality of perspectives.

40. according to the described system of claim 39, wherein every kind of video flowing comprises one of these perspectives.

41. system according to claim 31, wherein receiver is configured to receive many kinds of audio streams that interrelate with a plurality of perspectives.

42. according to the described system of claim 41, wherein every kind of audio stream comprises one of these perspectives.

43. system according to claim 31, wherein memory device comprises at least one cyclic buffer, is used to store at least one of a plurality of perspectives.

44. system according to claim 31, wherein processor is configured at least one of those perspectives of retrieve stored.

45. handle computer program broadcasted content, that comprise computer usable medium for one kind, wherein computer usable medium contains the machine readable code that embodies the following step:

Program receiving broadcasting, this broadcasting comprises a plurality of perspectives of program;

Present first of a plurality of perspectives to spectators;

Store at least one of a plurality of perspectives; With

Present at least one of a plurality of perspectives to spectators.

46., wherein present first perspective and exist and carry out simultaneously according to the described computer program of claim 45.

47. according to the described computer program of claim 45, at least one of wherein storing a plurality of perspectives carried out automatically.

48. according to the described computer program of claim 45, at least one that wherein presents a plurality of perspectives comprises at least one of those perspectives of presenting storage.

49., wherein present at least one of a plurality of perspectives and at least one of a plurality of perspectives of storage and carry out simultaneously according to the described computer program of claim 48.

50. according to the described computer program of claim 49, at least one of wherein storing a plurality of perspectives comprises all perspectives of storage.

51. according to the described computer program of claim 45, wherein receive broadcasting and comprise that receiving many kinds of associated videos flows, every kind of stream comprises one of these perspectives.

52. according to the described computer program of claim 45, wherein receive broadcasting and comprise that receiving many kinds of related audios flows, every kind of stream comprises one of these perspectives.

53. according to the described computer program of claim 45, wherein present second perspective that presents in the different windows that at least one that present first perspective in the window that first perspective is included in display and present a plurality of perspectives be included in display from a plurality of perspectives of storage.

54. according to the described computer program of claim 45, at least one of wherein storing a plurality of perspectives comprises those perspectives is stored at least one cyclic buffer.

Images