HK1083965B - Method and apparatus for scheduling a recording of a television program via an advertisement in a television broadcast stream - Google Patents

Description

Apparatus and method for scheduling television program recording via advertising of television broadcast data streams

This application is a divisional application of patent applications entitled "closed caption tagging system" filed on even 20/9/2000 and under application No. 00815914.9 (international application No. PCT/US 00/25847).

Technical Field

The invention relates to multimedia audiovisual data streams. More particularly, the present invention relates to tagging multimedia audiovisual television broadcast data streams.

Background

Video Cassette Recorders (VCRs) have changed the lives of television viewers worldwide. A VCR can provide time-shift-resilient television programming to suit its lifestyle.

The viewer may use a VCR to store the television program on tape. A VCR enables a viewer to play, reverse, fast forward, and pause for stored program content. The functions can enable the audience to pause the playing program at any time according to the preference of the audience; fast forward bypassing content or commercials that are not intended to be viewed; and replay the favorite data segments. However, a VCR cannot record and play information content simultaneously.

Recently, Digital Video Recorders (DVRs) have entered the commercial market. DVRs enable viewers to store television programs on hard disks. In this way, the viewer can be out of the limit of the tape. The viewer can pause, rewind, and fast forward a live broadcast program. However, the function of the DVR is not limited to recording programs.

Enabling programs to be stored digitally at the local end provides programmers with more options than ever before. Today, advertisements (commercials) can be changed in a dynamic manner and specifically targeted to specific viewers according to viewing habits. The commercial may be stored locally within the viewer's DVR and may be presented at any time.

DVRs enable viewers to broadcast interactive programs. Typically, a preview of the upcoming program will be broadcast during normal broadcast of the program. The viewer must then remember the date, time and channel on which the program was broadcast in order to record or view the program. DVRs enable a viewer to schedule the recording of the program immediately.

The only disadvantage is that: current DVRs are unable to interact with the audience at this level. It is currently not possible to inform the DVR of commercials that follow a particular program or other advertisement. In addition, the DVR is not currently informed of the commercial replaceability.

It would be advantageous to provide a closed caption system that allows program providers to deliver frame-specific data over broadcast media. It would also be advantageous to provide a closed caption tagging system that allows a receiver to interact with a viewer in a dynamic manner and to self-configure based on program content.

Disclosure of Invention

The invention provides a closed caption marking system. The present invention enables content providers to deliver frame-specific data and commands integrated into an av broadcast data stream via a broadcast medium. In addition, the invention enables the receiver to interact with the audience in a dynamic mode and to self-configure according to the content of the video and audio data stream.

The preferred embodiment of the present invention provides a mechanism for inserting tags into an audio or video television broadcast data stream. The tag is inserted into the broadcast data stream prior to or at the time of broadcast. These tags contain command and control information that the receiver interprets and acts upon.

The receiver receives a broadcast data stream and detects and processes tags within the broadcast data stream. The broadcast data stream is a storage device stored on the receiver. Program material from the broadcast data stream is broadcast from the storage device to the viewer.

At the stage of processing the tags, the receiver will perform appropriate actions in response to the tags. These tags provide a large amount of flexible utilization space for content providers or system administrators to construct an infinite number of operational functions.

The tags may indicate the start and end of a program segment. After the viewer presses a button on the remote input device, the receiver responds by skipping a program segment during the broadcast. The receiver may also automatically skip over program segments according to the viewer's preferences.

The receiver automatically replaces a program segment, such as a commercial, with a new program segment. The new program segments are selected according to different criteria, such as: according to the location, time of day, program material, viewing habits of the viewer, preferences of the viewer for the program, or personal data of the viewer. The new program segments are stored at the local or remote end of the receiver.

Menus, icons and web pages are presented to the viewer based on information containing tags. The interaction between the viewer and the menu, icon or web page is through an input device. The receiver performs actions associated with menus, icons or web pages and viewer input. If a menu or action requires the viewer to exit the showing of the program material, the receiver will reserve the exit point and return the viewer to the same exit point again after the viewer has completed the interaction process.

Menus and icons are used to generate product introductions, promotions, and scheduling of recorded programs. A one-touch recording option is provided herein. The viewer may see an icon that informs that an advertised program may be recorded at a future time. The viewer presses a single button on the input device to schedule the receiver to record the program. The receiver may also record the program currently in the broadcast stream in the storage device based on information in a tag.

The tags may be used to build an index of the program material. This allows the viewer to jump to a particular index within the program.

Other aspects and advantages of the invention will become apparent from the following description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.

Drawings

FIG. 1 is a high-level schematic block diagram of a preferred embodiment of the present invention;

FIG. 2 is a schematic block diagram of a preferred embodiment of the present invention utilizing multiple input and output modules;

FIG. 3 is a schematic diagram of a moving picture compression standard (MPEG) data stream and its video and audio components in accordance with the present invention;

FIG. 4 is a schematic block diagram of a parser and four Direct Memory Access (DMA) input engines included within a media switch of the present invention;

FIG. 5 is a schematic block diagram of the components of a packetized elementary data stream (PES) buffer of the present invention;

FIG. 6 is a schematic diagram of the PES buffer structure established by the parser of the media switch output ring buffer;

FIG. 7 is a schematic block diagram of a media switch and various components associated therewith in accordance with the present invention;

FIG. 8 is a high level schematic block diagram of the program logic of the present invention;

FIG. 9 is a category hierarchy block diagram of the program logic of the present invention;

FIG. 10 is a schematic block diagram of a preferred embodiment of a clip cache of the present invention;

FIG. 11 is a schematic block diagram of a preferred embodiment of the analog broadcast studio video mixer of the present invention;

FIG. 12 is a schematic block diagram of a closed caption parser of the present invention;

FIG. 13 is a high level schematic block diagram of a preferred embodiment of the present invention utilizing a VCR as the composite;

FIG. 14 is a schematic block diagram of a preferred embodiment of the present invention for inserting tags into a video data stream;

FIG. 15 is a schematic block diagram of a preferred embodiment of the present invention for server-based tag insertion into a video data stream;

FIG. 16 is a user interface illustrating the use of the present invention to insert tags into a video data stream;

FIG. 17 is a diagram illustrating a screen with a reminder icon, wherein the image is displayed in the lower left corner of the screen, in accordance with the present invention;

FIG. 18 is a schematic block diagram of a transmission path of a video data stream of the present invention;

FIG. 19 is a schematic block diagram of marking the start and end of a program segment of a video data stream and playing a new program segment in accordance with the present invention;

FIG. 20 is a schematic block diagram of a preferred embodiment for interpreting tags inserted into a video data stream in accordance with the present invention;

FIG. 21 is a diagram illustrating a screen displaying program recording options of the present invention;

FIG. 22 is a viewer remote control illustrating the present invention;

FIG. 23 is a schematic block diagram of a series of screens that generate product introduction and promotion.

Detailed Description

The invention is implemented in a closed caption marking system. A system constructed in accordance with the present invention allows a content provider to transmit frame-specific data, and commands, integrated into an AV broadcast data stream over a broadcast medium. The invention also allows the receiver to interact with the audience in a dynamic way and to perform self-setting according to the content of the video data stream.

The preferred embodiment of the present invention provides a marking and interpreting system that enables a content provider to mark a video data stream transmitted via a television broadcast medium with a specific picture. The receiver interprets and acts upon the tags embedded within the received data stream. The tag data enables the receiver to interact with the viewer in a dynamic manner through menus and action icons. The tag also provides for dynamic configuration of the receiver.

Please refer to fig. 1. The preferred embodiment of the present invention includes an input unit 101, a media switch 102, and an output unit 103. The input 101 may receive various forms of television input data streams, such as National Television Standards Committee (NTSC) or Phase Alternating Line (PAL) broadcasts, and digital data streams, such as Digital Satellite System (DSS), Digital Broadcast Service (DBS), or Advanced Television Standards Committee (ATSC) standards. DBS, DSS and ATSC are based on standards such as the active graphics video compression standard 2(MPEG2) and the MPEG2 transmission. The MPEG2 transport standard is a standard that formats a digital data stream from a television source transmitter so that a television receiver can unwrap the incoming data stream to find programs in a multiplexed signal. The input section 101 generates an MPEG data stream. MPEG2 transport multiplexes support multiple programs in the same broadcast channel where video and audio feeds and private data are present. The input section 101 may tune a channel to a specific program, extract a specific MPEG program therefrom, and provide the program to other parts of the system. Analog television signals are encoded into a similar MPEG format using separate video and audio encoders so that the process of acquiring the signal is not known to other parts of the system. Information may be modulated into the Vertical Blanking Interval (VBI) of an analog television signal using a variety of standard methods; for example: the North American Broadcast Teletext Standard (NABTS) may be used to modulate information onto the 10 th to 20 th scan lines of an NTSC signal, while the Federal Communications Commission (FCC) specifies line 21 for Closed Caption (CC) and Extended Data Services (EDS). Such signals are decoded by the input and sent to other components as if they were sent over the MPEG2 dedicated data channel.

The media switch 102 is responsible for coordinating operations between a microprocessor CPU (central processing unit) 106, a hard disk or storage device 105, and a memory 104. The input data stream is converted to an MPEG data stream and passed to the media switch 102. The media switch 102 buffers the MPEG data stream in memory. If the user watches a real-time television, the media switch 102 performs two tasks: the data stream is passed to the output 103 and the data stream is simultaneously written to the hard disk or storage device 105.

The output section 103 receives an incoming MPEG data stream and generates an analog television signal in accordance with NTSC, PAL, or other desired television standards. The output section 103 includes an MPEG decoder, an on-screen display (0SD) generator, an analog television encoder, and an audio logic circuit. The OSD generator enables the program logic to provide an image superimposed on the final analog television signal. In addition, the input section may modulate information provided by the program logic onto the VBI of the output signal using a variety of standard formats, including NABTS, CC, or EDS.

Please refer to fig. 2. The present invention can be easily expanded to accommodate multiple inputs (tuners) 201, 202, 203, 204, and each input can be adjusted to a different type of input. Also, a plurality of output modules (decoders) 206, 207, 208, 209 may be added. Special effects, such as picture-in-picture, can also be implemented using multiple decoders. When a user is watching a television program, the media switch 205 may record another program. This means that: while a data stream is being stored to disk, another data stream may be extracted from disk.

Please refer to fig. 3. The incoming MPEG data stream 301 has interleaved video 302, 305, 306 and audio 303, 304, 307 data segments. These data segments must be separated and recombined to form separate video 308 and audio 309 data streams or buffer information. This is a necessary process because individual decoders are used to convert the MPEG data segments into the original audio or video components. Such separate transmissions must generate time series information to enable the decoder to synchronize correctly for accurate signal playback.

The media switch enables the program logic to associate the correct time series information with each data segment, possibly embedding the information directly into the data stream. The time series information of the respective data segments is called a time stamp. Each time the system is started, the time stamps start with zero and increase in a monotonic manner. This allows the present invention to find any particular point in any particular segment of video data. For example, if the system needs to read five seconds into a streaming continuous video data stream that has been cached quickly, the system need only initiate a forward read and look for the appropriate time stamp.

Binary search can be performed on the stored file to direct to the data stream. Storing each data stream as a series of fixed-size data segments can improve the speed of binary search due to uniform time stamping. If the user wants to start from the middle of the program, the system will continue to perform the binary search on the stored data segments until the system finds the appropriate point, at which time the desired result is obtained with minimal information. If the signal is stored as an MPEG data stream, the data stream must be parsed in a linear fashion from the beginning to find the desired position.

Please refer to fig. 4. The media switch includes four input Direct Memory Access (DMA) engines 402, 403, 404, 405, each with an attached buffer 410, 411, 412, 413. Conceptually, each DMA engine has one pointer 406, a pointer boundary 407, a next pointer 408, and a next pointer boundary 409. Each DMA is affiliated with a specific type of information; such as video information 402, audio information 403, and parsing events 405. The buffers 410, 411, 412, 413 are ring-shaped and collect specific information. The DMA engine increments pointer 406 to the attached buffer until boundary 407 is reached, followed by the next pointer 408 and boundary 409. By setting the values of pointer 406 and next pointer 408 and their corresponding boundaries to be the same, a ring buffer can be created. The next pointer 408 may be set to a different address to provide vector DMA.

The input data stream will pass through parser 401. Parser 401 finds an MPEG event by parsing the data stream, which indicates the start of a video, audio or private data segment. For example, when the parser 401 finds a video event, it directs the data stream to the video DMA engine 402. The parser 401 buffers the data and directs it in a DMA manner through the video DMA engine 402 into the video buffer 410. At the same time, the parser 401 directs an event to the event DMA engine 405, and the DMA engine 405 generates an event in the event buffer 413. When the parser 401 encounters an audio event, it redirects the byte stream to the audio DMA engine 403 and generates an event in the event buffer 413. Similarly, when the parser encounters a private data event, it redirects the byte stream to the private data DMA engine 404 and directs an event to the event buffer 413. When the event is stored in the event buffer, the media switch notifies the program logic via the interrupt mechanism.

Please refer to fig. 4 and 5. Parser 401 fills events into event buffer 413. Each event 501 in the event buffer has fields for an offset 502, an event type 503, and a time stamp 504. After the events are placed in the buffer, the parser 401 provides the type and offset of each event. For example, when an audio event occurs, the event type field is set to audio event and the offset indicates the location in the audio buffer 411. Program logic knows the beginning of the audio buffer 411 and adds an offset to find the event in the data stream. Address offset 502 informs the program logic of the location where the next event occurs, but not the location where the end point occurs. The previous event is stored in the cache so that the end of the current event and the length of the data segment can be found.

Please refer to fig. 5 and 6. When the program logic is interrupted by the media switch 601, the program logic reads the events accumulated in the event buffer 602. From these events, program logic generates a series of logical data segments 603 corresponding to the parsed MPEG data segments 615. The program logic converts the offset 502 into the physical address 610 of each data segment and records the event length 609 using the previous cache event. If the data stream results from the encoding of an analog signal, it will not contain a Program Time Stamp (PTS) value. The PTS value is used by the decoder to provide the resulting result as appropriate. Thus, the program logic uses the generated time stamp 504 to calculate the simulated PTS value for each segment and stores the value in the logical segment time stamp 607. In the context of digital television broadcast data streams, PTS values have been programmed into the data stream. The program logic extracts this information and stores it in the logical data segment time stamp 607.

The program logic continues to collect logical data segments 603 until it reaches the fixed capacity of the buffer. When the capacity limit is reached, the program logic creates a new buffer called a Packetized Elementary Stream (PES)605 buffer containing sequentially arranged logical data segments 603 and accompanying control information. Each logical data segment is directed to a ring buffer 604, such as a video buffer 613 filled by the media switch 601. The new buffer is then transferred to other logic elements which may further process the data stream in the buffer in some manner, such as decoding or writing it to a storage medium. Thus, the processor does not copy the MPEG data from one location to another within the memory. Since both the required memory bandwidth and the processor bandwidth have been reduced, a more cost effective design is achieved.

The unique property of MPEG data stream conversion into PES buffers is that: the data accompanying the logical data segment need not be present in the buffer itself as described above. When the PES buffer is written to the storage medium, the logical data segments are written to the storage medium where they appear in logical order. This results in the components of the data stream being collected in a single stream data linear buffer on the storage medium, regardless of whether the components of the data stream are located in a video, audio, or private data circular buffer. The buffer is read back from the storage medium in a single transfer and the logical data segment information is updated to correspond to the actual location in the buffer 606. The higher level program logic does not know about the conversion process because it only processes logical data segments; thus, streaming data can be easily handled without requiring duplication of data between locations in a Dynamic Random Access Memory (DRAM) via a CPU at all.

One of the unique technical features of the media switch is its ability to process high speed data in an efficient and cost effective manner. The media switch may perform the following functions on a low cost platform: recording video and audio data, sending out video and audio data, sending video and audio data to a disk, and retrieving video and audio data from the disk. Typically, media switches operate asynchronously and independently of the microprocessor's CPU and use its DMA capabilities to move large amounts of data with minimal CPU involvement.

Referring to fig. 7, the input terminal of the media switch 701 is connected to the MPEG encoder 703. Also included are circuits dedicated to the MPEG audio data 704 and VBI data 702 provided to media switch 701. If digital television signals are being processed, the MPEG encoder 703 is replaced by an MPEG2 transport splitter, and the MPEG audio encoder 704 and VBI decoder 702 are eliminated. The splitter multiplexes the extracted audio, video and dedicated data channel streams through a video input media switch port.

The parser 705 is used to parse the input data stream from the MPEG encoder 703, the audio encoder 704 and the VBI decoder 702, or from the transport splitter in the case of a digital tv broadcast data stream. The parser 705 detects the start of all important events, frames and sequence headers in the video or audio stream, all of which are necessary for the program logic to correctly play the stream and perform special functions, such as: fast forward, rewind, play, pause, fast/slow play, index, and fast/slow reverse play.

When parser 705 recognizes a video, audio data segment, or given private data, it places tag 707 in first-in-first-out buffer 706. When these tags are fetched, the DMA 709 is controlled. Tag 707 and the DMA address of the segment are placed in event queue 708. Whether frame type information is a video I-frame, a video B-frame, a video P-frame, a video packet elementary stream, an audio packet elementary stream, a sequence header, or an audio frame or a private data packet is stored in the event queue 708 along with an offset stored in the associated ring buffer in which the information is stored. After program logic is transferred to DRAM 714, program logic operating in CPU713 checks for events in the ring buffer.

Media switch 701 has data bus 711 connected to CPU713 and DRAM 714. Address bus 712 is also shared among media switch 701, CPU713, and DRAM 714. A hard disk or storage device 710 is connected to one port of media switch 701. Media switch 701 outputs the data stream to MPEG video decoder 715 and separate audio decoder 717. The audio decoder 717 signals include audio prompts that are generated by the system in response to user commands on a remote control or other internal events. The decoded audio output by the MPEG decoder is digitally mixed 718 with the separate audio signal. The resulting signal, which includes video, audio, and on-screen display, is transmitted to the television 716.

The media switch 701 takes the 8-bit data and sends it to the disk, while another data stream is taken from the disk and sent to the MPEG decoder 715. All of the above described DMA engines can operate at the same time. Media switch 701 may be implemented in hardware using a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), or discrete logic circuitry.

Program logic need only look up the ring event buffer in DRAM 714 to identify the start location and type of each frame, rather than having to parse a large data stream to find the likely start location of each frame. This process saves a significant amount of CPU processing power and minimizes the real-time requirements of CPU 713. CPU713 need not have a very high rate at any one time. Media switch 701 provides the CPU713 with as much time as possible to complete the job. Parser mechanism 705 and event queue 708 eliminate the need for CPU713 to analyze audio data, video data and buffers, and the real-time nature of the data stream in a low cost manner. This approach also allows the use of very low clock rate buses in a CPU environment and works with less expensive memory than otherwise.

CPU713 has the function of queuing a DMA transfer and may establish the next DMA transfer when it is idle. This allows CPU713 a significant amount of time to servo DMA controller 709. Due to the large latency allowed, CPU713 may respond to DMA interrupts within a large window of time. Whether an MPEG data stream extracted from an MPEG2 transmission or encoded from an analog television signal, is typically encoded using a technique known as Variable Bit Rate (VBR) encoding. Such techniques change the amount of data needed to represent the sequence of images to the amount of movement between the images. This technique can significantly reduce the bandwidth of the signal, and fast moving sequences (such as basketball games) can be encoded with a larger bandwidth. For example, a Houss direct broadcast television satellite system encodes at a desired bandwidth at a rate of 1 to 10 megabits per second and varies from frame to frame. Without this structure, it is difficult for any computer system to keep up with such fast changing data rates.

Referring to fig. 8, the program logic in the CPU has three conceptual components: source 801, translation 802, and sink 803. Source 801 generates a data buffer. The translation 802 processes the data buffer while the sink 803 consumes the data buffer. The translation is responsible for the allocation of the data buffer on which it operates and for queuing. The buffer is allocated to the data source as "empty" and fed back as "full". The buffer is then queued and provided to the sink in a "full" state, while the sink will return with the buffer in a "empty" state.

Source 801 receives data from an encoder, such as a digital satellite receiver. The source takes the buffer of data from the downstream translation and seals the data into the buffer, and then pushes the buffer into the transmission channel in the manner described above. The source object 801 is not aware of other conditions with respect to the present system. The sink 803 consumes the buffer and takes the buffer from the up-conversion and transfers the data to the decoder, which then releases the buffer for reuse.

Two kinds of conversions 802 are used here: spatial and temporal transformations. Examples of spatial transformations are image convolution or compression/decompression as the buffered data passes through. The time transition is for when there is no representable relationship between the buffered data entering the system and the buffered data exiting the system. The conversion is to write the buffered data to a file 804 located in the storage medium. The buffered data is later retrieved and transmitted along the transmission channel and properly sequenced in the data stream.

Referring to FIG. 9, a C + + class hierarchy derived from program logic is shown. The TiVo media cores (Tmk)904, 908, and 913 transit the operating system cores. This core provides operational functions such as memory allocation, synchronization, and threading. TmkCore 904, 908, and 913 target memory from the media core, which may be provided to the operator for creating new targets or deleting targets. Each object (source 901, transform 902, and sink 903) has been defined as multi-threaded and can operate in parallel processing.

The TmkPipeline classes 905, 909, and 914 are responsible for controlling the traffic through the system. The transmission channel points in traffic from the source 901 to the sink 903 to the next transmission channel. For example, if the transmission channel is to be paused, an event called "pause" is sent to the first object in the transmission channel. The event is forwarded all the way along the transmission path to the next object. This process is performed in an asynchronous manner for data passing through the transmission channel. Thus, in applications such as telephony, the flow control of MPEG data streams is unsynchronized and separated from the data stream itself. This allows for the use of simple logic designs while having sufficient capacity to support the previously described features including pause, rewind, fast forward, and other functions. Furthermore, this structure allows for a fast and efficient switching between the sources of the data streams, since the buffered data is simply discarded and the decoder is reset with a single event, after which the data from the new data stream will pass through the transmission channel. This may be required, for example, when the input captures a channel switch, or when switching between a live broadcast signal from the input and a stored data stream.

The source object 901 is TmkSource 906 and the transform object 902 is TmkXfrm 910. The above are intermediate level categories that define standard behavior among the transmission channels. Conceptually, it exchanges buffers with the transmission channel. The source object 901 is a device that fetches data (e.g., media switch) from a physical data source and stores it in a PES buffer. To obtain this buffer, the source object 901 requests a buffer (allocastybuf) from the downstream object in the transmission channel. The source object 901 may be closed until sufficient memory is available. This means that the transmission channel is self-adjusting; with automatic flow control. When the source object 901 fills the buffer, it will hand the buffer back to the transform 902 through the pushFullBuf function.

The sink 903 is also flow controlled. The sink calls nextFullBuf to tell the transition 902 that it is ready for the next full buffer. This action may close the sink 903 until a buffer is ready. When the sink 903 completes a buffer (i.e., it has consumed the data in the buffer), it calls releaseEmptyBuf. The ReleaseEmptyBuf hands the buffer back to the translation 902. For example, the conversion 902 may then hand the buffer back to the source object 901 to be refilled. In addition to the advantages of automatic flow control, this approach also limits the amount of buffer-specific memory by allowing a forced translation buffer fixed configuration. This is an important feature to achieve cost-effectiveness in a limited DRAM environment.

The MediaSwitch class 909 calls Tmk the allocEmptyBuf method that clips the fast cache 912 object and receives the PES buffer from it. The ring buffer is then left in the media switch hardware and a PES buffer is generated. The MediaSwitch class 909 fills the buffer and pushes it back to Tmk to clip cache 912 object.

Tmk clip cache 912 retains a cache file 918 in the storage medium. Tmk clip cache 912 also retains two pointers in this cache area: a push pointer 919 that may show where the next buffer from source 901 is inserted; a current pointer 920, which points to the buffer currently in use.

The buffer to which the pointer is currently pointing is processed through the sail type decoder class 916. The sail type decoder class 916 communicates with the decoder 921 in hardware. Decoder 921 produces a decoded television signal that is subsequently encoded into an analog NTSC, PAL, or other analog format television signal. After the sail type decoder class 916 completes the buffer, it will call releaseEmpthBuf.

Various architectures may make the system easy to test and debug. Each level can be tested individually to ensure that it performs in an appropriate manner, and the various classes can be gradually combined to achieve the desired functionality while maintaining the ability to effectively test each object.

The control object 917 is used for receiving instructions from a user and sending events to the transmission channel to control the execution status of the transmission channel. For example, if the user has a remote control and is watching TV, the user presses pause, and then the control object 917 sends an event to the sink 903 to inform the pause. The sink 903 will stop requesting a new buffer. The pointer 920 now stays at its location. When the sink 903 receives another event informing it to play, the sink 903 will start to extract the buffer again. The system is in perfect synchronization and the fetching starts where the frame stops.

The remote control may also include a fast forward button. When the fast forward key is pressed, control object 917 sends an event to transition 902 to tell it to move forward for two seconds. Transition 902 finds that a two second length of time requires three buffers to move forward. It then issues a reset event to the downstream channel so that any data in the queue or state that may be present in the hardware decoder is forced out. This is a critical step because the structure of the MPEG stream requires that the state of multiple frames must be maintained, which will become invalid due to the relocation of pointers. Which then moves the current pointer 920 forward three buffers. The next time the sink 903 calls nextFullBuf, it will get a new current buffer. The same method can be applied to the fast rewind function, which is accomplished by moving the current pointer 920 backward with transition 902.

The system clock reference is located in the decoder. The system clock reference is accelerated at fast-play drop and decelerated at slow-play drop. The sink requires the entire buffer to be either sped up or slowed down, depending solely on the clock rate.

Referring to FIG. 10, two other objects derived from the TmkXfrm class are stored on the transport channel to facilitate disk access. One of which is referred to as Tmk clip reader 1003 and the other is referred to as Tmk clip writer 1001. The buffer enters Tmk the clip writer 1001 and is pushed into a file located on the storage medium 1004. Tmk clip reader 1003 requires a buffer of a file taken from storage medium 1005. Tmk clip reader 1003 provides only allocEmptyBuf and pushFullBuf methods, while Tmk clip writer 1001 provides only nextfullbuf and releaseEmptyBuf methods. Thus, the Tmk clip reader 1003 performs the same function as the "push" end of the input or Tmk clip cache 1002, while the Tmk clip writer 1001 performs the same function as the "pull" end of the output or Tmk clip cache 1002.

Referring to fig. 11, a preferred embodiment for achieving multiple functions is shown. The source 1101 has a television signal input. The source passes the data to the push switch 1102, which is a switch derived from TmkXfrm. The push switch 1102 has a plurality of outputs and is switchable by a control object 1114. This means that a part of the transmission channels can be stopped and the other transmission channels can be activated at will according to the user. The user can switch among different storage devices. The push switch 1102 may output to Tmk a clip writer 1106, which goes to storage 1107 or to write to fast cache transition 1103.

An important feature of the device is that it allows easy selection of the recording portion of the incoming signal under the control of program logic. Tmk clip writer 1106 may be switched to record a portion of the signal and shut off at some later time, depending on information such as the present time, a particular length of time, or via a viewer pressing a remote control. This toggling typically causes a "toggle" event to be sent to the push switch 1102 object.

Another method for enabling selective recording is by modulating the information to the VBI, or depositing to an MPEG private data channel. Data decoded from the VBI or the dedicated data channel is sent to program logic. Program logic examines the data to determine whether the data indicates that recording of the modulated television signal should begin. Similarly, the information may also indicate when the recording should terminate, or another data item may be tuned to a signal indicating when the recording should terminate. The start and stop indicators may be explicitly modulated into the signal or other information stored in the signal in a standard manner may be used to encode this information.

Referring to fig. 12, an example is shown of how program logic scans words contained in a Closed Caption (CC) field to determine the times of start and stop and to initiate recording with a particular word or phrase. Including the data stream of the NTSC or PAL field 1201. CC bytes are extracted from each odd field 1202 and added to a ring buffer 1203 for processing by a word parser 1204. Word parser 1204 collects words until a word boundary is encountered, which is typically a blank word, period, or other descriptive character. Recall that the MPEG audio and video data segments are collected into a series of PES buffers of fixed size. A special data segment is added to each PES buffer to hold the words extracted from the CC field 1205. Therefore, the information of the CC will be synchronized with the audio and video, and can be presented to the viewer correctly when the data stream is played. This also allows the stored data stream to be processed for CC information while the program logic is idle, which distributes load, reduces cost, and increases efficiency. In this case, the words stored in the special data field may be sent directly to the state table logic 1206.

During recording of the data stream, each word is locked in a table 1206 that indicates the identification action to be performed for that word. This action may simply be a change to the state of the recognizer state machine 1207, or may cause the state machine 1207 to issue an action request, such as "start recording," "stop recording," "view phrase," or other similar request. Indeed, the recognized word or phrase may cause the transmission channel to be switched; for example, if an unwanted language is used in the program, different soundtracks may be overlaid.

It should be noted that the resolution state table 1206 and the recognizer state machine 1207 may be modified or changed at any point in time. For example, different tables and state machines may be provided for each input channel. In another case, the cells may be switched based on time of day or other events.

Referring to fig. 11, a pull-out switch 1104 is added, which is output to a sink 1105. The sink 1105 calls nextFullBuf and releaseEmptyBuf to fetch or return the buffer from the pull switch 1104. The pull switch 1104 may have any number of inputs, one of which may be a clipping action 1113. The remote control may switch between input sources. The control object 1114 sends an event to the pull switch 1104 to signal it to switch from the current input source to any input source selected by the control object.

The clipping action is provided for arranging a number of different stored signals in a predictable and controllable manner and possibly incorporating control selected by the viewer via a remote control. It is therefore presented in the form of a derivative of the TmkXfrm object and receives a "switch" event for switching to the next stored signal.

This allows program logic or the user to create custom video output sequences. Any number of video data segments may be arranged and combined, as with program logic or user usage of a broadcast studio video mixer. Tmk clip readers 1108, 1109, and 1110 are assigned and hang in pull switch 1104. Pull switch 1104 switches between Tmk clip readers 1108, 110g and 1110 to combine the video and audio clips. Due to the construction of the transmission channel, the flow control is automated. Push and Pull switches are just like the video switch of a broadcast studio.

The derived classes and resulting objects described herein may be combined in any manner to create a number of different and useful configurations for storing, retrieving, switching and viewing television broadcast data streams. For example, if multiple outputs and inputs are available, one input may be viewed and another input stored, while a window of the image in the image may be generated by the second output to preview the previously stored data stream. These configurations represent a unique and novel application of software transformation that can achieve the results of expensive and complex hardware solutions with a single cost-effective device.

Referring to fig. 13, a high level system diagram is shown, and is shown implemented with a VCR backup appliance. The output module 1303 transmits the television signal to the VCR 1307. This allows the user to record the television program directly on the video tape. The present invention allows the user to queue up programs to be recorded from disk to videotape and to schedule the time that the program is sent to the VCR 1307. The title page (EPG data) may be fed into the VCR1307 before a program is transmitted. Longer programs can be adjusted to fit smaller video tapes by speeding up the playback rate or omitting frames.

The output of the VCR1307 may also be directed back to the input module 1301. In this configuration, the VCR serves as a backup system for the media switch 1302. All excess storage or lower priority programming will be sent to the VCR1307 for later retrieval.

The input module 1301 may decode and pass on to the rest of the system information encoded in the VBI. The output module 1303 can be encoded into the output VBI data provided by the rest of the system. The program logic may be arranged to program different kinds of identification information into the output signal which is recorded on the tape using the VCR 1307. Playback of the tape allows program logic to read back the identification information so that the television signal recorded on the tape can be properly processed. For example, a particular program may be recorded on tape along with information about when it was recorded and the source network. When the program is played back to the input module, this information can be used to control the storage of signals and presentation to viewers, etc.

As will be readily apparent to those skilled in the art, this mechanism may be applied to introduce various data items into program logic that are not considered television signals. For example, software updates or other data may be added to the system. Program logic receiving this data from the television broadcast data stream may enforce control of the data processing scheme, such as requiring some authentication procedure and/or decrypting embedded information based on some pre-mastered key. This method can also operate in conventional broadcast signals, introducing an efficient way of providing non-television control information and data to program logic.

In addition, it will be readily apparent to those skilled in the art that although described above with particular reference to a VCR, any multimedia recording device, such as a digital video disc-random access memory (DVD-RAM) recorder, can be readily substituted for a VCR.

Although the present invention has been described herein with reference to the preferred embodiments, those skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. For example, the present invention can be applied to detecting criminal behaviors in a gambling location. The input unit of the present invention is connected to a photographing monitoring system of a casino. The recorded video data is quickly buffered and simultaneously output to an external VCR. While the external VCR is loading real-time input video, the user can switch to either image feed to view (e.g., rewind, play, slow play, fast play, etc.) a particular segment of the recorded image.

Architecture for video data stream tagging

Please refer to fig. 12 again. Tags are abstract events that occur in the tv broadcast data stream 1201. These tags may be embedded in the VBI of an analog signal or in the MPEG2 multiplexed private data channel. As described above, tags may be embedded in the Closed Caption (CC) field and fetched into the ring buffer 1203 or memory allocation structure. The word parser 1204 identifies unique tags during the scanning of the CC data. These tags are decorated with standard CC control codes. Meanwhile, the label can be generated in a recessive way; for example: generated based on the current time and the program being viewed.

The present invention provides a system known as TiVo Video Tag Authoring (TVTAG) for inserting tags (TiVo tags) into a video data stream prior to broadcast. Referring to fig. 14, 16 and 17, the TVTAG system comprises a video output source 1401, a compatible device 1402 for inserting closed caption information into the VBI and outputting subtitled video, a video monitor 1405, and a software program for controlling the VBI inserting device to include a subtitled tag data object in the form of closed caption information into video data stream 1406. The tagged video data is immediately transmitted or stored on a suitable medium for subsequent transmission.

In most basic implementations, the TVTAG software 1406 is responsible for controlling the VBI insertion device 1402. The TVTAG software 1406 interfaces with the VBI insertion device 1402 via a standardized computer interface and device control code protocol. When an operator viewing the video monitor 1405 determines that the desired tag insertion point has been reached, he presses a key to generate a TiVo tag data object, which is simultaneously transmitted to the VBI inserting means 1402 and included in the video data stream for transmission 1404 or storage 1403.

The TVTAG software has the additional function of controlling the video input source 1401 and the video output storage device 1403. The operator selects a particular video 1602 and can pause the video input stream to help enable the image element 1702 to be overlaid on the monitor and positioned using a pointing device, such as a mouse. The positioning of the image elements 1702 may also be accomplished by the operator interface 1601. The operator inputs the image position using the X position 1605 and the Y position 1604.

The image elements and positioning information are then incorporated into the TiVo tag object (described below) and the timecode or video frame is recorded. When the operator is satisfied, the playback and recording can be resumed. The most accurate tag is then issued by the insertion device.

Please refer to fig. 15. In another preferred embodiment of the TVTAG system, the software program takes a web page of a standard internet protocol and displays the web page 1505 for the operator. The web page may cause a script executed on the remote server 1504 to generate the TiVo tag object. The server 1504 controls the VBI inserting apparatus 1502, the video source 1501 and the recording apparatus 1503. A remotely located operator 1505 can obtain a low bandwidth or high bandwidth version of a video data stream from the server 1504 as a reference for inserting a tag. Once the necessary tag data objects have been generated and transmitted, they may be later batched by the server 1504.

In another preferred embodiment of the present invention, it integrates software and a widely used non-linear video editing system into an "insert" object, thereby allowing the TiVo tag data object to be inserted during the process of generating video. In this embodiment, the non-linear editing system serves as the source and storage system controller, while also providing the function of placing images that allow for the placement of TiVo tag data objects with precise frames.

Please refer to fig. 18. The tags are integrated into the video data stream prior to or upon video source 1801. The video data stream is then transmitted via satellite 1802, cable or other terrestrial transmission methods 1803. The receiver 1804 receives the video data stream, identifies the tags, and performs appropriate actions in response to the tags. The viewer views the resulting video data stream via a monitor or television 1805.

The present invention provides an architecture that supports actions to be taken based on tags of a video data stream. Examples of elastic applications provided by TiVo tags include:

it is desirable to know when to watch a program preview of the broadcast network so that the viewer can choose to record a future upcoming program. The TiVo tag attached to the trailer will indicate the date, time and channel the program was broadcast. Details regarding active program announcements are described in detail below.

A common problem occurs in the case of extended baseball games. A VCR or Digital Video Recorder (DVR) shuts down the recording process whenever the baseball game time exceeds the time to enter the ad. The TiV0 tag is transmitted in the video data to indicate that the recording process must continue. When the race is over, the TiVo tag is also transmitted to tell the system to stop recording.

The boxing match often ends abruptly, thus causing the VCR and DVR to record alternate programs for the remaining reserved time. The TiV0 tag is transmitted to indicate that the program has ended and to inform the system to stop recording.

Please refer to fig. 19. The advertisements are tagged to facilitate the broadcasting of advertisements stored locally or remotely, thereby replacing advertisements that are national or regional. In video data stream 1901, the tags that are overlaid on program segments 1902 (commercials or other data segments) are tagged with a technique such as the TVTAG system described above. The TiVo tag will inform the invention 1905 about the start and end of the old program segment 1902. Separate tags 1903 may be attached to inform the invention 1905 about the duration of the old program segment 1902 or tags may be attached to the start and end points 1903, 1904 of the old program segment to indicate the start and end points of the data segment 1902. When the tag is detected, the present invention 1905 searches for a new program segment 1906 and broadcasts a new data segment directly to replace the old program segment 1902, and when the broadcast is completed, it reverts to the original program 1901. The viewer will not experience the above-described conversion process.

There are three options at this time:

1) the system 1905 may continue to cache the original program quickly; thus, if viewer 1907 reverses program 1901 and plays it back again, it will see overlapping data segments;

2) old program segment 1902 is also replaced in the fast cache; thus, the viewer will not see overlapping data segments; or

3) The system may cache the original data segment 1902 quickly and re-interpret the tags as played. However, without the intelligent tag prefetching function, this function works only if the viewer has sufficient backup to make the system aware of the first tag in the overlapped data segment.

The above problem can be solved by adding the length of the old program segment to the start 1903 and end 1904 tags. Another way of processing is tag matching, so that the system identifies the end tag 1904 from the start tag 1903. When the system 1905 performs a fast forward or reverse pass on one of the tags, the system 1905 can learn that it must look for the other tag. The tag 1903 and 1904 pair contains a unique identifier. The system 1905 can then search forward or backward for paired tags and replace the old program. The system performs tag prefetching with a limit on its time or frame length. This limitation is included in labeling or standardization. The inclusion of a boundary in a label is the most flexible approach.

For example, the program segments to be played are selected according to location, time of day, program content, or according to a preference engine (described in U.S. patent application No. 09/422,121 owned by the applicant of the present application). Using the preference engine, the appropriate program 1906 from the local or server store may be selected based on the viewer's personal data. The personal data contains the viewing habits of the viewer, program preferences, and other personal information. The stored program segments 1906 also have program objects describing their characteristics, which are used to search for the best-fit preference guidance.

Clearly, there must be a round-robin mechanism in the commercials to avoid having the commercials broadcast. The preference guide may be further biased to create a bias trend for the currently viewed program relative to the program data, and this bias trend may be used to bias the commercial schedule to match the on-disk program 1906. For example, if a viewer is watching a soap opera and the viewer's preference is toward sports, the present invention will select a beer advertisement instead of a diaper advertisement.

Labels may also be used for formation condition selection. The tag has its own way of measuring preference. In this case, the preference measure would be compared to the preference trend, and if the comparison shows a high correlation, the present invention would not alter the commercial. If the comparison result is low correlation, the method is triggered to be applied

It should be noted that in all cases, the system 1905 has sufficient time to make the selection. The transmission channel structure will typically cache 1/2 seconds of video data to provide much time between input and output to alter the data stream. If longer time is required, more buffers may be added for the transmission channel. When the program in the disc is broadcast, the system will create the same length of time delay by reading the previous data stream.

It should also be noted that the above method can also be used to detect commercials, which is described in U.S. patent application No. 09/187,967 entitled Analog video tagging and Encoding System, which is also owned by the present applicant. The same type of substitution as described above can also be applied when using the labels described in the earlier-published application.

Please refer to fig. 19 and 22. The tag may perform the commercial "fast move/skip" function. Tags may be used to indicate the start 1903 and end 1904 of a commercial, so that it may be skipped or taken ahead of time. The viewer simply presses the jump key 2205 on the remote control 2201. The system searches for the end point tag and replays immediately following the frame that accompanies the tag. The number of commercials skipped depends on the amount of buffered video data stream.

The system 1905 may skip itself through advertisements for the currently broadcast program, or through commercials for programs that are pre-recorded and stored in the memory 1906, depending on the preference settings subscribed by the viewer. If a commercial in a currently broadcast program is to be skipped, then a large amount of buffering must be available in the channel in accordance with the above-described method. Allowing the system to skip commercials recorded in the program allows the viewer to view the continuously broadcast program without interference from any commercials.

Tags attached to the program may be used as an index. For example, the viewer may jump to various indexes in the program by pressing jump key 2205 on remote control 2201.

Tags can also be used for system functions. As described above, the system may store the program content locally for its own use. The system 1905 must somehow retrieve the program material. Tuning to a particular channel at an off-line time can accomplish this. The system 1905 searches the data stream 1901 for a tag that tells it to start recording. The recorded content includes a number of program segments defined by tags 1903, 1904 that identify content or preference tendencies. The tag located at the end of the data stream tells the system 1905 to stop the recording process. The program segments 1906 are stored locally and are available for subsequent indexing in the manner described above.

The invention has the following design points:

the present design provides a clear mechanism and strategy for partitioning.

From an internal perspective, a tag can be considered an abstract event that triggers a policy module. The source channel object is responsible for mapping the received tag information to these internal abstractions.

The abstract tag is stored in the PesBuf data stream as if it were another piece of data. Any size tag with accurate time information can be processed in this way. It also allows the tag to survive as part of the recording so that appropriate action can be taken whenever the program is viewed.

The tag can update information about the current program as well as future programs. Such information is reserved for recorded programs.

The tags may be recorded as they pass through the system. In addition, the information can also be uploaded. It is not necessary to retain all the information accompanying a certain tag.

The tags may be generated according to a separate schedule. For example, the tag is generated by a broadcasting network workstation recording according to time and viewing broadcasting network. The tags generated according to time will remain in the recorded stream.

Time-based labels

Referring to fig. 20, time-based tags are processed by a time-based tag identifier 2012. This object 2012 listens to channel change events and when switching to a known broadcast network, the object receives a "time record" for that broadcast network. If there is a certain time record, the object 2012 will establish a tag schedule according to the current time. When the time for each tag occurs, object 2012 communicates an event to source 2001 to indicate that the tag has been inserted. The source object 2001 then inserts the tag into the next available location within the currently constructed PesBuf. The next "available" position may be determined based on frame boundaries or other conditions.

Role of Source object

The source object 2001 is responsible for inserting tags into the PesBuf data stream it processes. It is assumed here that there are source objects for the analog input and the digital television source, respectively.

Tags can appear in analog data streams in several different ways:

-in an EDS field.

Implicit use of the CC field.

Modulation to VBI, perhaps using ATVEF specification.

Time-based approach.

In a digital television broadcast data stream, or after conversion from analog to MPEG:

in-band, the TiVo tagging technique is used.

MPEG2 private data channel.

MPEG2 data stream characteristics (such as frame boundaries, etc.).

Time-based tags.

The source object 2001 is not responsible for resolving the tags and taking any action. Rather, the source object 2001 should only be responsible for recognizing potential tags in the data stream and adding them to the PesBuf data stream.

Identification and action of tags

Conceptually, all tags can be divided into two broad categories: tags that must take action when received (e.g., recording a program) and tags that must take some action when displayed-i.e., viewing a program.

Processing of received tags

The tags that must take action upon receipt are processed as follows: a new receive label mechanism subclass 2003 is established Tmk for the push switch class 2002. When the incoming data stream passes through this sub-class 2003 between the source object 2001 and the program cache transition 2013, the sub-class 2003 may recognize the receive tag and take appropriate action.

The receiving tag is typically processed once and then deactivated.

Processing of display labels

The labels that must take some action when displayed are processed as follows: a new display tagging mechanism subclass 2007 is established for the TmkPullSwitch class 2008. As the output data stream passes through this sub-class 2007 between the program cache transition 2013 and the sink object 2011, the sub-class 2007 may recognize the display tag and take the appropriate action.

Processing of tag program

The processing of the reception tag is only performed when the tagregepurationpolicy object 2009 appears in the current channel. The process of displaying the tag is performed only when the TagPresentationPolicy object of the source channel exists.

The TagPolicy object describes which tags are to be recognized and which actions are allowed.

When the input channel changes, the receive tag object is notified and the object fetches the tagrefepetionpolicy object 2009 for that channel (if it exists) and follows the procedure that has been defined.

When the outgoing channel changes, the tab object is notified and the object takes the TagPresentiationPolicy object 2010 for that channel (if this object exists) and follows the defined procedure.

Label recording

The receipt of the tag may be recorded in a database. This action only occurs when there is a tagrefepetionpolicy object 2009 and the tag is set to record attribute. For example, the record attribute may have been set, but is still not allowed to perform the receiving action. Passive recording of actions within the input data stream is thus achieved.

Change of channel processing

It is important to support data update regarding the current presentation content. The following strategies are proposed:

-whenever the input source changes or a new presentation starts, a sub-part of the presentation object is made and the other operations within the channel complement the sub-part.

-the update tag belongs to the receive tag; and updating the copied showing object when the strategy allows.

If the current show content is to be recorded, the copy of the show object is stored with it so that the stored program is provided with the appropriate information to be stored with it.

The video recorder must be aware that the presentation object has changed so that it does not prematurely cut off the video recording of, for example, a baseball game.

Tag interpretation and tag state machine

The use of tags is very flexible because once the TagPolicy object has been used to identify a valid tag, the tag interpreter 2005 interprets the standardized abstract tag, while the TiVo tag state machine 2006 executes the operational tags. The interpreted tag will initiate a predefined set of actions. Each set of actions is pre-programmed into the system.

The state machine tag belongs to an operational tag, which does not carry executable code, but executes program steps. This allows the tag originator to combine such tags to perform a specifically tailored action on the TiVo system. The state machine tag can achieve the same result as an interpreted tag, but it has the flexibility to dynamically change the set of actions performed.

Abstract interpretation tag

The set of abstract tags that can be employed is defined in a list called a tag/action list. Such lists are typically stored within database objects; where there are several defined abstract labels. These actions can be classified into three categories:

-actions (which may include interactions) viewable by the viewer.

Meta information about the data stream (e.g. channel, time, length of play, etc.).

TiVo control tag.

The label that causes the replacement of the on-disc database or the implicit recording must be authenticated. This is done by controlling the tags.

Tag viewable by a viewer

-menu

Such a tag indicates that a selection is to be provided to the viewer. The data accompanying such a tag indicates what the options are, as well as other relevant data such as the display style. The menu has the function of time-out deactivation.

The menu tab is intended to provide a selection for the viewer. If the viewer is not present or interested, the menu should disappear immediately. The operation principle of the menu can optionally pause the currently played program. The menu is not displayed as a list.

-conditionally deducing another program

Such labels represent: when a condition is true, another program should be broadcast. This condition is analyzed by the policy module. The condition may often be true.

-conditionally pop up another program

Such a tag will revert to the previous program. When a program ends, the stack of another program is automatically popped up. If the channel has changed or the viewer enters the TiVo central menu area, all other programs are pop-up.

Highlighting other programs is a method of inserting arbitrary sequences into the viewed program. No condition data was evaluated at the uppermost level. Instead, the policy module must examine this data for selection. For example, this method can be used to form "nested" advertisements.

-conditional display indicator

Such a label will cause an indicator to appear on the screen. The indicators will be named and the set of indicators in operation can be queried at any time. Such a tag or tag policy may indicate a timeout value for the boot indicator.

-conditional clear indicator bar

Such a tag clears the indicator in operation. If the channel has changed or the viewer enters the TiVo central menu area, all indicators are cleared.

An indicator is another way to provide the viewer with a choice without interrupting the showing of the program. Indicators may also be used to indicate conditions of a data stream that may be of interest. For example, an "active program guide (ActivePromo)" may be created by providing a program object identification code as part of the tag data so that the program may be selected. When the indicator is displayed, the program is scheduled to be recorded if the viewer presses a particular button.

Meta information label

Current broadcast information

Such tags are typically used to contain information about the currently broadcast program. Each tag typically links some item of information such as a start time, end time, duration, etc. Such tags may be used to "extend" the recording of an event.

Future broadcast information

Such a tag is similar to the one described above, but it contains information about future broadcasts. There are two relevant cases here:

the information is about a certain broadcasting information already existing in the database. The database object is updated in time.

The above information is broadcast information that does not exist. At this point a new playback object is created and initiated by the tag.

TiVo control label

-authorization modification

Such tags are typically encrypted using the current month's security key. The lifetime of this certification is set by the operating principles, which may be one or two hours. Therefore, if the state of the local TiVo system is allowed to be changed, the tag must be continuously rebroadcast.

The purpose of such tags is to stop the deliberate (or accidental) breach of the security by means of an intrinsic non-secure tag mechanism such as Extended Data Services (EDS). If the broadcast network provides EDS information, it should first be determined that the tags provided by the broadcast network are correct and that corruption of the tag transmission system is unlikely to occur. Secondly, it should cooperate with the broadcast network to provide an authentication system that only transmits the authentication tag in the broadcast network. Unauthenticated tags can never be inserted into the PES stream through the source object.

-conditionally recording the current program

Such tags cause the currently viewed program to be stored on disk from this point. Recording is stopped when the program ends.

-conditionally stopping recording of the current program

Such tags will stop recording the currently viewed program.

-conditional recording of future programs

Where the play object identification is provided (perhaps directly in the future play ticket). The program may be scheduled for recording at a background priority lower than the viewer's explicit choice.

Conditional cancellation of recording of future programmes

Where the play object identifier is provided. If the object is scheduled for recording by the previous tag, the recording is cancelled.

These tags and future play tags may be inserted in an encrypted secure format. If these tags have been properly verified, the source object will only insert these tags into the PES stream.

One use of such tags is to automatically launch the TiVo directory-e.g., loop set, advertisement, neutral, etc. A later download will cause this directory to be "installed" and made available.

-storing files conditionally

Such tags are used to transfer data within a data stream for storage within a disk. For example, a broadcast web page would pass through this mechanism.

-storing objects conditionally

Such tags are used to transfer objects within a data stream for storage in a disk. The stored procedure for the object may follow standardized object update rules.

The following is an example of an implementation using a presentation tag inserted in the Closed Caption (CC) portion of the data stream. The CC portion of the data stream is selected because a signal is retained before it is transmitted to the user receiver and decoded. The rest of the VBI signal is not guaranteed to do so. When encoded as MPEG-2, many satellite systems remove all but closed captioning.

There are very tight bandwidth restrictions on the CC data flow. The data rate of the CC data stream is two 7-bit bytes per video frame. In addition, in order to avoid collision with the control code, the data must start from 0x20, thus effectively limiting its transmission rate to about 6.5 bytes (for simplicity, the bytes are punctured to 6 bits). Furthermore, the bandwidth is equal to about 360 bits/second. If the channel is shared with the actual CC data, the transmission rate will be further reduced. Furthermore, additional control codes need to be transmitted to avoid the CC-capable television attempting to display the TiVo tag as CC text.

Basic label design

This paragraph will describe how the tags are arranged within the closed caption data stream. It is assumed here that there is a general understanding of the closed caption specification; however, this is not a determining factor.

Make the label invisible

The TiVo tag placed in the middle of the data stream should not affect the display on a closed captioning enabled television. This object is achieved by the following method: a "re-subtitle load" command is first sent (twice to avoid a miss) followed by a string of characters describing the tag, which are followed by an "erase non-display memory" command (twice to avoid a miss). The function of the above process is to load the text into the memory outside the picture and then to clear the memory. Typically, a television with closed captioning capability will not display this text (according to the EIA-701 standard).

The above method will necessarily work as long as the closed caption decoder is not in "scrolling" or "scrolling" mode. In this mode, a "re-subtitle load" instruction will cause the text to be deleted. To overcome this problem, the TiVo tag is received and identified even if it is transmitted to the second closed caption channel. In the above manner, even though closed caption channel 1 is set to scrolling text, the tag can still be transmitted through closed caption channel 2.

Encoding of labels

The text transmitted with the TiVo tag contains the letter "Tt," followed by a single character representing the length of the tag, followed by the tag contents, and then the Cyclic Redundancy Check (CRC) of the tag contents. The "Tt" letter is sufficiently unique that it is unlikely that the same letter will be encountered in general CC data. In addition, the general CC data usually starts with a position control code to indicate where the text should be displayed on the screen. Such position data is not needed since an on-screen display is not used. Thus, the very low probability of encountering the "Tt" letter immediately after the "subtitle load" control code is such that the combination is almost completely guaranteed to be a TiVo tag (although this implementation does not rely on the above).

The single character representing the length of the label is calculated by adding 0x20 to the length of the label. For example, if the length is 3 characters, the length character used is 0x23 ('#'). Thus, the implementation can be extended to a length of 95 (since there are only 96 characters in the word set), and the maximum length is defined as 63. If a longer tag is desired, an interpretation of the other 32 possible values may be added for the length character.

Possible values for the tag itself are the tag type sections defined below.

The CRC is a 16-bit CRC-CCITT standard code (i.e., polynomial ^ x ^16+ x ^12+ x ^5+ 1). The CRC code is placed in the data stream in three separate characters. The first word is calculated by adding 0x20 to the six most significant bits of the CRC. The next word is calculated by adding 0x20 to the second most significant bit of the CRC. The last word is calculated by adding 0x20 to the last four bits of the CRC.

Type of label

This paragraph will detail an example of a TiVo tag. It should be noted that each tag sequence starts with at least one byte, and the byte represents the type of tag.

iPreview label

Please refer to fig. 17. An iPreview label contains four items of information. The first item of information is a 32-bit program identification code of the trailer program. The second item of information contains the duration of the trailer. The third item of data is the position on the screen 1701 at which the iPreview hint 1702 is positioned, and the last item of information is the size used by the iPreview hint.

The screen position of iPreview hint is the ratio of the width and high resolution of the screen. The X coordinate uses 9 bits to divide the width, so the resulting coordinate is as follows: x ═ X _ resolution/511 × val. If xval is 10, then on a screen of 720X486 (using CCIR656 resolution), the X coordinate is 14. The Y coordinate uses 8 bits to distinguish the height, so the resulting coordinate is as follows: y ═ Y _ resolution/255 × yval. The X, Y coordinate indicates the position of the upper left corner of the adjustment effort.

If X, Y is set to the maximum possible value (i.e., x 511 and y 255), it indicates that the manufacturer has left the job of determining its position to the system for completion. The system will place the debug image in a predetermined location. The reason for using the maximum value to represent the preset position is: the "actual" position is never set at these values because this would cause the entire modulation pattern to go outside the picture.

The size bar is a four digit number that represents the size of all the hint images. The 16 possible values in this column correspond to the preset picture size, which the set-top box (video adapter) should provide in advance.

The timeout is limited to a 10-digit number that represents the number of frames left in the trailer. This would limit the lifetime of the tag to 34 seconds. If the duration of a particular forthcoming programme is long, the tag must be repeated. It should be noted that the timeout limit is "artificially limited" to 10 bits to avoid errors. This prevents a producer from affecting subsequent commercials when the producer uses a timeout constraint for misformatting in the label.

The version is a number used to program the version to identify the trailer itself. Instead of encapsulating the number in bits (thus limiting it to 6 bits), a complete closed caption character set is used, which results in 96 possible values instead of 64(2^6) possible values. Thus, the version number should be in the range of 0-95. Reserved characters are not used at present. Such a character must be present so that the control code can terminate correctly and align to the 2-byte boundary.

The first character of the Ipreview label is always "i".

All data fields are packed on the same bit boundary and then divided into six bit values, which are converted to words (by incrementing by 0x20) and transmitted. The order of the above columns is as follows:

32 bits: program identification code (ID)

9-bit: x coordinate

8-bit: y coordinate

4-position: image size

10-bit: time out limit

1 character: version(s)

1 character: retention fence

The data field has a total of 66 bits, which requires 11 characters to transmit, plus 1 character for the version field and 1 character for the reserved field. The exact contents of each character are as follows:

1)0x20+ identification code (ID) [31:26]

2)0x20+ID[25:20]

3)0x20+ID[19:14]

4)0x20+ID[13:8]

5)0x20+ID[7:2]

6)0x20+ID[1:0]X[8:5]

7)0x20+X[4:0]Y[7]

8)0x20+Y[6:1]

9) Size (size) 0x20+ Y0 (3: 0)

10)0x20+ Y0 size 3:0 timeout 9

11)0x20+timeout[8:3]

12)0x20+timeout[2:0]

13)0x20+ version (version)

14) Retention

The iPreview label is 14 characters +3 CRC characters in length, including the first character "i". A tag header (3 characters) is added, which makes the total length 20 characters and can be transmitted over 10 frames. Plus 4 additional frames for transferring two "re-subtitling loads" and "erasing undisplayed memory", this means that the iPreview tag will take 14 frames (0.47 seconds) when broadcast.

A full iPreview label contains:

"subtitle reload", Tt1(0x20+17 0x 31-0110001-1) i <13 character iPreview tag >, 3 character CRC, erasing undisplayed memory

Parity debug character

Currently, parity bits are used as parity. However, since the CRC is already included therein, the parity bits need not be provided with the function of error checking. Further development of this design allows the use of parity bits in a more subtle way. Since the closed caption receiver tends to ignore any characters with erroneous parity bits, a preferred way of utilizing a bandwidth limited CC channel is by deliberately using the parity of the errors. This eliminates the need to reload and delete the undisplayed memory with subtitles while making it easier to "intersperse" the TiVo tags in the existing CC data.

iPreview audience interaction

Please refer to fig. 17, fig. 20, fig. 21 and fig. 22. The iPreview tag causes the tag interpreter 2005 to display an iPreview hint 1702 on the screen 1701. The iPreview hint 1702 informs the viewer that the running program preview can be used, and the viewer can inform the TiVo system to record the program to be broadcast in the future. The viewer responds to the iPreview prompt 1702 by pressing the selection button 2204 on the remote control 2201.

The tag interpreter 2005 waits for user input. Pressing the selection button 2204 causes the label interpreter 2005 to automatically schedule the recording of the program according to the preferences preset by the viewer, thereby becoming a one-touch recording function, i.e., presenting the viewer with the recording options screen 2101. At this point, the viewer may highlight option 2102 on the recording menu and press select key 2204 to schedule the program for recording.

The tag itself has been interpreted by the tag interpreter 2005. The tag interpreter 2005 waits for any input from the viewer via the remote 2201. Upon the viewer pressing select key 2204, the tag interpreter 2005 informs the TiVo system of the schedule for scheduling the recording of the 32-bit program identifier described in the iPreyiew tag.

Please refer to fig. 20, 22 and 23. iPreview also has other uses. Each application is governed by the program content and the display screen image. Obviously, the system cannot interpret the program content, but the combination of the image and the program identification code can tell the tag interpreter 2005 what action should be taken. The following are two examples of the creation of rentals and sales.

The process of creating a rental may occur, for example, in an automobile advertisement being shown. When the iPreview image appears on the screen, the viewer knows to enter the interactive menu by pressing the selection key 2204.

Menu screens 2302 are displayed by tab interpreter 2005, which provide the viewer with the option to obtain more information 2303 or view a car movie 2304. The viewer can press the live tv button 2202 to leave at any time. If the viewer selects to get more information 2303 through the up/down direction keys 2203 and the selection keys 2204, the information desired by the viewer is transmitted by the tag interpreter 2005 to the manufacturer 2305, and thereby a product introduction is generated. The viewer may return to the program by pressing select key 2204.

Sales may occur when advertising a product, such as a music album. An iPreview image 2301 appears on the screen. The viewer can press the select key 2204 to cause the tag interpreter 2005 to display the menu screen 2307.

Menu screen 2307 provides the viewer with the option to purchase product 2308 or leave 2309. If the viewer selects "yes" 2308 to purchase the product, tag interpreter 2005 transmits the purchase order to the manufacturer along with the viewer's purchase information 2310. If the advertisement is a music album advertisement, the viewer can also choose to view the music movie produced by the music producer.

Whenever the system returns the viewer to the program, the viewer will return to the original departure. This gives the viewer the feeling of continuous play.

The redirect concept can be easily extended to the internet. The iPreview icon will appear as described above. When the viewer presses the selection key 2204 on the remote control 2201, the web page is presented to the viewer accordingly. The viewer may then interact with the web page and, when this interaction is complete, the system returns the viewer to the place where he or she previously left the program being viewed.

With the preference engine described above, information presented to an audience in forming a product introduction or sale can be readily tailored to the particular audience. The viewing habits, program preferences and personal information of the viewer are used to select menus, options and on-screen displays to be provided to the viewer. Each menu, option and on-screen display has associated program objects that are compared to the viewer's preference.

For example, if a viewer is male and the advertisement is provided by Chevrolet, a photo of a van will appear when the viewer presses the select key. If the audience is female, a picture of a convertible car will appear.

It should be noted that: the tag state machine 2006 described below is fully capable of performing the same steps as the tag interpreter 2005 in the example described above.

TiVo label state machine

Please refer to fig. 20 again. The preferred embodiment of the present invention provides a Tag State Machine (TSM) 2006. Such a tag state machine is a mechanism for handling abstract tags that produce actions that are viewable by viewers with a TiVo receiver.

Creating an active program preview is one of the simple examples. As described above, an active program announcement is an announcement that a program is about to be shown, and the viewer has the immediate option of having the TiVo system record when the program is actually shown.

There are several potential complexities in the simple example described above: an indicator must be generated to alert the viewer of the opportunity; the indicator must be accurately presented and disappeared; the correct identification code for the program discussed herein must be provided; programs that appear to have active announcements may be viewed at a time that is different from the time they were originally aired.

Challenges also exist in the creation and management of TiVo tags. It is important to minimize changes to existing broadcast means and techniques. This representation mechanism should be as simple as possible, making it easy to integrate into the broadcast data stream, and obtaining robust and reliable operating characteristics.

Operating principle of label

As mentioned above, the available bandwidth for transmitting tags is considered to be limited. For example, VBIs have limited available space for competing uses. Even in digital television signals, the amount of out-of-band data that is transmitted is not significant, since most users of the signal are primarily concerned with the choice of television programming.

Thus, tags become simple objects of only a few bytes in size. More complex actions are created by transferring multiple tags in succession.

The nature of broadcast transmissions implies that tags will be lost due to signal problems, sun blackouts, etc. The TSM utilizes some mechanism to handle the missing tag and ensures that no unexpected action is taken by the missing tag.

Generally, the tag actions viewable by a viewer are only related to the channel they are watching; it is assumed here that: the tag state is discarded after the channel change.

The physical tags are translated into abstract tags by the source object 1901 that receives the physical tags. Since the tag does not have executable code attached to it, the tag is not an "active agent"; the TSM is operated to obtain modifications and changes that are visible to the viewer, but the basic operation of the TiVo reception system is not affected by the tag sequence. If the tag can contain executable code, such as a Java byte stream for ATVEF, the quality of the TiVo viewing experience can be adversely affected by poorly written or malicious software.

All tag actions are managed by a matching policy object that matches the current channel. Any action is activated or deactivated by this object; if the policy object does not exist, all tag actions are blocked.

Basic abstract label

All abstract tags have a common infrastructure. The following components will appear in all abstract labels:

type of label (1 byte)

Type 0 is not allowed. Type 255 indicates an "extended" tag in case more than 254 tag values may be needed in the future.

-tag sequence (1 byte)

Such unsigned columns increment with each tag in the sequence. For tags that do not belong to a sequence, this column must be set to 0. 1 represents the start of a new sequence; conceptually, a sequence can be any length, but it must consist of ordered fragments of less than 255 tags.

Each tag type has an embedded sequence length (which may be 0); sequence numbers are introduced to handle tag misses or other forms of tag loss in the data stream. Generally, each time a sequence is erroneous, the entire tag sequence is discarded and the state machine is reset.

The tag should be checked for sums within the physical range. If the checksum does not match, the source object discards the tag. This causes sequence errors and resets the state machine.

Label time stamp (8 bytes)

This is the synchronization time within the television broadcast data stream at which the tag is identified. This time will be synchronized with all other display content generated by the TiVo receiver. This part will not be transmitted but will be generated by the receiver itself.

Label data length (2 bytes)

This is the length of any data in parallel with the tag. This data is interpreted based on the tag type. The entity scope interpreter should perform at least some degree of error checking on the data.

Label state machine (TSM)

The TSM is part of a label rendering mechanism that is in-line with video playback.

Conceptually, a TSM is an abstract stack of managed integers, where the accuracy of the integers is at least 32 bits, or the size is sufficient to hold an object ID. This object identification code is an abstraction that may or may not indicate the real object on the TiVo receiver-otherwise the object may have to be mapped to the correct object. The size of the stack is limited to 255 data items to prevent the destruction of denial of service.

The TSM also manages a set of variables. The variable is named a 2-byte integer. The variable name 0 is reserved. The tag sequence may be manipulated for a "user" variable; such variables are between 1 and 2^ 15-1. "system" variables are processed by the TSM and include values for the current TiVo receiver, such as: object identification code, TSM revision, and other useful information for the current program. The names of these variables are between 2^5 and 2^ 16-1. The number of user variables is limited within the TSM; and the TSM variable indicates the boundary.

The tag data is a sequence of TSM instructions. When the tag is identified and allowed, execution of the instructions begins. TSM instructions are byte-oriented, and certain instructions may include additional bytes to support their functionality.

The available TSM instructions can be distinguished into several types:

data movement instructions

push _ byte-following the instruction pushes the byte onto the stack.

push _ short — push short onto the stack following the instruction.

push _ word-follows the instruction to push the word onto the stack.

Variable access instruction

push _ var-pushes down the variable named 16 bits following the instruction.

pop _ var-pops a variable named 16 bits following the instruction.

copy _ var-copies from the stack to a variable named in 16 bits following the instruction.

Stack operation instruction

swap-interchange the stack values of the top two layers.

pop-stack value of top-up layer.

Arithmetic instructions

add byte-add number byte is added to the top level of the stack following the instruction.

add _ short-following the instruction, the numbered short is added to the top level of the stack.

add word-following the instruction the numbered word is added to the top level of the stack.

and-add the data items of the top and next layer stacks together, pop up the stack and push down the new value.

or-OR's the data items of the top and lower-level stacks, pop the stack up and push down the new value.

Conditional instruction

(for comparison of unsigned variables only)

brif zero if the top of the stack is zero, branch to signed 16-bit offset following the instruction.

brif _ nz — if the stack top level is not zero, then branch to the signed 16-bit offset following the instruction.

brif _ gt — if the stack top level is larger than the next stack data item, branch to signed 16-bit offset following the instruction.

brif ge — if the stack top level is greater than or equal to the next stack data item, branch to 16-bit numbered offset following the instruction.

brif _ le-if the stack top level is less than or equal to the next stack data item, then branch to the 16-bit signed offset following the instruction.

brif _ lt-if the stack top level is less than the next stack data item, then branch to the signed 16-bit offset following the instruction.

brif _ set — if there is a bit set when the top of the stack AND the next stack data item are concatenated together with the AND operator, then branch to the signed 16-bit offset following the instruction.

Motion command

exec-execution of tag actions against object identifiers named at the top of the stack

fin-terminating the tag without taking any action

System variable

32768(TAG) -value of the current TAG

.

.

.

Greenwich standard time:

32769(YEAR) -current YEAR (starting with 0).

32770(MONTH) -the current MONTH (1-12).

32771(DAY) -the date of the month (1-31).

32772(WDAY) -current week date (1-7, starting from sunday).

32773(HOUR) -the HOUR of day (0-23).

32774(MIN) -minutes in hours (0-59).

32775(SEC) -seconds in minutes (0-59).

State of TiVo receiver:

32800(SWREL) -software version (denoted x.x.x.x in bytes)

32801(NTWRK) -object identification code of currently tuned network

32802(PRGRM) -object ID of currently tuned program

32803(PSTATE) -the current state of the output channel:

0-Normal Play

1-pause

2-slow motion

10-inversion speed 1

11-reverse speed 2

...

20-fast forward speed 1

21-fast forward speed 2

The tag execution state:

32900(IND) -the indicator symbol code displayed or eliminated.

32901(PDURING) -the state of the channel in the tag execution process.

32902(ALTP) -advances the playing of another program object identifier of the stack.

32903(SELOBJ) -record program object identifier when the indicator is selected.

33000(MENU1) -string object number of MENU item 1.

33001(MENU2) -string object number of MENU item 2.

...

33009(MENU10) -string object number of MENU item 10.

33100(PICT1) -image object number of menu item 1.

33101(PICT2) -image object number of menu item 2.

...

3310g (PICT10) -image object number of menu item 10.

33200(MSELOBJ1) -record object identification code when menu item is selected.

33201(MSELOBJ2) -record object identification code when menu item is selected.

...

33209(MSELOBJ10) -record object identifier when menu item is selected.

Label (R)

-push down another program

-pop up another program (automatic pop up after program end)

-a rising indicator

-lowering the indicator

-menu

Tag enforcement policies

The execution policy is determined by the TSM. Several proposals are as follows:

-menu

The menu is arranged according to a standardized TiVo menu reference. Generally, the menu will appear on live video. Selecting an option typically leads to a recording session. Preferably, the data channel can be paused while the menu is being operated.

-an indicator

Please refer to fig. 17 and 22. The indicators are arranged and displayed on the bottom in the form of small images. In a normal viewing state, the up-down direction key 2203 on the remote controller 2201 is not operated. For the indicator, the up arrow key 2203 turns the indicator to the left, while the down arrow key turns it to the right. A small box may appear around the selected indicator. Pressing select key 2204 initiates the action. According to the preset condition, a new indicator is selected; when an indicator is deleted, it will be highlighted if there is a previously selected indicator.

-alternative programmes

The alternative programs should appear as part of the video data stream and have full fast forward/rewind control. Jump to live program button 2202 first pop the stack of programs available for selection as an empty stack.

Those skilled in the art will readily appreciate that: although the closed caption data stream is specifically illustrated here, other transmission methods may be used, such as: EDS fields, VBI, MPEG2 private data channels, and the like.

Although the present invention has been described in terms of preferred embodiments, those skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. Accordingly, the invention is limited only by the claims.

Claims (8)

1. A method for scheduling a recording of a television program via an advertisement in a television broadcast data stream, comprising:

receiving the television broadcast data stream;

broadcasting the promotion advertisement in the television broadcast data stream for the program broadcast in the future;

displaying an icon to inform a viewer that the program is available for recording, the icon being displayed in accordance with a tag inserted into the television broadcast data stream;

receiving a single key press of the audience on a remote control input device;

scheduling the recording of the program.

2. The method of claim 1 wherein the program is stored on a storage device when the scheduled time arrives.

3. The method of claim 2, wherein the storage device is a hard disk.

4. The method of claim 2, further comprising:

detecting whether the program ends prematurely during storage of the program; and

if the program ends prematurely, the storage of the program is automatically terminated.

5. An apparatus for scheduling a recording of a television program via an advertisement of a television broadcast data stream, the apparatus comprising:

a module for receiving the television broadcast data stream;

a module for broadcasting the promotion advertisement in the television broadcast data stream for the program to be broadcast in the future;

a module for displaying icons to inform viewers that the programs are available for recording, the icons being displayed according to tags inserted into the television broadcast data stream;

the module is used for receiving the single key pressing of the audience on a remote control input device;

and the module is used for scheduling the recording time of the program.

6. The apparatus of claim 5, further comprising:

a storage device; and

wherein the program is stored on the storage device when a scheduled time is reached.

7. The apparatus of claim 6, wherein the storage device is a hard disk.

8. The apparatus of claim 6, further comprising:

a detection module, wherein the detection module detects whether the program ends prematurely during storage of the program; and

a termination module, wherein the termination module automatically terminates storing the program when the program ends prematurely.