HK1071977B - Multimedia time warping system - Google Patents

Description

Multimedia time migration system

The present application is a divisional application of an invention patent application having an application number of 998090202.9 and entitled multimedia time migration system, filed on 4.3.1999.

Technical Field

The present invention relates to time shifting of televised signals. More particularly, the present invention relates to the instant recording, storage and playback of televised signals.

Background

Video Cassette Recorders (VCRs) have changed the lifestyle of television viewers around the world. The cassette recorder provides the viewer with time-shift flexibility of the television program to conform to the viewer's lifestyle.

The viewer uses a cassette recorder to store the television program on the tape. The VCR enables a viewer to play, reverse, fast forward, and pause recorded program content. These functions allow viewers to pause the playing of programs or fast forward through unwanted program content or advertisements as desired, and to replay favorite program content. However, a cassette recorder cannot record and play information at the same time.

One way to solve this problem is to use several cassette recorders. For example, if there are two recorders, the operation can be alternated between the two recorders (Ping-Pong). At this point, the first recorder/player starts at the beginning of the program of interest to the viewer. If the viewer wants to rewind the played content, the second recorder will start recording, and the first recorder will stop recording and rewind to the proper position and start playing. However, if the viewer wants to fast forward to a certain point in time after rewinding, at least a third recorder/player is needed. At this point, when the second recorder/player stops and falls back to the proper position, the third recorder/player can start recording the broadcasted data stream. If the above operations are continued, it is observed that the device becomes unwieldy, unreliable, expensive and difficult to operate, and cannot support all functional needs. Furthermore, the value of the above solution is greatly reduced, since the tape has a fixed length and may be interrupted at an inappropriate moment.

Digital computer systems have been proposed to address this problem. United states patent 5,371,551 to Logan et al, 12/6/1994, provides a method for simultaneous recording and playback of images. The above patent proposes a broadcasting and playing device controlled by a microprocessor. The apparatus compresses and stores video data on a hard disk. However, this approach is difficult to implement because the processor must keep up with the high speed video rate, making the device expensive and problematic. The processing rate of the microprocessor must be very fast in order to keep up with the incoming and outgoing video data.

If a multimedia time-shift system could be provided, it would be advantageous for the user to be able to record and play back the programs broadcast by the tv at the same time. It would also be advantageous to reduce the microprocessor and system requirements if a multimedia time-warping system could be provided that would eliminate the need for the microprocessor to handle high speed video data rates, which is desirable

Disclosure of Invention

The invention provides a multimedia time migration system. The invention uses a multimedia storage device and a display system which are easy to operate and low in cost, so that a user can watch television programs and simultaneously select and review previous contents in the programs in real time. In addition, the present invention enables a user to store a selected television program and the user to simultaneously watch or review another program.

One aspect of the present invention provides a method for simultaneously storing and playing back multimedia data in a computer environment, comprising the steps of: providing a plurality of input signal tuners; wherein the tuner receives analog and/or digital televised signals; wherein each of said tuners is individually tuned to a particular broadcast signal; converting the analog televised signal to a signal in a digital format; storing the digital formatted signal and/or the digital televised signal in a storage device as a digital signal; providing a plurality of output devices; wherein each of said output devices extracts a particular digital signal from said storage device; decoding a particular digital signal into a television output signal; transmitting the television output signal to a television monitor; wherein the plurality of output devices allow an image to be displayed in an image display on the television monitor; and allowing the user to view the specific digital signal with at least one of the following functions: rewind, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

Another aspect of the present invention provides an apparatus for simultaneously storing and playing back multimedia data in a computing environment, comprising: a plurality of input signal tuners; wherein the tuner receives analog and/or digital televised signals; wherein each of said tuners is individually tuned to a particular broadcast signal; means for converting the analog televised signal to a signal in a digital format; means for storing said digitally formatted signals and/or digital televised signals in a storage device as digital signals; a plurality of output devices; wherein each of said output devices extracts a particular digital signal from said storage device; means for decoding a particular digital signal into a television output signal; a module for transmitting said television output signal to a television monitor; wherein the plurality of output devices allow an image to be displayed in an image display on the television monitor; and a module for allowing a user to view a specific digital signal with at least one of the following functions: rewind, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

Yet another aspect of the present invention provides a method for simultaneously storing and playing back multimedia data in a computer environment, comprising the steps of: providing a plurality of input signal tuners; wherein the tuner receives analog and/or digital televised signals; wherein each of said tuners is individually tuned to a particular broadcast signal; converting the analog televised signal to a digital signal; separating the digital signal or digital televised signal into its video and audio components; storing the video and audio components in a storage device; providing a plurality of output devices; wherein each of said output devices extracts specific video and audio components from said storage device; decoding the particular video and audio components into a television output signal; transmitting the television output signal to a television monitor; wherein the plurality of output devices allow an image to be displayed in an image display on the television monitor; and allowing the user to view specific video and audio components with at least one of the following: rewind, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

Yet another aspect of the present invention provides an apparatus for simultaneously storing and playing back multimedia data in a computer environment, comprising: a plurality of input signal tuners; wherein the tuner receives analog and/or digital televised signals; wherein each of said tuners is individually tuned to a particular broadcast signal; a module for converting the analog televised signal to a digital signal; means for separating the digital signal or digital televised signal into its video and audio components; means for storing the video and audio components in a storage device; a plurality of output devices; wherein each of said output devices extracts specific video and audio components from said storage device; means for decoding the particular video and audio components into a television output signal; means for transmitting the television output signal to a television monitor; wherein the plurality of output devices allow an image to be displayed in an image display on the television monitor; and a module for allowing a user to view specific video and audio components in at least one of the following functions: rewind, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

A preferred embodiment of the present invention may receive multiple types of Television input data streams, such as analog NTSC (National Television Standards Committee) or Phase Alternating Line (PAL) Broadcast types, and Digital Satellite System (DSS), Digital Broadcast Services (DBS) or ATSC (advanced Television Standards Committee) types. The analog television data stream is converted into a (Moving Picture Experts Group, MPEG) formatted data stream for internal transmission and manipulation, and the pre-formatted MPEG data stream is obtained from the digital television signal and presented in a similar format to be encoded into the analog data stream.

The resulting MPEG data stream is parsed and split into video and audio components. These components are then stored in a buffer. The recorded events indicate the type of component found, its location, and when it occurred. Program logic is notified of the event that has occurred and retrieves data from the buffer.

The parser and event buffer may relieve a Central Processing Unit (CPU) from parsing the MPEG data stream and the real-time nature of the data stream. This CPU-less approach allows the use of lower speed CPUs and data buses, thereby reducing the cost of the system.

The video and audio components are stored in a storage device. When a program is requested, the video and audio components are retrieved from storage and reassembled into an MPEG data stream. The MPEG data stream is transmitted to a decoder. The decoder converts the MPEG data stream to a television output signal and transmits the television output signal to a television receiver.

The control command of the user is received and transmitted by the system. The instructions may affect the transport of the MPEG data stream and allow a user to view stored programs with at least one of the following functions: reverse play, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

Other technical features and advantages of the present invention will be readily apparent to the reader from the following detailed description, taken in conjunction with the accompanying drawings, wherein the principles of the invention are best understood by the reader.

Drawings

FIG. 1 is a schematic block diagram at a high level according to a preferred embodiment of the present invention;

FIG. 2 is a schematic block diagram of a preferred embodiment of the present invention in which the multiple input and output module of the present invention is used;

FIG. 3 is a schematic diagram of an MPEG data stream and its video and audio components according to the present invention;

FIG. 4 is a block diagram of a parser and four Direct Memory Access (DMA) input engines included in a media switch according to the present invention;

fig. 5 is a schematic diagram of a Packetized Elementary Stream (PES) buffer according to the present invention;

FIG. 6 is a schematic diagram of the structure of a packet elementary data stream buffer for parsed data components from the output ring buffer of the media switch;

FIG. 7 is a block diagram of a media switch and various elements for exchanging information with media in accordance with the present invention;

FIG. 8 is a high level schematic block diagram of program logic in accordance with the present invention;

FIG. 9 is a block diagram of a class hierarchy of program logic in accordance with the present invention;

FIG. 10 is a schematic block diagram of a preferred embodiment of a clip cache element according to the present invention;

FIG. 11 is a schematic block diagram of a preferred embodiment of a video mixer of the analog broadcast studio in accordance with the present invention;

FIG. 12 is a schematic block diagram of a closed caption parser in accordance with the present invention; and

fig. 13 is a schematic block diagram at a high level of a preferred embodiment of the present invention utilizing a cassette recorder as an integrated component.

Detailed Description

The invention is implemented in a multimedia time transformation system. A system according to the present invention provides a multimedia storage device and display system that allows a user to watch a television program while simultaneously selecting to review previous content in the program. The present invention also enables a user to store a selected television program and to simultaneously view or review another program, and to view the stored program by at least one of: rewind, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

Referring to fig. 1, the preferred embodiment of the present invention has an input area 101, a media switch 102 and an output area 103. The input area 101 obtains various types of Television input data streams, such as NTSC (national Television Standards Committee) or PAL (national Television Standards Committee) broadcasts, and Digital Satellite System (DSS) Digital Broadcasting Service (DBS) or ATSC (Advanced Television Standards Committee) broadcasts. DBS, DSS and ATSC are based on standards called Moving Picture Experts Group 2 (MPEG 2) and MPEG2 transport. MPEG2 transport is a standard for formatting a digital data stream from a television signal source transmitter so that a television receiver can parse an incoming data stream to find programs in a multiplexed signal. The input area 101 generates an MPEG data stream. The multiplexing of MPEG2 transmissions supports multiple programs on the same broadcast channel and has video and audio data feeds and private data. The input area 101 tunes the channel to a particular program, acquires a particular MPEG program therein, and provides it to other parts in the system. Analog television signals are encoded into a similar MPEG format using separate video and audio encoders so that the rest of the system is unaware of how the signal was obtained. Information may be modulated into the Vertical Blanking Interval (VBI) of an analog television signal in a number of standard ways; for example, the north american television Broadcast Standard (NABTS) may be used to modulate information into a 10-wire to 20-wire NTSC signal, while the Federal Communications Commission (FCC) manages 21-wire signals for Closed Caption (CC) and Extended Data Services (EDS). These signals are decoded with the input field and transmitted to other parts as in the private data channel via MPEG 2.

The media switch 102 coordinates operations between a Central Processing Unit (CPU)106, a hard disk or storage device 105, and a memory 104. The input data stream is converted to an MPEG data stream and passed to the media switch 102. The media switch 102 buffers the MPEG data stream in memory. Then, if the user watches the real-time tv program, the media switch 102 performs two operations: the data stream is transferred to the output area 103 and the data stream is simultaneously written to the hard disk or storage device 105.

Output section 103 takes as input the MPEG data stream and generates analog television signals in accordance with NTSC, PAL, or other desired television standards. The output section 103 includes an MPEG decoder, on-screen display generator, analog television encoder, and audio logic. The on-screen display generator enables the program logic to provide an image that will be overlaid by the resulting analog television signal. In addition, the output section may modulate information provided by the program logic onto the Vertical Blanking Interval (VBI) of the output signal using a variety of standard formats, including NABTS, Closed Caption (CC), and EDS.

Fig. 2, the present invention is easily scalable to multiple input zones (tuners) 201, 202, 203, and 204, each of which can be tuned to a different type of input. Multiple output modules (decoders) 206, 207, 208 and 209 may also be added simultaneously. Special effects such as picture-in-picture (pip-in-picture) may be implemented by multiple decoders. The media switch 205 simultaneously records another program while the user is watching the other program. This means that when one data stream is stored on disk, another data stream can be retrieved from disk.

Referring to fig. 3, an input MPEG data stream 301 has been inserted into segments 302, 305, and 306 of video data and segments 303, 304, and 307 of audio data. These data segments must be separated and recombined to create separate video 308 and audio 309 data streams or buffers. This is so because a separate decoder is used to convert the MPEG information back to the audio or video analog information component. This separate transmission requires the generation of time series information to enable the decoder to be properly synchronized for accurate signal playback.

The media switch enables the program logic to associate the appropriate time series information with each data segment, perhaps embedded directly in the data stream. The time series information of each data segment is called a time stamp. These time stamps are incremented in a unidirectional manner from zero each time the system is powered on. Thus, the invention can search a specific play point in a specific data segment. For example, if the system needs to read five seconds into an incoming continuous video data stream being processed by the cache, the system need only begin reading the stream forward and looking for the appropriate time stamp.

Binary search methods can search for stored files to point to a data stream. Since the time stamps are uniformly consistent, each data stream is stored as a sequence of fixed-size data segments in a manner that enables fast binary searching. If the user wants to start from the middle of the program, the system performs a binary search on the stored data segments until the appropriate point is found and the desired result is achieved with the least amount of information. If the signal is stored as an MPEG data stream, it may be necessary to parse the data stream from the beginning in a linear manner to find the desired position.

Referring to FIG. 4, the media switch includes four input Direct Memory Access (DMA) engines 402, 403, 404, and 405, each having an associated buffer 410, 411, 412, and 413. Conceptually, each DMA engine has a pointer 406, a limit 407 for the pointer, a next pointer 408, and a limit 409 for the next pointer. Each DMA engine is dedicated to a particular type of information; such as video 402, sound 403, and parsed events 405. Buffers 410, 411, 412, and 413 are ring-type and collect specific information. The DMA engine increments the pointer 406 to the associated buffer until it reaches the limit, and then loads the next pointer 408 and limit 409. The ring buffer is established by setting the pointer 406 and the next pointer 408 to the same value and their corresponding thresholds. The next pointer 408 may be set to a different address to provide vector DMA.

The input data stream passes through parser 401. Parser 401 parses the data stream for MPEG differentiated events and indicates the start of video, audio or private data segments. For example, when parser 401 finds a video event, it directs the data stream to video DMA engine 402. The parser fetches the information from the buffer via the video DMA engine 402 and accesses it directly into the video buffer 410. At the same time, the parser 401 directs an event to the event DMA engine 405, which generates an event to the event buffer 413. When the parser encounters an audio event, it directs the byte stream to the audio DMA engine 403 and generates an event to the event buffer 413. Similarly, when parser 401 encounters a private data event, it directs the byte data stream to private data DMA engine 404 and directs an event to event buffer 413. When an event is stored in the event buffer, the media switch notifies the program logic through an interrupt mechanism.

Referring to fig. 4 and 5, the event buffer 413 is filled with events by the parser 401. Each event 501 in the event buffer has an offset 502, an event type 503, and a timestamp field 504. When events are deposited into the buffer, the parser 401 may provide the type and offset for each event. For example, when a sound event occurs, the event type field is set to a sound event and the offset indicates the location in the audio buffer 411. Program logic may know the beginning of the audio buffer 411 and add an offset to find an event in the data stream. The address offset 502 informs the program logic of the occurrence of the next event, but not the end of the event. The previous event is cached to find the end point of the current event and the length of the data segment.

Referring to fig. 5 and 6, when the program logic is interrupted by the media switch 601, the program logic reads the events accumulated in the event buffer 602. From these events, program logic generates a series of logical data segments 603 corresponding to parsed MPEG data segments 615. The program logic converts the offset 502 to the real address 610 of each data segment and records the event length 609 with the last cached event. If the data stream results from the encoding of an analog signal, it does not contain the program time stamp value, which can be used by the decoder to properly render the final output result. Thus, the program logic uses the generated time stamp 504 to calculate a simulation program time stamp for each data segment and places the result in the logical data segment time stamp 607. For digital television data streams, the program time stamp values are already encoded in the data stream. The program logic takes this information and places it in the logical data segment time stamp 607.

Program logic continues to collect logical data segments 603 until it reaches a fixed size buffer amount. When this occurs, the program logic creates a new buffer called packet elementary stream 605 containing the sequential logical data segments 603 and accompanying control information. Each logical data segment 604 is directed to the ring buffer (e.g., video buffer 613) filled by the media switch 601. The new buffer is then transferred to other logic elements, and the logic elements may further process the data stream in the buffer in some manner, such as decoding or writing to a storage medium. Thus, MPEG data is not copied by the processor from one location in memory to another. This results in a cost effective design because less memory and processor bandwidth is required.

The unique property of the MPEG data stream conversion to packet elementary data stream (PES) buffer information is that the data associated with the logical data segment need not be present in the buffer data itself, i.e. as explained above. When the buffered data is written to the storage device, the logical data segments are written to the storage medium in the logical order in which they appear. This allows the data component of the data stream to be collected into a separate linear buffer of the data stream in the storage medium, regardless of whether the data component is located in a ring buffer of video, audio or private data. The information in the buffer is read back from the storage medium in a separate transmission, and the logical data segment information is updated to correspond to the actual location in the buffer 606. Higher level program logic is not aware of this transition process because it only processes logical data segments, so data stream data can be easily processed without the need for a CPU to copy information between Dynamic Random Access Memories (DRAMs).

One of the unique technical features of the media switch is its ability to process high speed data in an efficient and cost effective manner. The media switch may perform the following functions on a low cost platform: recording video and audio data, sending video and audio data to a disk, and retrieving video and audio data from the disk. Typically, media switches operate asynchronously and independently of the CPU of a microprocessor and use their DMA capabilities to transfer large amounts of information with minimal CPU involvement.

Referring to fig. 7, an input terminal of the media switch 701 is connected to the MPEG encoder 703. Also included are circuits 704 and 702, particularly for MPEG audio and VBI data provided to media switch 701. If digital television signals are processed, the MPEG encoder 703 is replaced by an MPEG2 transport demultiplexer, and the MPEG audio encoder 704 and VBI data decoder 702 are eliminated. The demultiplexer multiplexes the extracted audio, video, and data streams of the dedicated data channels through the video input media switch port.

The parser 705 is used to parse the incoming data stream from the MPEG encoder 703, the MPEG audio encoder 704 and the VBI data decoder 702, or from the transport demultiplexer (in the case of a digital television data stream). The parser 705 detects the start of all important events, all frames and sequence headers in the video or audio stream, all of which are needed by the program logic to correctly play the stream and perform special functions, such as: fast forward, rewind, play, pause, fast/slow play, index, and fast/slow reverse play.

When the parser 705 recognizes a video, audio data segment, or is given private data, a tag 707 is placed in the fifo buffer 706. When these tags are fetched, the DMA709 is controlled. Segment tag 707 and the DMA address are placed in event queue 708. Information about the frame type, i.e., video I-frame, video B-frame, video P-frame, video PES, audio PES, sequence header, normal data frame, or special data packet, is stored in the event queue 708 along with the offset stored in the associated ring buffer in which the information is stored. After program logic is transferred to DRAM714, program logic running in CPU713 checks for events in the ring buffer.

Media switch 701 has data bus 711 connected to CPU713 and DRAM 714. Address bus 712 is also shared among media switch 701, CPU713, and DRAM 714. The hard disk or storage device 710 is connected to one of the ports of the media switch 701. Media switch 701 outputs the data stream to MPEG video decoder 715 and separate audio decoder 717. The audio decoder 717 signal contains a sound indication signal. It is generated by the system in response to user instructions on a remote control or other internal event. The decoded sound output by the MPEG decoder is digitally mixed 718 with the separate sound signal. The resulting signal, which includes video, audio, and on-screen display, is transmitted to the television 716.

The media switch 701 takes the 8-bit data and sends it to the disk, while another data stream is taken from the disk and sent to the MPEG decoder 715. All of the above described DMA engines can operate at the same time. The media switch 701 may be implemented in hardware using a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), or discrete logic.

Program logic need only browse the ring event buffer in DRAM714 to identify the start location and type of each frame, rather than having to parse a large data stream to find the likely start location of each frame. This process saves a significant amount of CPU power and minimizes the real-time requirements of CPU 713. CPU713 need not have a very high rate at any one time. Media switch 701 provides the CPU713 with as much time as possible to complete the job. Parser 705 and event queue 708 cost-effectively frees CPU713 from analyzing the real-time nature of voice, video, and buffers, as well as the data stream. This approach also allows the use of bus structures in a CPU environment that uses very low clock rates and operates with less expensive memory than otherwise.

CPU713 has the function of queuing one DMA transfer and may establish the next DMA transfer when it is idle. This allows CPU713 a significant amount of time to servo DMA controller 709. CPU713 may respond to the DMA off in a larger timeframe because of the larger latency allowed. Whether an MPEG data stream obtained from an MPEG2 transmission or encoded from an analog television signal, is typically encoded using a technique known as Variable Bit Rate (VBR) encoding. This technique would change the amount of data needed to represent a series of images to the amount of movement between the images. This technique can significantly reduce the bandwidth of the signal, while fast moving strings (e.g., basketball) can be encoded with much greater bandwidth. For example, a Houss direct broadcast television satellite system encodes at a rate of 1 to 10Mb/s per second of the required bandwidth; and varies from frame to frame. Without this structure, it is difficult for any computer system to keep up with such fast changing data rates.

Referring to fig. 8, the program logic in the CPU has three conceptual elements: source region 801, conversion region 802, and sink 803. The source region 801 creates a data buffer. The conversion region 802 handles data buffers, while the sink 803 consumes data buffers. The translation region is responsible for allocating and arranging the data buffers on which to operate. The buffer is allocated to the data source as "empty" and fed back as "full". The buffer then goes into line and is provided to the sink in a "full" state, and the sink will return the buffer to a blank state.

Source region 801 receives data from an encoder, such as a digital satellite receiver. The source switches from downstream to get the buffer of the data and seals the data into the buffer, and then pushes the buffer into the transmission channel in the manner described above. The source target 801 is unaware of the status of other parts of the system. The sink 803 consumes the buffer and takes the buffer from upstream conversion and passes the data to the decoder, which then releases the buffer for reuse.

The conversion 802 used here is of two types: spatial conversion and temporal conversion. Examples of spatial transformations are image convolution or compression/decompression as the buffered data passes through. The time transition is used when there is no representable time relationship between the buffers entering the system and the buffers leaving the system. This translation writes the buffered data to a file 804 located on the storage medium. The buffered data is later retrieved and transmitted along the transmission channel and properly sequenced in the data stream.

Referring to FIG. 9, a hierarchy of C + + categories derived from program logic is shown. The TiVo (a personalized tv service) media cores (TmkCore)904, 908 and 913 operate within the operating system core. This core provides operational functions such as memory allocation, synchronization, and threading. Tmkcre 904, 908 and 913 targets memory taken from the media core, which may be provided to the operator for either constructing new targets or deleting targets. Each destination (source 901, conversion 902, and sink 903) is defined as a multiplexed thread and can operate in parallel processing.

TmkPipeline (Tmb channel) categories 905, 909, and 914 are responsible for controlling traffic through the system. The transmission channel is directed in the traffic from the source region 901 to the sink 903 to the next transmission channel. For example, if the transmission channel is to be suspended, an event called "suspend" is sent to the first destination in the transmission channel. The event is forwarded all the way along the transmission path to the next destination. This process is performed asynchronously with respect to the data passing through the transmission channel. Thus, in applications such as telephony, the flow control of an MPEG data stream is asynchronous and separate from the data stream itself. This allows for the use of simple logic designs while having sufficient capabilities to support the features previously described, including pause, rewind, fast forward, etc. Furthermore, this structure allows for a fast and efficient switching between the sources of the data streams, since the buffered data can be discarded and the decoder reset with a single event, after which the data from the new data stream will pass through the transmission channel. This may be required, for example, when a channel switch is acquired by the input area, or when switching between a live signal from the input area and a stored data stream.

Source region target 901 is TmkSource (Tmk source region) 906 and transition target 902 is TmkXfrm 910. The above are intermediate level categories defined as standard behavior among the transmission channels. Conceptually, they exchange buffers along the transmission channel. The source object 901 fetches data (e.g., media switch) from the physical data source and stores it in a PES buffer. To obtain this buffer, the source target 901 requests a buffer (alloemptybuf) from the downstream stream target in the transmission channel. The source region object 901 will be closed until enough memory is available. This means that the transmission channel is self-adjusting; it has automatic flow control. When the source target 901 fills the buffer, it will hand the buffer back to the conversion area 902 through the pushFullBuf function.

The sink 903 is also flow controlled. The sink calls nextFullBuf to tell the conversion area 902 that it is ready for the next full buffer. This operation closes the sink 903 until a buffer is ready. When the sink 903 has completed a buffer (i.e., it has consumed the data in the buffer), it calls releaseEmptyBuf. The ReleaseEmptyBuf hands the buffer back to the translation area 902. For example, the transition area 902 may then hand the buffer back to the source target 901 to refill. This approach, in addition to having the advantage of automatic flow control, is used to force a fixed configuration of the translation buffer to limit the amount of memory dedicated to the buffer. This is an important feature to achieve cost-effectiveness in a limited DRAM environment.

The MediaSwitch category 909 employs the allocastybuf method of the TmkClipCache 912 target and receives the PES buffer therefrom. It then leaves the ring buffer in the hardware of the media switch and generates a PES buffer. The MediaSwitch category 909 will fill the buffer and push it back to the TmkClipCache 912 target.

The TmkClipCache 912 will retain a cache file 918 in the storage medium. It also keeps two pointers in this cache memory: a push pointer 919 that can display where the next buffer from source 901 is inserted; and a current pointer 920 that points to the buffer currently in use.

The buffer to which the pointer is currently pointing is processed via the Vela decoder class 916. The Vela decoder class 916 interfaces with the decoder 921 in hardware. Decoder 921 produces a decoded television signal that is subsequently encoded into a television signal in an analog NTSC, PAL, or other analog format. After the Vela decoder class 916 completes the buffer, it will call releaseEmpthBuf.

The class structure can make the system easy to test and debug. Each level can be tested individually to ensure that it performs in an appropriate manner, and classes can be progressively combined to achieve the desired functionality while maintaining the ability to effectively test each target.

The controller 917 is used for receiving instructions from a user and sending events to the transmission channel to control the execution status of the transmission channel. For example, if the user has a remote controller and is watching TV, the user presses the pause key, and the controller 917 sends an event to the sink 903 to notify the user of the pause. The sink 903 will stop requiring a new buffer. The pointer 920 now stays at its location. When the sink 903 receives another event to notify it to play, the sink 903 will start to acquire the buffer again. The system has perfect synchronization function; it starts where the frame stops.

The remote control also includes a fast forward button. When the fast forward button is pressed, the control object 917 sends an event to the transition area 902 to tell it to move forward for two seconds. The transition region 902 finds that a two second time span requires three buffers to be moved forward. It then issues a reset event to the downstream transmission channel so that any data in the queue or state that may be present in the hardware decoder is forced out. This is a critical step because the structure of the MPEG stream requires that the state of multiple data frames must be maintained, which will become invalid due to the reconfiguration of the pointers. Which then moves the current pointer 920 forward three buffers. The sink 903 will get a new current buffer the next time it calls nextFullBuf. The same method can be applied to the fast rewind function when the transition area 902 moves the current pointer 920 backward.

The system clock reference is located in the decoder. The system clock reference is accelerated when fast-play is dropped and decelerated when slow-play is dropped. The sink requires the entire buffer to speed up or slow down, depending purely on the clock rate.

Referring to FIG. 10, two other objects derived from the TmkXfrm class are stored in the transport channel to facilitate disk access. One of which is referred to as TmkClipReader1003 and the other as TmkClipWrite 1001. The buffer comes to TmkClipWrite 1001 and is pushed into a file located on storage medium 1004. The TmkClipReader1003 will require a buffer from the file in the storage medium 1005. The TmkClipReader1003 provides only allocEmptyBuf and pushFullBuf methods, while the TmkClipWrite 1001 provides only nextFullBuf and releaseEmptyBuf methods. Thus, the TmkClipReader1003 performs the same function as the "push" side of the input or TmkClipCache 1002, while the TmkClipWrite 1001 performs the same function as the "pull" side of the output or TmkClipCache 1002.

Referring to FIG. 11, a preferred embodiment for performing multiple functions is shown. The source region 1101 has a television signal input. The source region passes the data to PushSwitch1102, which is from the transition of TmkXfrm. PushSwitch1102 has multiple outputs that can be switched by a control target 1114. This means that a part of the transmission channels can be stopped and another transmission channel can be started as desired by the user. The user can switch to different storage devices. PushSwitch1102 may output to TmkClipWriter 1106 to storage 1107 or write cache switch 1103.

An important feature of the device is that the recording of the input signal can be easily selected under the control of the program logic. The TmkClipWriter 1106 can be switched to record a portion of the signal and shut down at a later time based on information such as the current time, a particular length of time, or via a viewer pressing a remote control. This switchover is typically caused by sending out a "switchover" event to the PushSwitch1102 target.

Another method for enabling selective recording is by modulating the information to the VBI, or depositing to an MPEG private data channel. The decoded data from the VBI or dedicated data channel is sent to program logic. Program logic examines the data to determine whether the data indicates that recording of the modulated television signal should begin. Likewise, this information may also indicate when the recording should terminate, or another data item may be tuned to a signal indicating when the recording should terminate. The start and stop indicators may be explicitly modulated into the signal or other information stored in the signal in a standard manner may be used to encode this information.

Referring to fig. 12, an example is shown of how program logic scans words contained in the closed caption line to initiate recording with a particular word or phrase to determine the time of initiation and termination. Including the data stream of the NTSC or PAL field 1201. The close caption tuples are obtained from each of the odd fields 1202 and added to the ring buffer 1203 for processing by the word parser 1204. The lexical analyzer 1204 collects characters until a word boundary is encountered, which is typically a space, period, or other descriptive character. Recall that the MPEG audio and video data segments are collected into a series of PES buffers of fixed size. A special data segment is added to each PES buffer to hold the word obtained from the closed caption field 1205. Therefore, the closed caption information is synchronized with the audio and video and can be presented to the viewer correctly during the playback of the data stream. This also allows the stored data stream to provide processing for closed caption information while program logic is idle, which distributes throughput, reduces cost, and increases efficiency. In this case, the words stored in the special data field may be sent directly to the state table logic 1206.

During recording of the data stream, each word is locked in a table 1206 that indicates the identification action to be performed for that word. This action may simply change the state of the recognizer state machine 1207, or may cause the state machine 1207 to issue an action request, such as "start recording," "stop recording," "words viewed," or other similar request. Indeed, the recognized words or phrases cause the transmission channel to be switched; for example, if an unwanted language is used in the program, a different sound track may be overlaid.

It should be noted that the parser state table 1206 and recognizer state machine 1207 may be modified or changed at any time. For example, different tables and state machines may be provided for each input channel. In another case, these elements may be switched depending on the time of day or other events.

Referring to fig. 11, PullSwitch1104 is added, which outputs to sink 1105. The sink 1105 calls nextFullBuf and releaseEmptyBuf to get or return the buffer from PullSwitch 1104. PullSwitch1104 may have any number of inputs, one of which may be ActionClip 1113. The remote control may switch between input sources. The control target 1114 communicates an event to the PullSwitch1104 to inform it to switch from the current input source to whichever input source the control target selected.

The ActionClip category provides for arranging a number of different stored signals in a predictable and controllable manner and may incorporate controls selected by the viewer via a remote control. It is therefore presented as a derivative of the TmkXfrm target and receives a "switch" event for switching to the next stored signal.

This allows program logic or the user to create custom video output sequences. Any number of segments of video data may be arranged and combined, as may program logic or user use of a broadcast studio video mixer. TmkClipReader 1108, 1109 and 1110 would be allocated and hung in PullSwitch 1104. PullSwitch1104 switches between TmkClipReader 1108, 1109 and 1110 to combine video and sound slices. The flow control is automated due to the construction of the transmission channel. Push and Pull switches are just like the video switch of a broadcast studio.

The derived categories and resulting targets described herein may be combined in any manner to create a number of different and useful configurations for storing, retrieving, switching and viewing television data streams. For example, if multiple output and input regions are available, one of the input regions may be viewed while the other input region is stored, while a picture-in-picture window may be created from the second output region to view a previously stored data stream. This configuration represents a unique and novel software conversion application that can be implemented using a single cost-effective device to achieve the results obtained with expensive and complex hardware solutions.

Referring to fig. 13, a system diagram of a high level hierarchy implemented with a jukebox backup is shown. The output module 1303 transmits the television signal to the cassette recorder 1307. This allows the user to record the television program directly on the video tape. The present invention allows the user to arrange the programs to be recorded on the disk and to sort the programs by time they are sent to the cassette recorder 1307. The title page (EPG data) may be fed to the VCR 1307 before a program is transmitted. Longer programs may be adjusted by speeding up the playback rate or omitting frames to fit smaller video tapes.

The output of the cassette recorder 1307 may also be directed back into the input module 1301. In this configuration, the VCR acts as a backup system for the media switch 1302. All excess stored or lower priority programming will be sent to the cassette recorder 1307 for later access.

The input module 1301 may decode and pass on to the rest of the system information encoded on the Vertical Blanking Interval (VBI). The output module 1303 can be encoded into the output VBI data provided by the rest of the system. The program logic may be arranged to encode different types of identification information into an output signal that is recorded on the film by the videocassette recorder 1307. Returning the tape to play allows the program logic to return to reading the identification information so that the television signal recorded on the tape can be properly processed. For example, a particular program may be recorded onto film along with information regarding when it was recorded and the source network. When the program is played to the input module, the information can be used to control the storage of signals and presentation to viewers, etc.

Those skilled in the art will readily appreciate that this mechanism may be applied to the introduction of various data items into the program logic, which are not considered television signals. For example, software updates or other data may be communicated to the present system. The program logic receiving this data from the television datastream may control the manner in which the data is processed, e.g., by requiring some sort of authentication procedure and/or decrypting the embedded information based on some sort of pre-fetched key. This method also works on normal broadcast signals and introduces an efficient way to provide non-television control information and data to the program logic.

Furthermore, one of ordinary skill in the art will readily recognize that, although the above description is directed specifically to a VCR, any multimedia recording device, such as a digital versatile disc-random access memory (DVD-RAM) recorder, may be readily substituted for a VCR.

Although the present invention has been described herein with reference to the preferred embodiments, those skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. For example, the present invention may be applied to detecting criminal activity at a gambling location. The input field of the present invention is connected to the photo monitoring system of the casino. The recorded images are cached and simultaneously output to an external VCR. The user can switch to either video input and simultaneously view (e.g., rewind, play, slow play, fast play, etc.) a particular data segment of the recorded video signal while the external cassette recorder loads the real-time input video signal. Accordingly, the invention is not to be restricted except in light of the attached claims.

Claims (42)

1. A method for simultaneously storing and playing back multimedia data in a computing environment, comprising the steps of:

providing a plurality of input signal tuners;

wherein the tuner receives analog and/or digital televised signals;

wherein each of said tuners is individually tuned to a particular broadcast signal;

converting the analog televised signal to a signal in a digital format;

storing the digital formatted signal and/or the digital televised signal in a storage device as a digital signal;

providing a plurality of output devices;

wherein each of said output devices extracts a particular digital signal from said storage device;

decoding a particular digital signal into a television output signal;

transmitting the television output signal to a television monitor;

wherein the plurality of output devices allow an image to be displayed in an image display on the television monitor; and

allowing a user to view a particular digital signal with at least one of the following functions: rewind, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

2. The method of claim 1, further comprising the steps of:

control commands from a user are received.

3. The method of claim 2, wherein a user selects an image in the image display displayed on the television monitor.

4. The method of claim 2, wherein a user selects which of the plurality of output devices to display the image in an image display.

5. The method of claim 2, wherein the user selects a display position of each image in the image display.

6. The method of claim 2, wherein a user selects an individual tuner and a particular broadcast signal for the individual tuner.

7. The method of claim 2, wherein a user selects a particular digital signal to be retrieved and decoded from the storage device.

8. The method of claim 1, further comprising the steps of:

inserting an on-screen display into the television output signal.

9. The method of claim 1, wherein the particular broadcast signal for each tuner is automatically selected based on time of day.

10. The method of claim 1, wherein the specific broadcast signal for each tuner is automatically selected based on a specific word or phrase in the broadcast signal.

11. An apparatus for simultaneously storing and playing back multimedia data in a computing environment, comprising:

a plurality of input signal tuners;

wherein the tuner receives analog and/or digital televised signals;

wherein each of said tuners is individually tuned to a particular broadcast signal;

means for converting the analog televised signal to a signal in a digital format;

means for storing said digitally formatted signals and/or digital televised signals in a storage device as digital signals;

a plurality of output devices;

wherein each of said output devices extracts a particular digital signal from said storage device;

means for decoding a particular digital signal into a television output signal;

a module for transmitting said television output signal to a television monitor;

wherein the plurality of output devices allow an image to be displayed in an image display on the television monitor; and

a module for allowing a user to view a specific digital signal with at least one of the following functions: rewind, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

12. The apparatus of claim 11, further comprising:

and the module is used for receiving a control command from a user.

13. The apparatus of claim 12, wherein a user selects an image in the image display displayed on the television monitor.

14. The apparatus of claim 12, wherein a user selects which of said plurality of output devices to display said image in an image display.

15. The apparatus of claim 12, wherein the user selects a display position of each image in the image display.

16. The apparatus of claim 12, wherein a user selects an individual tuner and a particular broadcast signal for the individual tuner.

17. The apparatus of claim 12, wherein a user selects a particular digital signal to be retrieved and decoded from the storage device.

18. The apparatus of claim 11, further comprising:

means for inserting an on-screen display into the television output signal.

19. The apparatus of claim 11, wherein the particular broadcast signal for each tuner is automatically selected based on the time of day.

20. The apparatus of claim 11, wherein the specific broadcast signal for each tuner is automatically selected based on a specific word or phrase in the broadcast signal.

21. A method for simultaneously storing and playing back multimedia data in a computing environment, comprising the steps of:

providing a plurality of input signal tuners;

wherein the tuner receives analog and/or digital televised signals;

wherein each of said tuners is individually tuned to a particular broadcast signal;

converting the analog televised signal to a digital signal;

separating the digital signal or digital televised signal into its video and audio components;

storing the video and audio components in a storage device;

providing a plurality of output devices;

wherein each of said output devices extracts specific video and audio components from said storage device;

decoding the particular video and audio components into a television output signal;

transmitting the television output signal to a television monitor;

wherein the plurality of output devices allow an image to be displayed in an image display on the television monitor; and

allowing a user to view specific video and audio components with at least one of the following functions: rewind, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

22. The method of claim 21, further comprising the steps of:

control commands from a user are received.

23. The method of claim 22 wherein a user selects an image in the image display displayed on the television monitor.

24. The method of claim 22, wherein a user selects which of the plurality of output devices to display the image in an image display.

25. The method of claim 22, wherein the user selects a display position of each image in the image display.

26. The method of claim 22, wherein a user selects an individual tuner and a particular broadcast signal for the individual tuner.

27. The method of claim 22, wherein a user selects specific video and audio components to be extracted and decoded from the storage device.

28. The method of claim 21, further comprising the steps of:

inserting an on-screen display into the television output signal.

29. The method of claim 21, wherein the particular broadcast signal for each tuner is automatically selected based on time of day.

30. The method of claim 21, wherein the specific broadcast signal for each tuner is automatically selected based on a specific word or phrase in the broadcast signal.

31. The method of claim 21, further comprising the steps of:

extracting other signal components from the digital signal or the digital televised signal;

wherein said storing step stores said other signal component in said storage device;

wherein each of said output devices extracts from said storage device the associated signal component of said particular video and audio component;

the correlated signal component is reproduced at an appropriate location in the television output signal.

32. An apparatus for simultaneously storing and playing back multimedia data in a computing environment, comprising:

a plurality of input signal tuners;

wherein the tuner receives analog and/or digital televised signals;

wherein each of said tuners is individually tuned to a particular broadcast signal;

a module for converting the analog televised signal to a digital signal;

means for separating the digital signal or digital televised signal into its video and audio components;

means for storing the video and audio components in a storage device;

a plurality of output devices;

wherein each of said output devices extracts specific video and audio components from said storage device;

means for decoding the particular video and audio components into a television output signal;

means for transmitting the television output signal to a television monitor;

wherein the plurality of output devices allow an image to be displayed in an image display on the television monitor; and

a module for allowing a user to view specific video and audio components in at least one of the following functions: rewind, fast forward, play, pause, index, fast/slow reverse play, and fast/slow play.

33. The apparatus of claim 32, further comprising:

and the module is used for receiving a control command from a user.

34. The apparatus of claim 33 wherein a user selects an image in the display of images displayed on the television monitor.

35. The apparatus of claim 33, wherein a user selects which of said plurality of output devices to display said image in an image display.

36. The apparatus of claim 33, wherein the user selects a display position of each image in the image display.

37. The apparatus of claim 33, wherein a user selects an individual tuner and a particular broadcast signal for the individual tuner.

38. The apparatus of claim 33 wherein a user selects particular video and audio components to be extracted and decoded from the storage device.

39. The apparatus of claim 32, further comprising:

means for inserting an on-screen display into the television output signal.

40. The apparatus of claim 32 wherein the particular broadcast signal for each tuner is automatically selected based on the time of day.

41. The apparatus of claim 32, wherein the specific broadcast signal for each tuner is automatically selected based on a specific word or phrase in the broadcast signal.

42. The apparatus as recited in claim 32, further comprising:

means for extracting other signal components from the digital signal or the digital televised signal;

wherein the means for storing stores the other signal components in the storage device;

wherein each of said output devices extracts from said storage device the associated signal component of said particular video and audio component;

means for reproducing said associated signal component at an appropriate location in said television output signal.