HK1140326B - Methods, apparatus, and systems for insertion of overlay content into a video signal with transrating capabilities - Google Patents

Abstract

Methods, apparatus, and systems for inserting overlay content into a video signal are provided. An overlay content signal containing overlay content is received. In addition, a compressed video signal is received from a video source and decoded. An insertion window portion of the decoded video signal is identified. The overlay content signal and the insertion window portion of the decoded video signal are processed to produce a modified insertion window portion of the video signal. The video signal and the modified insertion window portion are then processed to produce a modified video signal. The data rate of the video signal may be modified during the overlay content insertion process.

Description

Method, apparatus and system for inserting overlay content into a video signal having a slew rate capability

This application claims the benefit of U.S. provisional application No. 60/845,707, filed on 8, 2006, 9, 18, which is incorporated herein by reference and forms a part hereof.

Technical Field

The present invention relates to the field of video processing. More particularly, the present invention provides methods, apparatus and systems for inserting overlay content into a video signal. Additionally, certain example embodiments of this disclosure are directed to inserting overlay content into a video signal during video signal transrating.

Background

Today's television images are often overlaid with additional content (referred to herein as "overlay content"), such as text messages, logos, animations, or sub-windows representing full motion video. Such edited signals may be used to convey emergency information, targeted advertising, or to customize messages associated with full screen programs or advertisements. There may be more than one candidate overlay content available for insertion into a particular video stream at a particular time. While the apparatus for inserting or overlaying new content into full-screen video has been readily available, such apparatus is typically designed to operate with uncompressed video signals, and its use is only for those applications.

Encoded video signals based on compression standards such as MPEG present special challenges to the video editing process. A prior art system for inserting overlay content into a video signal is shown in fig. 1. In the prior art, insertion of overlay content (auxiliary video) at the video editor 12 fully decodes the video signal prior to the decoder 10. The modified video signal from video editor 12 is then re-encoded at encoder 14 before being forwarded to the final destination. Not only is this solution expensive, it can also degrade video quality. For example, it is not uncommon to select expensive high quality encoders for the original encoding process, and then process the signals in smaller cable headends or other distribution sites serving more limited viewers, as needed to reduce the cost of additional hardware.

Although it is possible to directly modify a video signal without using a separate decoder, editor, and encoder, there are a number of problems that need to be solved. For example, compression standards such as MPEG-2 and h.264 use motion compensation to predict the movement of features from one video frame to another. However, if one of these frames is changed by inserting or overlaying a different video image, the synchronization between the decoder and the original encoder is disrupted. Thus, if a region of an image is encoded using motion vectors that span the boundary between a modified region and an unmodified region of the image, the motion prediction process will fail.

It would be advantageous to provide a simplified way of managing the insertion of overlay content into a video signal. In particular, it would be advantageous to enable overlay content to be closely correlated with a video signal into which the overlay content is to be inserted and/or with video signals in a particular geographic region. It would also be advantageous to achieve this close correlation between the overlay content and the video signal using a tag having identifying information appended to the overlay content and/or the video signal. It would be further advantageous to be able to modify a compressed video signal with overlay content without the use of a decoder and encoder. This results in considerable cost savings, especially when large flows need to be processed. In addition, it would be advantageous to be able to modify the compression ratio of a video signal using the same insertion and overlay device.

The method, apparatus and system of the present invention provide the above and other advantages. In particular, the present invention is particularly advantageous when combined with advanced slew rate systems, such as statistical multiplexers used in cable and other video distribution centers.

Disclosure of Invention

The present invention provides a method, apparatus and system for inserting overlay content into a video signal. Additionally, certain example embodiments of this disclosure are directed to inserting overlay content into a video signal during a slew rate of the video signal.

In one example embodiment of the present invention, an overlay content signal containing overlay content is received. In addition, a compressed video signal is received from a video source and decoded. An insertion window portion of the decoded video signal is identified. Processing the overlay content signal and the insertion window portion of the decoded video signal to generate a modified insertion window portion of the video signal. The video signal and the modified insertion window portion are then processed to generate a modified video signal.

In another example embodiment of this disclosure, the data rate of the video signal may be modified during the overlay content insertion process. A closed loop transrating path may be provided for modifying the data rate of the video signal. In addition, an open loop transrating path may also be provided for modifying the data rate of the video signal. The closed loop transrating path may be selected for modifying the data rate of the inserted window portion of the video signal.

Pixel quantization error control may be provided in a closed loop slew rate path. Pixel quantization error control may include obtaining an error from quantization of transform coefficients of an incoming pixel of the video signal in the form of an error transform coefficient. An inverse transform process may be performed on the error transform coefficients to provide pixel errors in the pixel domain. These pixel errors may be stored and then retrieved during modification of the data rate of the associated pixel. Pixel errors may be added to incoming pixels obtained from an inverse transform process to obtain corrected pixels in order to prevent the accumulation of re-quantization artifacts during quantization of transform coefficients corresponding to the incoming pixels. A transform process may be performed on the corrected pixels to provide error-corrected transform coefficients for quantization.

The closed loop transrating path may be selected for modifying the data rate of I-frames and P-frames of the video signal. The open-loop transrating path may be selected for modifying the data rate of B-frames of the video signal. In general, it is advantageous to use a closed loop transrating path for modifying the data rate of the reference frame. There are pictures that can be used to predict other pictures. In the case of MPEG-1 and MPEG-2, the reference frames are I-frames and P-frames.

The closed-loop and open-loop slew-rate paths may operate simultaneously. For example, the same system may be used to process multiple video streams, and in this case it is possible to keep each of the open and closed loop processors busy by allocating frames from any of the video streams where the next frame is available. The closed-loop transrating path may be selected for modifying the data rates of I-frames and P-frames of various video signals, and the open-loop transrating path may be selected for modifying the data rates of B-frames of various video signals.

In some cases, both the open-loop and closed-loop transrating paths may be used to process I-frames and P-frames of a video signal. Similarly, both the open-loop and closed-loop transrating paths may be used to process B-frames of the video signal.

Additional open loop transrating paths may be provided for modifying the data rate of the video signal. Both open-loop and closed-loop transrating paths may be operated simultaneously. Those skilled in the art will appreciate that additional open or closed loop slew rate paths may be provided as needed in order to increase the processing speed of the system.

The open-loop transrating path may include: variable length decoding of a video signal to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients to scale the transform coefficients; performing a quantization operation on the scaled transform coefficients to re-scale the transform coefficients to achieve a desired data rate; and variable length coding of the re-scaled transform coefficients to provide a video signal at a desired data rate.

In another example embodiment of the present disclosure, at least one of an obfuscation processing mode and an insertion processing mode may be provided for carrying out a processing step. The blur processing mode is operable for processing a blur region of a video signal in which a motion vector of a current block of pixels of the video signal references a portion of an insertion window. The insertion processing mode is operable for processing an insertion window portion of the overlay content signal and the video signal to generate a modified insertion window portion.

Additionally, a normal processing mode may be provided for processing the video signal when the video signal does not contain an insertion window. The normal processing mode may include: variable length decoding of a video signal to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients to scale the transform coefficients; performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients into pixel values; summing the pixel value with the motion compensated pixel error to provide an error compensated pixel value; performing a transform process on the error-compensated pixel values to generate error-compensated transform coefficients; performing a quantization operation on the error compensated transform coefficients to rescale the transform coefficients to achieve a desired data rate; and variable length coding of the re-scaled transform coefficients to provide a video signal at a desired data rate.

Motion compensated pixel errors may be obtained by: an error is obtained from the quantization operation in the form of a pixel error and the pixel error is stored. The stored pixel errors may be retrieved when performing motion compensation operations on associated pixels. The pixel error and associated pixels may then be processed to provide a motion compensated pixel error. Summing the pixel values with the motion compensated pixel errors prior to the transform process prevents the accumulation of re-quantization artifacts during the quantization operation.

A restricted processing mode may be provided for carrying out at least one of the processing steps when the obfuscation processing mode, the insertion processing mode and/or the normal processing mode are not selected. The restricted processing mode may include: variable length decoding of a video signal to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients to scale the transform coefficients; performing a quantization operation on the scaled transform coefficients to re-scale the transform coefficients to achieve a desired data rate; variable length coding of the re-scaled transform coefficients to provide a video signal at a desired data rate; performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients into pixel values; adding the pixel value to the motion compensated pixel retrieved from the pixel memory to obtain a decoded pixel value; and storing the decoded pixel values.

The blur processing mode may include: variable length decoding of blurry regions of a video signal to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients to scale the transform coefficients; performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients into pixel values; adding the pixel value to the motion compensated pixel retrieved from the pixel memory to obtain a decoded pixel value; storing the decoded pixel values; and processing the decoded pixel values to modify the current block of pixels such that the current block of pixels does not need to be decoded with reference to any portion of the insertion window of the video signal.

The processing of the decoded pixel values may include: performing a transform process on the decoded pixel values to regenerate scaled transform coefficients; performing a quantization operation on the scaled transform coefficients to re-scale the transform coefficients to achieve a desired data rate; and variable length coding of the rescaled transform coefficients to provide an intra-coded block of pixels at a desired data rate in place of the current block of pixels.

Alternatively, the processing of the stored decoded pixel values may comprise: performing a motion compensation operation on the stored decoded pixel values to reconstruct a blurry region of the video signal; performing a transform process on the motion compensated pixel values to regenerate scaled transform coefficients; performing a quantization operation on the scaled transform coefficients to re-scale the transform coefficients to achieve a desired data rate; and performing variable length coding of the re-scaled transform coefficients using the motion vectors from the motion compensation operation to provide a modified block of pixels in the video signal at the desired data rate in place of the current block of pixels.

The insertion processing mode may include: variable length decoding of an inserted window portion of a video signal to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients to scale the transform coefficients; performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients to pixel values corresponding to the insertion window portion; adding the pixel value to the motion compensated pixel retrieved from the pixel memory to obtain a decoded pixel value; storing decoded pixel values corresponding to the insertion window portion; combining the pre-stored pixel values corresponding to the overlay content with at least a portion of the stored decoded pixel values corresponding to the insertion window portion to provide modified decoded pixel values; performing a transform process on the modified decoded pixel values to provide modified scaled transform coefficients; performing a quantization operation on the modified scaled transform coefficients to rescale the modified transform coefficients to achieve a desired data rate; and variable length coding of the rescaled modified transform coefficients to provide a modified video signal at a desired data rate.

The pre-stored pixel values corresponding to the overlay content may be obtained by: variable length decoding of the overlay content signal to provide quantized transform coefficients corresponding to the overlay content; performing an inverse quantization operation on the transform coefficients corresponding to the overlay content to scale the transform coefficients; performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients into pixel values corresponding to the overlay content; and storing pixel values corresponding to the overlay content to obtain pre-stored pixel values.

When the overlay content is inserted into the video signal, the process may transition from the normal processing mode to the insertion processing mode. Transitioning from the normal processing mode to the insertion processing mode may include: waiting for reception of an I frame in the video signal; switching from a normal processing mode to a limited processing mode; waiting to receive a next reference frame in a current portion of the video signal; setting the window mode to "on" to enable an insertion window in the video signal; switching to an insertion processing mode when processing a region of the video signal corresponding to the insertion window; switching to a blurring processing mode if a motion vector corresponding to a region of a video signal that is outside an insertion window portion references an insertion window portion in a previously decoded frame when processing the region; and otherwise switching to a limited processing mode.

The process may also transition back to the normal processing mode after completing the insertion of the overlay content. Transitioning to the normal processing mode may include: waiting for receiving a reference frame; setting an initial mode to "on" to initiate a reset of a memory for processing a reference frame; setting the window mode to "off" to prevent creation of an insertion window in the current portion of the video signal; waiting for a next frame of the video signal; determining whether a next frame is a B frame or other picture type to be displayed before the received reference frame; (a) if the next frame is a B frame: then the window mode is set to "on" to enable an insertion window in the video signal; and waiting for more frames after the next frame until the next reference frame is received; or (B) if the next frame is not a B frame: then the window mode is set to "off"; setting the initial mode to "off"; and switching to an abnormal processing mode.

A pixel combiner may be provided along with the first and second multiplexers. The first multiplexer may be provided for selecting at least one of the transform coefficients from the inverse quantization operation or the signal from the pixel combiner. The second multiplexer may be provided for selecting at least one of a pixel error from a quantization operation or a signal from a pixel combiner. The signal from the pixel combiner may include at least one of stored overlay content pixels or primary video content pixels obtained from processing of the video signal.

An overlay controller may be provided for controlling at least one of the first multiplexer, the second multiplexer, the pixel combiner, and the motion compensation operation. A synchronization signal may be provided for synchronizing the operation of the first multiplexer, the second multiplexer, the pixel combiner and the motion compensation operation.

Processing the overlay content signal and the insertion window portion of the decoded video signal to generate a modified insertion window portion of the video signal may comprise: processing the interpolated window portion of the video signal to obtain a corresponding first set of absolute pixel values; storing a first set of absolute pixel values corresponding to the insertion window portion; processing the overlay content signal to obtain a respective second set of absolute pixel values corresponding to the overlay content; and storing a second set of absolute values corresponding to the overlay content such that the modified insertion window portion is generated by processing the first and second sets of absolute pixel values.

Overlay content may be inserted into the video signal in place of the insertion window portion of the video signal to generate a modified insertion window portion. Alternatively, the overlay content may be mixed with the insertion window portion of the video signal to produce a modified insertion window portion.

Different overlay content may be provided in one or more overlay content signals. The tag may be appended to at least one of: (a) a video signal; and (b) at least one of the one or more overlay content signals. The tag may contain identification information. Overlay content may be selected from one of the one or more overlay content signals for insertion according to the identifying information.

The present invention encompasses methods, apparatus and systems corresponding to the above-described embodiments.

Drawings

The present invention will hereinafter be described in conjunction with the appended drawings, wherein like designations denote like elements, and:

FIG. 1 shows a block diagram of a prior art system for inserting overlay content into a video signal;

FIG. 2 shows a block diagram of an example embodiment of a system for inserting overlay content into a video signal, according to this disclosure;

FIG. 3 shows a block diagram of an example embodiment of a video processor according to this disclosure;

FIG. 4 shows a block diagram of a prior art open loop transrater;

FIG. 5 shows a block diagram of a prior art closed loop transrater with error control;

FIG. 6 shows a block diagram of an example embodiment of a hybrid slew rate system in accordance with the present disclosure;

FIG. 7 shows a block diagram of another example embodiment of a hybrid slew rate system in accordance with the present disclosure;

FIG. 8 shows a flowchart of the operation of an overlay controller according to an example embodiment of the present invention;

FIG. 9 shows a flow diagram for a transition from a normal processing mode to a mode compatible with overlay processing, according to an example embodiment of the present invention;

FIG. 10 shows a flow diagram for a transition from a mode compatible with an overlay processing mode to a normal processing mode, according to an example embodiment of the invention;

FIG. 11 shows a block diagram of an example embodiment of a pixel combiner according to this disclosure; and

FIG. 12 shows a block diagram of an example embodiment of a multiprocessor slew rate system with insert and cover capability in accordance with the present invention.

Detailed Description

The following detailed description provides exemplary embodiments only, and is not intended to limit the scope, applicability, or configuration of the invention. Indeed, the following detailed description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing embodiments of the invention. It being understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the invention as set forth in the appended claims.

The present invention provides a method, apparatus and system for inserting overlay content into a video signal. Additionally, certain example embodiments of this disclosure are directed to inserting overlay content into a video signal during video signal transrating.

A simple block diagram of an example embodiment of a video processing system 20 with insert and overlay capability according to this disclosure is shown in fig. 2. In this example, the video signals are received from one or more video sources, such as a satellite 21, a video camera 22 coupled to a video encoder 23, a video storage device/server 24, an IP network 25, and the like. The overlay content generator 26 creates an overlay content signal to be inserted into the incoming video signal, for example, using information provided from a console for user input. As an example, the overlay content generator 26 and user console may be a PC installed with software for generating text, graphical features, or more general video content. In this example, the PC may also include software or hardware for encoding the rendered video into a suitable compression format. The insertion or overlay of content into the main video signal is performed by a video processor 28, which video processor 28 receives the video signal and the overlay content signal via a network switch 27 or other suitable mechanism.

To manage which overlay content is poked into which video signal at video processor 28, a marker is appended to at least one of: (a) a video signal; and (b) at least one of the one or more overlay content signals. The tag contains identifying information for matching the overlay content to the appropriate video signal. Video processor 28 is then able to select overlay content from one of the overlay content signals according to the identification information and insert the selected overlay content into the video signal to generate modified video content.

The tag may be appended to the overlay content signal by the overlay generator 26. The tag may be appended to each incoming video signal by a tag processor 29 at the video source, such as satellite 21, video camera 22 coupled to video encoder 23, video storage device (server) 24, or IP network 25. The marking processor 29 may be implemented as part of an encoding device at the video source (e.g., as part of encoder 23 or video server 24) or as a separate device (e.g., downstream from IP network 25 or satellite 21).

The appended tag may be inserted into a header of the elementary video or audio stream, or it may be multiplexed into a packetized stream consisting of multiple video, audio and data streams corresponding to one or more programs. In this case, the header information may be packaged into one or more packets and assigned a unique Packet Identifier (PID). In the case of an MPEG-2 transport stream, the packet type and program correspondence are determined by matching these PIDs with entries listed in special packets called Program Association Table (PAT) and Program Map Table (PMT) contained in the same multiplexed stream.

In another example embodiment, a tag may be appended to the video signal and to each of the one or more overlay content signals. Video processor 28 may select overlay content from one of the overlay content signals by extracting a tag from the video signal and each of the one or more overlay content signals, comparing the tag from the video signal to each tag from the overlay content signals, and selecting the overlay content for insertion from the overlay content signal having a tag that best matches the tag extracted from the video signal.

The identifying information contained in the tag may include at least one of: identifying geographic information of a geographic region in which overlay content is to be inserted into the video signal; a downlink control device identifier; a destination QAM; channel numbering; an insertion start time; a duration of the overlay content; inserting an identifier; inserting a window position; insert window size; a classification identifier; mixing information; keywords to enable overlay content to match the video signal, etc.

For example, a tag with an insertion identifier may be appended to the video signal and used to match the video signal to the particular overlay content signal that video processor 28 should have received. Video processor 28 will identify this overlay content signal by matching the keywords in the overlay content signal tag with the insertion identifier contained in the tag appended to the video signal.

Geographic information such as zip code, downlink control device ID, oral ground QAM, channel number, etc. may be included in the tag to enable better targeted insertion of overlay content (e.g., for advertising). Content descriptors may also be included in the markup, which may include at least one of format information (text, still pictures, MPEG2, MPEG4 video, audio type, etc.) and corresponding component descriptors. Additionally, a text component descriptor may be provided that may include at least one of a list position, speed, font, and the like. Further, the markers may include video component descriptors, which may include at least one of resolution, location, speed of movement of the animation, and the like. An audio descriptor may be provided that may indicate a policy (which is not typically required) to replace the primary audio.

The blending information may include information to enable alpha blending of the overlay content with a corresponding portion of the video signal to obtain a modified video signal containing the overlay content.

The classification identifier may be used to assist video processor 28 in selecting the most appropriate overlay content to insert into the video signal at a particular time and location within the video signal or frame of the video signal. Which is a parameter that can be interpreted as a subject matter classification of the video signal with respect to a particular time, or which can be interpreted as a more general filtering filter conveying information such as the priority of existing windows, characteristics of the window background, monthly ground QAM, destination channel or downlink control device, etc. Note that the overlay content may or may not include audio, and the classification identifier may also specify whether to allow overriding of the audio provided with the primary video stream.

The process of selecting particular overlay content for insertion into the main video program may be implemented by first pre-assigning one or more classification identifiers to the tags for each available overlay content signal. Then, when a tag in the primary video stream signals an opportunity to insert, a classification identifier may be extracted from the tag in the primary video program and compared to one or more classification identifiers in each of the available overlay content signals. Any overlay content signal having a matching identifier will contain overlay content suitable for insertion into the main video program.

The classification identifier may include at least one of: theme information for the video signal, theme information for the overlay content, priority information for an existing insertion window, characteristics of the insertion window, audio override information for the overlay content, resolution of the overlay content, channel number, target program name for inserting the overlay content, regional program rating of the target program, delivery identifier for the target program, format descriptor, text component descriptor including at least one of text position, speed, font size, font type, and font color, and video component descriptor including at least one of video resolution, video position, video speed of animation, and the like.

The selection process can be further optimized in the case where an insertion opportunity has been signaled and there are more than one suitable overlay content signals. For example, in addition to pre-assigning one or more classification identifiers to each overlay content signal, a "quality of fit" parameter may also be pre-assigned to the overlay content signals. That is, for each category identifier, there may be a respective quality of fit parameter indicating the relevance of the content to the particular category. Thus, if there are multiple overlay content signals representing the same classification identifier, and if this identifier matches the identifier specified in the tag included in the primary video stream, the overlay content signal with the highest corresponding quality of fit parameter will be selected. This approach can be used to maximize the efficiency of targeted advertising when using partial-screen video insertion.

In an example embodiment using both the classification identifier and the quality of fit parameter, a tag may be appended to the video signal (e.g., at tag processor 29) and each of the one or more overlay content signals (e.g., at overlay content generator 26). A classification identifier may be provided as at least a portion of the identification information. A respective quality of fit parameter may be assigned to the overlay content signal (e.g., at the overlay content generator 26) for each classification identifier. The quality of fit parameter may indicate a relative correspondence between each overlay content and the classification identifier. In this example embodiment, overlay content signals having the same classification identifier as the video signal may be identified by video processor 28. Video processor 28 may then select the overlay content from the identified overlay content signals having the quality of fit parameter indicating the highest correspondence with the classification identifier.

The overlay content generator 26 may be located at a central distribution site. The video processor 28 may be located remotely from the overlay content generator 26. Multiple video processors 28 may be provided at respective remote locations, such as various downstream locations (e.g., cable or satellite headends or hubs, telephone company central offices or nodes, etc.). The overlay content signals may be forwarded from the overlay content generator 26 to at least one video processor 28 at a respective remote location for storage prior to insertion. For example, video processor 28 may be located at one of a cable headend, a central office, a cable distribution hub, a satellite distribution hub, or the like. In these cases, the identification information contained in the tag may include geographic information. The video processor 28 at a particular remote location may select overlay content from the overlay content signal for insertion into the video signal having geographic information corresponding to the particular remote location of the video processor 28. For example, the markers may be used to match the overlay content signals to a particular geographic region. Each video processor 28 may then compare the markers to pre-assigned region codes specific to the location of each respective video processor location. In this way, it is possible to create a different message for each video processor 28, as the video processor is now able to scan through all messages to find the message that best suits the local audience.

The selection and insertion of overlay content may be performed by the respective video processors 28 at those locations at least one remote location. Alternatively, the selection and insertion of the overlay content may be performed by the video processor 28 at a central location. Multiple copies of the modified video content may then be distributed from the central location to one or more remote locations for further distribution.

The tag may be appended to the overlay content signal by an overlay content generator 26 at a central distribution site. Such a marker may determine which of the one or more video processors 28 at respective remote locations will insert video content in a particular video stream.

The video processor may insert the overlay content signal into an insertion window of the video signal in place of the corresponding portion of the video signal. Alternatively, the overlay content may be inserted into an insertion window of the video signal and mixed with a corresponding portion of the video signal. Alpha blending may be used to blend the overlay content with the corresponding insertion window portion of the video signal.

An example embodiment of video processor 28 according to this disclosure is shown in fig. 3. The example embodiment of video processor 28 shown in fig. 3 includes an optional memory allocator 30 and an optional video transrater 31 for enabling modification of the data rate of the video stream while insertion of the overlay content occurs. Those skilled in the art will appreciate that the video processor 28 may be implemented without the transrating capability provided by the optional memory allocator 30 and optional video transrater 31, where insertion of overlay content is desired without any modification to the data rate of the resulting modified video stream. Alternatively, the video transrater 31 may be instructed to maintain the original data rate of the video signal.

In an example embodiment where a slew rate is desired, incoming packets of a video signal (e.g., a video signal from video sources 21, 22, 24, and/or 25, or an overlay content signal from overlay content generator 26 of fig. 2) are scanned by memory allocator 30 for header information specifying the horizontal and vertical dimensions of the encoded image. This information may be needed by video transrater 31 if it is provided with memory for storing one or more images of each incoming video signal. In addition to the optional memory space for the individual video signals, the transrater 31 also includes memory for overlay content to be inserted or overlaid on top of the video signals. An example of a prior art memory allocator is described in U.S. patent No. 7,046,677.

Incoming video packets are not sent directly to the video transrater 31 but are first stored into the packet dram 33 via a dynamic random access memory (dram) controller 32. The Central Processing Unit (CPU)34 is notified of the arrival of each packet by depositing a marker into the rx info module 36, which rx info module 36 communicates with the CPU 34. The tag identifies the packet and the CPU 34 maintains a list that matches the address in the packet dram 33 with information corresponding to the received packet.

Although the video transrater 31 is capable of handling multiple video streams, it must first be organized into complete frames and multiplexed at the boundaries between frames. CPU 34 tracks the sequence of packets making up each frame and determines the sequence in which the frame will be forwarded from packet dram 33 to video transrater 31. The CPU 34 instructs the dram controller 32 to forward the selected packets from the packet dram 33 to the video transrater 31 in the desired sequence.

In addition to adjusting the data rate of each stream, the video transrater 31 may also implement insertion and overlay. CPU 34 may analyze the identification information contained in the tags that are placed into rx information module 36 to determine whether a particular video stream has an insertion window that may be used to insert overlay content. Once an insertion opportunity is identified in a particular video stream, the CPU may select particular overlay content for insertion based on the identifying information contained in the overlay content and/or the markers of the video stream, as discussed in detail above. The CPU 34 may then direct the dram controller 32 to provide the appropriate packet from the packet dram 33 to the transrater 31. For example, the CPU 34 may direct the dram controller 32 to provide packets from the packet dram 33 to the transrater 31 that correspond to overlay content that has been matched to a particular video stream. The transrater 31 may blend the overlay content with the corresponding insertion window portion of the video signal using various alpha blending techniques.

Those skilled in the art will appreciate that in embodiments where transrating is not required, a suitable processor may be used in place of the memory allocator 30 and the video transrater 31 for implementing the insertion and overlay.

Once the video transrater 31 has processed the frames, the resulting sequence of packets (e.g., transrated packets and/or modified packets containing the overlay content) is returned to the packet dram 33 via the dram controller 32. At the same time, the CPU 34 is notified of each packet transfer. This is done by again depositing the marker into the rx info module 36 so that the CPU 34 is again aware of the location of each packet in the packet dram 33. In this case, the flag is provided by the transrater 31 to the rx information module 36. If the CPU 34 requires additional information about a particular video stream, it may submit a request to the dram controller 32 in order to receive data comprising any selected packets.

The CPU 34 also manages the sequencing and formatting of packets for eventual output. The statistical multiplexing scheme is easily implemented by managing the slew rate process to achieve similar video quality on each stream while utilizing the full capacity of the output channel. The CPU 34 manages the formation of the output multiplex by instructing the dram controller 32 to pass selected packets from the packet dram 33 to the tx reformatter module 38. In this case, the CPU 34 may also have the ability to modify the header (including the marker) of each packet as it passes through the tx reformatter module 38.

The video stream or overlay content stream with the markers can be pre-conditioned by modifying existing protocols, such as the SCTE-30 and SCTE-35 protocols, currently used to implement full screen digital ad insertion.

If the main video signal is transmitted at different resolutions at different times, the same overlay content may be sent at different resolutions. For example, "resolution" may be one of the parameters that determine "quality of fit," or different classification IDs may be assigned for different resolutions.

The same method can be extended for use with DPI (digital program insertion) in the sense that the video transmitted by the DPI server can become the "main video" signal during the time window. In these cases, the system providing the splicing functionality (or the DPI server itself) may insert the tags, and the classification IDs may be used to insert or overlay content targeted for the advertisements. In this way, the DPI server can leverage the capabilities of the present invention without having to modify the ad content itself. This provides flexibility to a small headend that only overlays or inserts on specific portions of the advertising content that was originally transmitted by the larger headend.

The topic classification about the main program can be balanced by the existing characteristics of the program. For example, an ATSC system may use a combination of parameters that are available at the time it handles, such as primary and secondary channel numbers, program names, regional ratings of programs, transport IDs, and so on.

Targeted advertising may be achieved by inserting the event originator's overlay content at a specific time. If an event/segment has many initiators, the topic information may use the same classification ID for all initiators but with different "fitness level" parameters. Of course, if an application wants to alternate between all initiators' inserted overlay content at different times, the "level of fit" parameter may be dynamically changed.

The overlay content signal may be transmitted to the video processor 28 via a variety of different transport protocols. If the overlay content happens to be a sub-window representing a full motion video, bandwidth and storage requirements may become critical. In these cases, it may be easier to transmit such data via the MPEG-2 transport protocol at times closer to the actual insertion time.

The overlay content generator 28 may also be enabled to make decisions on when and where the overlay content will be inserted. This gives the overlay content generator 28 the flexibility to overlay the main video signal at any time without having to wait and depend on the markers in the main video program. Alternatively, a flag may be provided in the overlay content signal header to override any of the markers in the main video program. This capability may be advantageous for emergency alert applications where an overlay signal consisting of a text message will have priority over all video programs. In this case, the overlay content signal header may provide information such as insertion time, location, size, etc. Other parameters, such as opacity, may be modified for each overlay content.

Those skilled in the art will appreciate that other effects may also be given to the overlaid content in accordance with the present invention.

A simple prior art video transrater 40 compatible with a compression standard such as MPEG-1, MPEG-2 or h.264 is shown in fig. 4. A Variable Length Decoder (VLD) module 42 receives a series of variable length codewords and reconstructs the quantized transform coefficients represented by these codewords. An Inverse Quantizer (IQ) module 44 performs inverse quantization steps in order to recover the proper scale and range of each transform coefficient. The quantizer (Q) module 46 again scales the transform coefficients again, but this time controls the amount of quantization in order to maintain the desired output data rate. Finally, Variable Length Encoder (VLE) module 48 further compresses the quantized transform coefficients by representing them as a series of variable length codewords.

The transrater 40 of fig. 4 has two significant disadvantages. First, they cannot be used to assist the insertion or overlay process, and second, the accuracy of the recompressed video signal begins to differ due to the predictive coding component of the compression process. That is, when quantizer module 46 introduces inaccuracies, the resulting artifacts may be amplified when information derived from the resulting quantized coefficients is used to predict new information in a subsequent frame. Both of these disadvantages of transrater 40 are addressed by a modified prior art transrater system 50 shown in fig. 5.

In the prior art slew rate system 50 shown in fig. 5, VLD module 42, IQ module 44, Q module 46, and VLE module 48 function as discussed above in connection with fig. 4. However, in the system 50 shown in fig. 5, the quantization error introduced by the slew rate process (in particular, by the quantization module 46) is provided at the output of the adder 52 as a result of subtracting the transform coefficient obtained from the IQ module 51 from the transform coefficient obtained from the adder 58. The blocks of transform coefficients from adder 52 are then converted back to the pixel domain by Inverse Discrete Cosine Transform (IDCT) module 53, and the resulting pixel errors are then stored in DRAM55, which DRAM55 is controlled by DRAM controller 54. When a pixel error is needed for prediction of other pixels in a nearby image frame, Motion Compensator (MC) module 56 retrieves the pixel error, and the pixel error is again converted to the coefficient domain by Discrete Cosine Transform (DCT) module 57. This result is added to the incoming stream of transform coefficients at adder 58, which prevents the accumulation of re-quantization artifacts.

The prior art transrating system shown in fig. 5 represents many predictive transform-based encoders, such as MPEG-1, MPEG-2, and h.264. Other compression systems may follow the same structure, but may include additional steps or alternate versions of the same module. For example, DCT is an example of a broader class of frequency-domain transforms, but any such transform may be used in place of DCT without changing the scope of this disclosure.

Disadvantages of the motion compensated slew rate system shown in fig. 5 are the added complexity and additional hardware cost, and the compression inefficiency due to the introduction of error compensation data at the output of the motion compensated tracking loop. Complexity is also introduced because DCT, IDCT, and motion compensators are typically implemented by processing all coefficients or pixels of each block. In contrast, when implementing the open-loop transrater in fig. 4, it is only necessary to process non-zero coefficients, and scan order transitions are negligible. This allows the use of a narrower data bus and achieves higher throughput. However, if a hybrid solution is employed according to an example embodiment of the present invention as shown in fig. 6, it is possible to benefit from the drift correction advantages of this closed loop transrater while minimizing the disadvantages.

The specific example embodiment of the hybrid slew rate system 60 of the present invention shown in fig. 6 includes one closed-loop subsystem 64 (e.g., the system 50 of fig. 5) and two dedicated open-loop subsystems 66, 68 (e.g., each corresponding to the fig. 4 slew rate 40), and the operation of these subsystems is substantially as described above in connection with fig. 4 and 5. The three subsystems may be operated simultaneously by providing different video frames to each of the subsystems simultaneously. Note that the tasks of the assigned frames and the scheduling packets corresponding to the plurality of video signals are processed by the CPU 34 shown in fig. 3, and therefore only software adjustment is required. Packets arriving at Demultiplexer (DMUX)62 in fig. 6 will be routed to the appropriate subsystem (e.g., subsystem 64, 66 or 68) according to the routing information provided in the packet header. The transrated packets may then be multiplexed together at a Multiplexer (MUX) 63.

The full cost and performance advantages of the hybrid transrating system 60 are achieved by only strategically assigning video frames to different transrating subsystems. With compression systems such as MPEG-2 and h.264, some types of frames (i.e., intra-coded or I-frames, and predictively coded or P-frames) are used to predict other frames, while frames of the third type (i.e., bi-directionally predicted or B-frames) are typically not (although h.264 allows the use of B-frames as reference frames for prediction). The frame types also differ in the way they are encoded. I-frames do not benefit from motion compensation and are therefore compressed least efficiently. However, for the same reason, these are the only types of frames that can be reconstructed by a decoder trying to synchronize with the new stream. This initial synchronization is necessary when tuning to a new stream or when recovering from errors caused by noise in the transmission channel. For this reason, the encoder typically selects the I-frame encoding method at intervals sufficient for decoder initialization or resynchronization.

P-frames benefit from motion compensation using motion predictors that reference previous frames. Both I and P frames can serve as reference frames when motion compensation is used to predict other frames. Thus, any error introduced into an I-frame or P-frame may be magnified each time the affected region of a reference frame is accessed. The closed loop slew rate process of subsystem 64 prevents the accumulation of this error and for this reason it is advantageous to select a closed loop slew rate subsystem 64 to slew rate for both I and P frames.

In general, B-frames are most frequently used because they benefit from multiple motion compensated predictors referencing either I-frames or P-frames in the forward or backward direction. Since these frames are not typically used to predict other pictures, drift correction becomes less important and the reduced compression efficiency due to correction may even outweigh any advantages resulting from reduced drift within a single image. Therefore, it is advantageous to route B-frames through one of the open-loop transrating subsystems 66 or 68. Note that in the case of h.264, it makes more sense to identify pictures that are not used as reference frames for other pictures and to only assign these pictures to open-loop transrating subsystem 66 or 68.

Many encoders are configured to use the B-frame encoding method for 2 out of every 3 frames or pictures, and this is the reason for matching 2 specialized open-loop transraters 66 and 68 with a single closed-loop transrater 64 in the example embodiment of fig. 6. However, open-loop and closed-loop subsystems may be provided in any combination. Note that in the case where the open loop systems 66 and 68 are oversubscribed, B frames are easily forwarded to the closed loop transrater 64. Similarly, if the closed loop transrater 64 is overbooked, it is possible to forward an I-frame or a P-frame to one of the open loop transraters 66 or 68, however, the benefits of drift compensation will not be realized during such periods.

Both the prior art closed-loop transrating system 50 in fig. 5 and the example embodiment of the hybrid transrating system 60 in fig. 6 may be adjusted to support insertion and overlay. Fig. 7 shows an example embodiment of a slew rate system 70 supporting insertion and overlay in accordance with the present invention.

In the example embodiment of the invention shown in fig. 7, an overlay content signal containing overlay content is received (e.g., from overlay generator 26 of fig. 2) and decoded at VLD module 42. In addition, a compressed video signal is received from a video source (e.g., video source 21, 22, 24, or 25 of fig. 2) and decoded at VLD module 42. An insertion window portion of the decoded video signal is identified at the overlay controller 71. The transrate system 70 processes the overlay content signal and the insertion window portion of the decoded video signal to generate a modified insertion window portion of the video signal. The transrate system 70 then processes the video signal and the modified insertion window portion to generate a modified video signal.

The data rate of the video signal may be modified during the overlay content insertion process. A closed loop transrating path 72 may be provided for modifying the data rate of the video signal. In addition, an open-loop transrating path 73 may also be provided for modifying the data rate of the video signal (e.g., open-loop transrating path 73 may be provided by setting MUX 80 to select only input 0 from IQ module 44). The closed-loop transrating path 72 may be selected for modifying the data rate of the inserted window portion of the video signal.

Pixel quantization error control may be provided in the closed loop transrating path 72. Pixel quantization error control may include obtaining an error from quantization of transform coefficients of an incoming pixel of the video signal in the form of an error transform coefficient. For example, the quantization error introduced by the slew rate process (in particular, by quantization module 46) is provided at the output of adder 52 as a result of subtracting the transform coefficient obtained from IQ module 51 from the transform coefficient obtained from MUX 80. The blocks of transform coefficients from adder 52 are then converted back to the pixel domain by Inverse Transform (IT) module 74. These pixel errors may be stored in DRAM55 and then retrieved during modification of the data rate of the associated pixel. Pixel errors in DRAM55 may be added to incoming pixels obtained from the inverse transform process at IT module 75 at pixel combiner 76 to obtain corrected pixels in order to prevent the accumulation of re-quantization artifacts during quantization of transform coefficients corresponding to the incoming pixels at Q module 46. A transform process may be performed on the corrected pixels at transform (T) module 77 to provide error corrected transform coefficients for quantization at Q module 46.

It should be appreciated that the transform process performed at T module 77 may be a discrete cosine transform process (DCT) or any other transform process that may be used with newer codecs, such as h.264. Similarly, the inverse transform process performed at the IT modules 74 and 75 may be an inverse discrete cosine transform process (IDCT) or any other transform process that may be used with newer codecs (e.g., h.264).

The closed loop transrating path may be selected for modifying the data rate of I-frames and P-frames of the video signal. The open-loop transrating path may be selected for modifying the data rate of B-frames of the video signal.

The closed-loop transrating path 72 and the open-loop transrating path 73 may operate simultaneously, for example, when processing frames of multiple video signals. For example, the closed-loop transrating path 72 may be selected for modifying the data rate of I-frames and P-frames of one video signal, and the open-loop transrating path 73 may be selected for modifying the data rate of B-frames of a different video signal. In some cases, where the closed-loop transrating path 73 is oversubscribed and there is available processing bandwidth in the open-loop transrating path 72 for I-frames and P-frames, both the open-loop transrating path 73 and the closed-loop transrating path 72 may be used to process I-frames and P-frames of the video signal. Similarly, where the open-loop transrating path 72 is overbooked and there is available processing bandwidth for B-frames in the closed-loop transrating path 73, both the open-loop transrating path 73 and the closed-loop transrating path 72 may be used to process B-frames of the video signal. One or more additional open loop transrating paths may be provided for modifying the data rate of the video signal, as discussed below in connection with fig. 12. The two open-loop transrating paths 121 and 122 and the closed-loop transrating path 72 may operate simultaneously. Those skilled in the art will appreciate that additional open or closed loop slew rate paths may be provided as needed in order to increase the throughput of the system.

Note, however, that the system 70 of fig. 7 is a transrating system capable of coverage processing. When in the overlay processing mode, drift compensated closed loop slew rate cannot be performed. However, it is still possible to perform open loop slew rates while the overlay process is taking place.

The open-loop transrating path 73 may include: variable length decoding of the video signal at VLD module 42 to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients at IQ module 44 to scale the transform coefficients; performing a quantization operation on the scaled transform coefficients at Q module 46 to re-scale the transform coefficients to achieve a desired data rate; and variable length coding of the re-scaled transform coefficients at VLE module 48 to provide a video signal at a desired data rate.

In another example embodiment of the present disclosure, at least one of an obfuscation processing mode and an insertion processing mode may be provided for carrying out a processing step. The blur processing mode is operable for processing a blur region of a video signal in which a motion vector of a current block of pixels of the video signal references a portion of an insertion window. The insertion processing mode is operable for processing an insertion window portion of the overlay content signal and the video signal to generate a modified insertion window portion. The overlay controller 71 may identify which processing mode is to be used by the system 70, as discussed in detail below in connection with fig. 8.

Additionally, a normal processing mode may be provided for processing the video signal when the video signal does not contain an insertion window. The normal processing mode may include: variable length decoding of the video signal at VLD module 42 to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients at IQ module 44 to scale the transform coefficients; performing an inverse transform process on the scaled transform coefficients at IT module 75 to convert the scaled transform coefficients into pixel values; summing the pixel values with the motion compensated pixel errors retrieved from DRAM55 at pixel combiner 76 to provide error compensated pixel values; performing a transform process on the error compensated pixel values at T module 77 to generate error compensated transform coefficients; performing a quantization operation on the error compensated transform coefficients at Q module 46 to re-scale the transform coefficients to achieve a desired data rate; and variable length coding of the re-scaled transform coefficients at VLE module 48 to provide a video signal at a desired data rate.

The motion compensated pixel error is obtained by: the error is obtained from the quantization operation at the Q module 46 in the form of a pixel error and stored in DRAM55 (as discussed above). The pixel error is retrieved from DRAM55 when performing a motion compensation operation on the associated pixel. The associated pixels and pixel errors may then be processed at the MC module 78 to provide motion compensated pixel errors. Summing the pixel values from the IT module 57 with the motion compensated pixel errors at the pixel combiner 76 prior to the transform process at T module 77 prevents the accumulation of re-quantization artifacts during the quantization operation at Q module 46. A quantization operation may then be performed on the error compensated transform coefficients at Q module 46 to re-scale the transform coefficients to achieve the desired data rate without accumulation of quantization artifacts.

A restricted processing mode may be provided for carrying out at least one of the processing steps when the obfuscation processing mode, the insertion processing mode and/or the normal processing mode are not selected. The restricted processing mode may include: variable length decoding of the video signal at VLD module 42 to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients at IQ module 44 to scale the transform coefficients; performing a quantization operation on the scaled transform coefficients at Q module 46 to re-scale the transform coefficients to achieve a desired data rate; variable length coding of the re-scaled transform coefficients at VLE module 48 to provide a video signal at a desired data rate; performing an inverse transform process on the scaled transform coefficients at IT module 75 to convert the scaled transform coefficients into pixel values; adding the pixel values to the motion compensated pixels retrieved from the pixel memory (DRAM 55) at the pixel combiner 76 to obtain decoded pixel values; and storing the decoded pixel values in DRAM 55.

The blur processing mode may include: variable length decoding of blurry regions of the video signal at VLD module 42 to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients at IQ module 44 to scale the transform coefficients; performing an inverse transform process on the scaled transform coefficients at IT module 75 to convert the scaled transform coefficients into pixel values; adding the pixel values to the motion compensated pixels retrieved from the pixel memory (DRAM 55) at the pixel combiner 76 to obtain decoded pixel values; storing the decoded pixel values in DRAM 55; and processing the decoded pixel values to modify the current block of pixels such that the current block of pixels does not need to be decoded with reference to any portion of the insertion window of the video signal.

The processing of the decoded pixel values may include: performing a transform process on the decoded pixel values at T module 77 to regenerate the scaled transform coefficients; performing a quantization operation on the scaled transform coefficients at Q module 46 to re-scale the transform coefficients to achieve a desired data rate; and variable length encoding of the re-scaled transform coefficients at VLE module 48 to provide an intra-coded block of pixels at a desired data rate in place of the current block of pixels.

In another example embodiment, the processing of the stored decoded pixel values may comprise: performing motion compensation operations on the stored decoded pixel values at the MC module 78 to reconstruct a blurry region of the video signal; performing a transform process on the motion compensated pixel values at T module 77 to regenerate the scaled transform coefficients; performing a quantization operation on the scaled transform coefficients at Q module 46 to re-scale the transform coefficients to achieve a desired data rate; and variable length encoding of the re-scaled transform coefficients at VLE module 48 using new motion vectors obtained from the motion compensation operation performed at MC module 78 to provide a modified block of pixels in the video signal at the desired data rate in place of the current block of pixels.

The insertion window portion of the overlay content signal and the decoded video signal may be processed with the transrater 70 set to the insertion processing mode to generate a modified insertion window portion of the video signal. The insertion processing mode may include: variable length decoding of the inserted window portion of the video signal at VLD module 42 to provide quantized transform coefficients; performing an inverse quantization operation on the transform coefficients at IQ module 44 to scale the transform coefficients; performing an inverse transform process on the scaled transform coefficients at IT module 75 to convert the scaled transform coefficients to pixel values corresponding to the insertion window portion; adding the pixel values to the motion compensated pixels retrieved from the pixel memory (DRAM 55) at the pixel combiner 76 to obtain decoded pixel values; storing the decoded pixel values corresponding to the insertion window portion in DRAM 55; combining, at a pixel combiner 76, the pre-stored pixel values corresponding to the overlay content with at least a portion of the stored decoded pixel values corresponding to the insertion window portion to provide modified decoded pixel values; performing a transform process on the modified decoded pixel values at T module 77 to provide modified scaled transform coefficients; performing a quantization operation on the modified scaled transform coefficients at Q module 46 to re-scale the modified transform coefficients to achieve a desired data rate; and variable length encoding of the rescaled modified transform coefficients at VLE module 48 to provide a modified video signal at a desired data rate.

The pre-stored pixel values in the DRAM55 corresponding to the overlay content may be obtained by: variable length decoding of the overlay content signal at the VLD module 42 to provide quantized transform coefficients corresponding to the overlay content; performing an inverse quantization operation on the transform coefficients corresponding to the overlay content at IQ module 44 to scale the transform coefficients; performing an inverse transform process on the scaled transform coefficients at IT module 75 to convert the scaled transform coefficients into pixel values corresponding to the overlay content; and storing the pixel value corresponding to the overlay content in the DRAM55 to obtain the pre-stored pixel value corresponding to the overlay content.

The overlay content may be inserted into the video signal in place of the insertion window portion of the video signal to produce a modified insertion window portion. In this case, the pixel combiner 76 would replace the decoded pixel values corresponding to the inserted window portion of the video signal with the absolute pixel values corresponding to the overlay content. Alternatively, the overlay content may be mixed with the insertion window portion of the video signal to produce a modified insertion window portion. In this case, pixel combiner 76 would blend the decoded pixel values corresponding to the insertion window portion of the video signal with the absolute pixel values corresponding to the overlay content using, for example, an alpha blending technique.

A system operating in a normal processing mode (e.g., utilizing the closed-loop transrating path 72) must transition to a limited processing mode (e.g., utilizing the open-loop transrating path 73) before coverage can be performed in an insertion processing mode. The flowchart of fig. 9 shows an example embodiment in which such a transition from the normal processing mode is performed without introducing video artifacts. The first step of transitioning from the normal processing mode (at the beginning, step 901) to the insertion processing mode may include waiting to receive an I-frame in the video signal (step 902). The I-frame is required because the pixel errors currently stored in DRAM55 must be replaced by fully decoded pixels. The I-frame serves as the access point that initiates the decoding process. Although it would be possible to insert the overlay content into this I-frame, it would not be possible to insert the overlay content into any non-reference frame (e.g., B-frame) immediately following the reference frame. Accordingly, insertion of the overlay content is deferred until the next reference frame is encountered (step 904). At the same time, the processing mode type (mbx type) is set to the restricted processing mode (step 903) in order to decode the reference frame and store the reference frame in DRAM55, and to avoid accessing DRAM55 while processing a non-reference frame that may follow. The transition is complete when the next reference frame is encountered at step 904. At this point, the window mode is switched from "off" to "on" to enable an insertion window in the video signal (step 905). Completion of this transition enables the regular overlay process to begin and the processing mode can then switch from the limited processing mode to the insertion processing mode to process the region of the video signal corresponding to the insertion window. When processing a region of a video signal that is outside an insertion window portion, the processing mode may switch from the limited processing mode to the blurry processing mode if a motion vector corresponding to the outside region references the insertion window portion in a previously decoded frame.

Transitions from the overlay processing mode back to the off-normal slew rate mode must also be sequenced in order to avoid introducing visible artifacts. The flowchart in FIG. 10 shows an example embodiment of a suitable sequenced transition to normal processing mode. At the beginning (step 1001), the transrater 70 is set to the insertion processing mode. The first step of transitioning from the insertion processing mode to the normal processing mode may include waiting to receive a reference frame (step 1002). Once the reference frame is received, the initial mode may be set to "on" to initiate a reset of the memory (e.g., DRAM 55) used to process the reference frame (step 1003). For example, during this process, data that would normally be written to DRAM55 during the process of processing the reference frame would be discarded and replaced with zeros instead. This initializes the DRAM55 in the same manner as the re-quantization error during the closed loop operation of the I-frame or other reference frame reset transrater 70. At the same time (step 1003), the window mode is set to "off" to prevent the creation of an insertion window in the current portion of the video signal (e.g., in the current frame). It should be noted, however, that the overlay window may not be suppressed in any frame (e.g., B frame) that is received after the reference frame (but will be displayed before the I or P frame or other reference frame) because the information needed to reconstruct the blurred block is no longer available. Accordingly, once the next frame of the video signal is received (step 1004), it may be determined whether the next frame is a B frame or other frame to be displayed prior to the reference frame received at step 1002. If the next frame is a B frame, the window mode may be set to "on" to enable an insertion window in the video signal, thereby allowing the overlay window to remain visible when these unordered pictures are displayed. Otherwise, the window mode may be set to "off," the initial mode may be set to "off," and the processing mode may be switched to the normal processing mode (step 1006).

The transition procedure can be easily adjusted to accommodate advanced codecs such as h.264 that can utilize more than two reference frames for motion compensation. For this reason, the transrater 70 need only keep track of where the overlay window may be located in each reference frame. For example, if a motion vector references a coverage area, but if the coverage window is not active when processing a particular reference frame, then the coverage can be ignored and no special processing is necessary. Similarly, if the region of the overlay window that has just been deactivated is processed, special processing is necessary only if the corresponding overlay region of the reference frame and the overlay window are active in processing the reference frame.

A pixel combiner 76 may be provided along with a first multiplexer 80 and a second multiplexer 81, as shown in fig. 7. The first multiplexer 80 may be provided for selecting at least one of the transform coefficients from the inverse quantization operation at IQ module 44 or the signal from pixel combiner 76 (e.g., via T module 77). The second multiplexer 81 may be provided for selecting at least one of the pixel errors from the quantization operation (e.g., provided by the IT module 74) or the signal from the pixel combiner 76. The signal from the pixel combiner 76 may include at least one of stored overlay content pixels or primary video content pixels obtained from processing of the video signal.

An overlay controller 71 may be provided for controlling at least one of the first multiplexer 80, the second multiplexer 81, the pixel combiner 76, and the MC module 78. A synchronization signal may be provided from synchronizer 79 for synchronizing the operation of first multiplexer 80, second multiplexer 81, pixel combiner 76, and MC module 78. The synchronization signal may be generated by synchronizer 79 in response to a signal from overlay controller 71 identifying which processing mode (e.g., normal, limited, fuzzy, or interpolated) is to be used. The synchronizer may also be used to synchronize the inverse quantization operation, the quantization operation, and the variable length coding operation, but the signal path of the synchronization signal is not shown in the figure.

Different overlay content may be provided to the transrater 70 in one or more overlay content signals. The tag may be appended to at least one of: (a) a video signal; and (b) at least one of the one or more overlay content signals, as discussed above in connection with fig. 2 and 3.

A block diagram of an example embodiment of a pixel combiner 76 according to the present disclosure is provided in fig. 11. The pixel combiner 76 may be configured by the signal from the synchronizer 79 to perform simple insertion (e.g., add overlay content into the video signal), or it may be configured to overlay content on top of an existing frame of the video signal (e.g., mix the overlay content with a portion of the video signal). For example, synchronizer 79 may send control signals that are received and decoded at decoder 111 of pixel combiner 76. In the case of a simple insertion, the multiplexer 112 will select the input from the inverse transform module 75, and the adder 113 will add this signal to the output from the MC module 78 of fig. 7. The resulting pixel values will be forwarded directly to the transform module 77 via the multiplexer 114, thereby bypassing the mixer module 115.

In the case of a mixing operation, the multiplexer 114 will select the output of the mixer module 115 to forward to the transform module 77. Many mixing effects are possible and in this case it is advantageous to utilize a portion of the DRAM55 to control the mixing operation. For example, in addition to storing luma and chroma information for each pixel to be inserted into a frame, it is also useful to allocate additional bits to guide the blending process. These control bits may control the weighting (alpha blending) of both the interpolated and background pixels, among other special effects. Note that if blending is to be performed, a second access to DRAM55 is required to retrieve the background pixels. When retrieving background pixels from DRAM, the MUX module 112 is configured to select the input fixed at zero, thereby causing the signal from DRAM received via the MC module 78 to be forwarded directly to the output of the adder 113.

The pixel combiner in fig. 11 assumes that pixels are delivered from DRAM55 (via MC module 78) in some order by DRAM controller 54. If blending is to be applied for a particular block, the background pixels are first retrieved from the master picture, followed by control bits that direct the blending process, and finally the pixels that make up the corresponding block of the insertion window. Simple fixed delay block FIFO memories 116 and 117 are then used to provide control data to input port 3 of mixer 115, background data to input port 2 of mixer 115, and insertion data at input port 1 of mixer 115.

Referring to the example embodiment shown in FIG. 7, consider the case where MUX 80 is configured to select input 1, MUX 81 is configured to select input port 0, and the pixel combiner simply adds the input at port 1 from MC module 78 with the input at port 0 from IT module 75. The resulting system thus becomes functionally equivalent to the closed loop transrater of fig. 5. Note that the first adder 58 in fig. 5 has been replaced by a pixel combiner 76 in the fig. 7 example embodiment, and a second IT module 75 has been added to allow the pixel combiner 76 to operate on pixels instead of transform coefficients. This modification is advantageous when the overlay process is implemented and pixels from two different images (i.e., images from the overlay content and the original signal) are combined.

Overlay content should be received, processed and stored in DRAM55 before the overlay or insertion is implemented. During insertion, the overlay content must be stored in DRAM55 as decoded pixels rather than pixel errors for error compensation. Essentially, during the insertion process, the system must perform a decoding function rather than a slew rate function. This result is achieved by configuring MUX 81 to select input 1 and configuring pixel combiner 76 to add the input at port 1 from MC module 78 and the input at port 0 from IT module 75. Note that the overlay content encoded as an I-frame may be forwarded directly from input port 0 of pixel combiner 76 to the output port of pixel combiner 76.

One of the problems that can occur when processing frames or other portions of the signal containing an insertion window is that the motion compensator MC 78 may need to access a region of the reference frame that coincides with the position of the insertion window. The problem is that the decoder has lost this reference frame information because this section of its memory has now been replaced by the contents of the inserted window. Thus, the decoder is no longer synchronized with the original encoding process, and the motion compensation process can lead to severe visible artifacts, such as in the case of this. One task of the example embodiment shown in fig. 7 is to repair or prevent this breach of the integrity of the signal. However, this is only possible if the transrater 70 has access to reference frame pixels that the decoder has lost. Unfortunately, the closed loop transrater of fig. 5 only maintains pixel errors in DRAM55, and this information is not available to prevent motion compensation errors. One solution is to duplicate DRAM55 and associated input and output modules in order to maintain access to both the pixel error required for the conversion rate and the absolute pixel required for the repair motion compensation process. The second option is to switch from the closed loop slew rate mode to the open loop slew rate mode while the insertion is being performed. This is the preferred solution and will be described further in view of a slight but temporary reduction in cost, complexity and image quality. However, the memory copy selection is more straightforward for those skilled in the art and should not require additional explanation.

The overlay controller 71 in fig. 7 manages the combination of the insert video (overlay content) and the main video signal. The overlay controller 71 may generate an MBX type code that specifies which processing mode (e.g., normal, interpolated, restricted, or obfuscated processing mode) the system will use. The MBX type code is forwarded from overlay controller 71 to synchronizer 79 where it is used to configure the settings of MUX 80, MUX 81, pixel combiner 76 and MC module 78. The settings corresponding to each MBX type result are listed in table 1 below. Note that both blocks are processed when the MBX type is set to insert, so that both background pixels and pixels to be inserted are delivered to the pixel combiner.

TABLE 1

When operating in closed loop slew rate mode, the MBX type must be set to normal. However, this mode cannot be used when processing pictures that are to be modified by inserting overlay content. In fact, the system must transition out of normal mode, as previously described with reference to fig. 9. The flow chart in FIG. 8 depicts an example of the operation of the overlay controller 71 once the system has completed the transition from the normal processing mode.

Typically, when processing a picture that includes one or more insert windows, the overlay controller 71 will set the MBX type to restricted. This causes synchronizer 79 to configure MUX 80 to select input 0. Meanwhile, the pixel combiner 76 is configured to add data received at port 1 from the MC module 78 with port 0 from the IT module 75, and the MUX 81 is configured to select the input from port 1 (i.e., the signal from the pixel combiner 76) thereby forwarding the output from the pixel combiner 76 to the DRAM55 via the memory controller 54. In effect, the system 70 decodes and stores incoming frames while they are transrated.

At step 802 of fig. 8, the location of the macroblock is compared to the location of the insertion window. If the macroblock does not fall within the insertion window, overlay controller 71 may determine that decoding of the particular block requires access to a portion of the reference frame that overlaps with the insertion window (step 803 of fig. 8). If the block uses motion compensation, each applicable motion vector (with coordinates mvx, mvy) must be compared to the current position in order to detect possible references to the footprint of the corresponding reference frame. If MPEG-1 or MPEG-2 is used, prediction is limited to one or two reference frames, but more advanced codecs such as H.264 can use more than two reference frames. If any of the motion vectors result in any kind of overlap with the footprint of the reference frame, the MBX type is set to blurry, step 806, and special processing is required. Note from the flowchart in fig. 8 that the MBX type may also be set to fuzzy if the footprint of the current frame is processed and the window mode is set to off (step 805). This is necessary to prevent transient artefacts when opening or closing the overlay window. This is described in detail above in connection with fig. 9 and 10.

When the MBX type is set to fuzzy, the block is decoded and stored in the usual way. However, the block needs to be modified before it can be transrated and sent to the decoder. A preferred way to do this is to regenerate the blocks using intra coding instead of motion compensated predictive coding. This is accomplished by setting MUX 80 to select input 1. In this way, the decoded pixels provided at the output of pixel combiner 76 are transformed by transform module 77, then re-quantized by quantizer block 46 and further compressed by VLE module 48. The block may then be decoded since predictive coding is not involved. Alternatively, the block may be recompressed by changing the motion vector so that it no longer accesses the area that overlaps the insertion window. However, this would require another access to DRAM55 and additional logic to derive the appropriate alternative motion vector.

If it is determined at step 802 that the macroblock falls within the boundaries of the insertion window, and if the window mode is on, step 805, then an insertion processing mode will be selected, step 810 or step 811. However, additional logic is included in the flowchart of fig. 8 in order to determine whether a macroblock should be intra-coded or coded using motion compensated prediction. If it is determined at step 807 that the picture is an I frame, then the intra mode is automatically selected (step 811).

The overlay controller 71 is able to accommodate time varying insertions. This is controlled by the horizontal and vertical motion vector codes, dmvx and dmvy, respectively, which can be included in the header before the overlay content. This vector represents the displacement relative to the most recent frame of overlay content to be displayed in the insertion window. For complex motions, dmvx and dmvy can be selected to reference the location of the independent images in dram. In this case, the use of motion compensation will be prevented by the comparison step 808 of the flowchart in fig. 8, and the overlay content will instead be intra-coded. However, for simple displacements, such as horizontal or vertical scrolling of text messages, dmvx and dmvy may specify horizontal and vertical displacements relative to the most recently inserted image. Depending on the direction and amount of displacement, the comparison step 808 will now allow motion compensation to be used in most regions of the insertion window.

The purpose of the comparison step 809 is to allow the use of the parameters bx and by signalling mix effect. In this case, bx specifies the width of the blending region at the left and right edges of the insertion window, and by specifies the height of the blending region at the top and bottom edges of the insertion window. If mixing is used in the entire zone, bx and by should be set to at least the horizontal and vertical window sizes of the insertion window, respectively. If the block is to be rendered without the blending effect, motion compensation is allowed (step 810). Similarly, if no mixing is used at all, bx and by should be set to 0. The overlay controller 71 will avoid using motion compensation in the areas where special blending effects have been applied and will instead use intra coding (step 811).

It should now be appreciated that several efficiencies can be achieved by implementing the insertion and overlay in a design that supports video slew rates. Not only are the hardware and software requirements similar, but the changes in data rate resulting from the modification of the video content may emphasize the requirements for data rate control. It has been shown that the insertion process can be combined with closed loop video transrating design according to example embodiments of this disclosure. It has also been shown that the slew rate process may be advantageously distributed across multiple closed loop and open loop processors (also referred to herein as "slew rate paths" or "slew rates") according to example embodiments of the present invention.

According to example embodiments of this disclosure, a hybrid multiprocessor transrater design (e.g., of fig. 6) may also be modified to support video insertion and overlay. The system in fig. 12 is an example embodiment of a multiprocessor transrater 120 with insertion and overlay capabilities in accordance with the present invention. As before, when sending B pictures (or other types of non-reference frames) to either of the two dedicated open-loop transrating paths 121 and 122, it is advantageous to send I and P pictures (or other types of reference frames) to the closed-loop transrating path 72. As discussed above in connection with fig. 7, an additional open-loop slew-rate path 73 may be provided by setting MUX 80 to accept only input 0.

Several differences between this system 120 of FIG. 12 and the multiprocessor design of FIG. 6 that does not support interleaving should be noted. As with fig. 7, the adder in fig. 6 has been replaced by a pixel combiner module 76, and a second transform module (T)77 has been added to allow the pixel combiner 76 to operate on pixels rather than transform coefficients. It should also be noted that the two open loop slew rate paths 121 and 122 are now able to access data retrieved from DRAM55 via MC 78, pixel combiner 76, T block 77, Q block 46, and MUX blocks 80, 84, and 85. This path is utilized when the overlay controller 71 associated with one of the open-loop transrating paths 121, 122 determines that data needs to be inserted (MBX type: insert) or if motion prediction requires access to a portion of the reference frame that overlaps with the insertion window (MBX type: blur). Also, to implement the mixing effect, a MUX82 has been added so that data from the special open loop transrating paths 121 and 122 can be forwarded to the pixel combiner 76 via IT module 75.

Notably, efficient implementation of the VLD and VLE modules generally benefits by operating on only non-zero coefficients, while the transform module, inverse transform module, motion compensator, pixel combiner, and DRAM controller are generally designed to receive and process every coefficient or pixel present. Although the example embodiment of fig. 12 does not show the scan conversion modules needed to bridge the two sets of components together, these modules will be placed optimally at the output of MUX82 in closed loop circuit 72, as well as at the output of Q module 46. A larger buffer is also required for effective decoupling between the different processing rates of the two sets of components. Note that the operation of IQ module 44 and Q module 46 of open-loop subsystems 121 and 122 and first IQ module 44 of closed-loop subsystem 72 will be limited to non-zero coefficients.

The synchronizer module 79 has been modified to accept parameters from each of the VLD module 42 and the overlay controller module 71. The output of synchronizer 79 is used to control new MUX modules 82, 84, and 85 in addition to the pre-existing closed loop module MUX 80, MUX 81, and pixel combiner 76. Each of the three processors submits the MBX type parameters to the synchronizer 79 and waits for the synchronizer 79 to accept the request. For each block to be processed, the synchronizer 79 selects one of the three MBX type parameters and configures certain modules as specified in table 2 below. In most cases, there will be no contention for resources, and the synchronizer 79 will be able to accommodate requests from two or more processors simultaneously. However, if synchronizer 79 determines that more than one bank requires access to a shared resource, such as DRAM55 or pixel assembler 76, the requests will be sequenced and processed one at a time.

TABLE 2

It should now be appreciated that the present invention provides advantageous methods, apparatus and systems for inserting overlay content into a video signal. In addition, the present invention provides advantageous methods, devices and systems for inserting overlay content into a video signal during video signal transrating.

While the invention has been described in connection with various illustrated embodiments, many modifications and adaptations may be made thereto without departing from the spirit and scope of the invention as set forth in the claims.

Claims (38)

1. A method for inserting overlay content into a video signal, comprising:

receiving an overlay content signal containing overlay content;

receiving a compressed video signal;

decoding the compressed video signal;

identifying an insertion window portion of the decoded video signal;

processing the overlay content signal and the insertion window portion of the decoded video signal to generate a modified insertion window portion of the video signal; and

processing the video signal and the modified insertion window portion to generate a modified video signal.

2. The method of claim 1, further comprising:

providing a closed loop transrating path for modifying a data rate of the video signal;

providing an open-loop transrating path for modifying the data rate of the video signal; and

selecting the closed-loop transrating path for modifying the data rate of the insertion window portion of the video signal.

3. The method of claim 2, further comprising:

pixel quantization error control is provided in the closed loop slew rate path.

4. The method of claim 3, wherein the pixel quantization error control comprises:

obtaining an error in the form of error transform coefficients from quantization of transform coefficients of incoming pixels of the video signal;

performing an inverse transform process on the error transform coefficients to provide pixel errors in a pixel domain;

storing the pixel error;

retrieving the stored pixel error during modification of the data rate of an associated pixel;

adding the pixel error to the incoming pixel obtained from the inverse transform process to obtain a corrected pixel,

so as to prevent accumulation of re-quantization artifacts during the quantization of transform coefficients corresponding to the incoming pixels; and

a transform process is performed on the corrected pixels to provide error-corrected transform coefficients for the quantization.

5. The method of claim 2, wherein the closed-loop transrating path is selected for modifying the data rate of I-frames and P-frames of the video signal.

6. The method of claim 2, wherein the open-loop transrating path is selected for modifying the data rate of B-frames of the video signal.

7. The method of claim 2, wherein the closed-loop transrating path and the open-loop transrating path are operable simultaneously to process frames of a plurality of video signals.

8. The method of claim 7, wherein:

selecting the closed-loop transrating path for modifying the data rate of I-frames and P-frames of the video signal; and

selecting the open-loop transrating path for modifying the data rate of B frames of the video signal.

9. The method of claim 7, wherein:

both the open-loop transrating path and the closed-loop transrating path may be used to process I-frames and P-frames of the video signal.

10. The method of claim 7, wherein:

both the open-loop transrating path and the closed-loop transrating path may be used to process B-frames of the video signal.

11. The method of claim 2, further comprising:

an additional open loop transrating path for modifying the data rate of the video signal.

12. The method of claim 11, wherein the two open-loop transrating paths and the closed-loop transrating path are operable simultaneously.

13. The method of claim 2, wherein the open-loop transrating path comprises:

variable length decoding the video signal to provide quantized transform coefficients;

performing an inverse quantization operation on the transform coefficients to scale the transform coefficients;

performing a quantization operation on the scaled transform coefficients to re-scale the transform coefficients to achieve a desired data rate; and

variable length coding the re-scaled transform coefficients to provide a video signal at the desired data rate.

14. The method of claim 1, further comprising:

providing at least one of a blur processing mode and an insertion processing mode for carrying out the processing step;

the blur processing mode is operable for processing a blur region of the video signal, wherein a motion vector of a current block of pixels of the video signal references a portion of the insertion window; and is

The insertion processing mode is operable for said processing the overlay content signal and the insertion window portion of the video signal to generate the modified insertion window portion.

15. The method of claim 14, further comprising:

a normal processing mode is provided for processing the video signal when the video signal does not contain an insertion window.

16. The method of claim 14, further comprising:

a limited processing mode is provided for carrying out at least one of the processing steps when the blur processing mode and the insertion processing mode are not selected.

17. The method of claim 15, further comprising:

a restricted processing mode is provided for carrying out at least one of the processing steps when the normal processing mode, the blur processing mode and the insertion processing mode are not selected.

18. The method of claim 15, wherein the normal processing mode comprises:

variable length decoding the video signal to provide quantized transform coefficients;

performing an inverse quantization operation on the transform coefficients to scale the transform coefficients;

performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients into pixel values;

summing the pixel value with a motion compensated pixel error to provide an error compensated pixel value;

performing a transform process on the error-compensated pixel values to generate error-compensated transform coefficients;

performing a quantization operation on the error compensated transform coefficients to rescale the transform coefficients to achieve a desired data rate;

variable length coding the re-scaled transform coefficients to provide a video signal at the desired data rate.

19. The method of claim 18, wherein the motion compensated pixel error is obtained by:

obtaining an error from the quantization operation in the form of a pixel error;

storing the pixel error;

retrieving the stored pixel errors when performing a motion compensation operation on the associated pixel;

processing the pixel error and the associated pixel to provide the motion compensated pixel error;

wherein the summation of the pixel values with the motion compensated pixel errors prior to the transform process can prevent accumulation of re-quantization artifacts during the quantization operation.

20. The method of claim 16, wherein:

the restricted processing mode includes:

variable length decoding the video signal to provide quantized transform coefficients;

performing an inverse quantization operation on the transform coefficients to scale the transform coefficients;

performing a quantization operation on the scaled transform coefficients to re-scale the transform coefficients to achieve a desired data rate;

variable length coding the re-scaled transform coefficients to provide a video signal at the desired data rate;

performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients into pixel values;

adding the pixel value to a motion compensated pixel retrieved from a pixel memory to obtain a decoded pixel value; and

storing the decoded pixel values.

21. The method of claim 14, wherein the blur processing mode comprises:

variable length decoding the blurry region of the video signal to provide quantized transform coefficients;

performing an inverse quantization operation on the transform coefficients to scale the transform coefficients;

performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients into pixel values;

adding the pixel value to a motion compensated pixel retrieved from a pixel memory to obtain a decoded pixel value;

storing the decoded pixel values;

processing the decoded pixel values to modify the current block of pixels such that the current block of pixels does not need to be decoded with reference to the portion of the insertion window of the video signal.

22. The method of claim 21, wherein the processing of the decoded pixel values comprises:

performing a transform process on the decoded pixel values to regenerate the scaled transform coefficients;

performing a quantization operation on the scaled transform coefficients to re-scale the transform coefficients to achieve a desired data rate; and

variable length coding the re-scaled transform coefficients to provide an intra-coded block of pixels at the desired data rate in place of the current block of pixels.

23. The method of claim 21, wherein the processing of the stored decoded pixel values comprises:

performing a motion compensation operation on the stored decoded pixel values to reconstruct the blurry region of the video signal;

performing a transform process on the motion compensated pixel values to regenerate the scaled transform coefficients;

performing a quantization operation on the scaled transform coefficients to re-scale the transform coefficients to achieve a desired data rate; and

the re-scaled transform coefficients are variable length encoded using a new motion vector to provide a modified block of pixels in the video signal at the desired data rate in place of the current block of pixels.

24. The method of claim 14, wherein the insertion processing mode comprises:

variable length decoding the insertion window portion of the video signal to provide quantized transform coefficients;

performing an inverse quantization operation on the transform coefficients to scale the transform coefficients;

performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients to pixel values corresponding to the insertion window portion;

adding the pixel value to a motion compensated pixel retrieved from a pixel memory to obtain a decoded pixel value;

storing the decoded pixel values corresponding to the insertion window portion;

combining pre-stored pixel values corresponding to the overlay content with at least a portion of the stored decoded pixel values corresponding to the insertion window portion to provide modified decoded pixel values;

performing a transform process on the modified decoded pixel values to provide modified scaled transform coefficients;

performing a quantization operation on the modified scaled transform coefficients to rescale the modified transform coefficients to achieve a desired data rate; and

variable length coding the re-scaled modified transform coefficients to provide the modified video signal at the desired data rate.

25. The method of claim 24, wherein the pre-stored pixel values corresponding to the overlay content are obtained by:

variable length decoding the overlay content signal to provide quantized transform coefficients corresponding to the overlay content;

performing an inverse quantization operation on the transform coefficients corresponding to the overlay content to scale the transform coefficients;

performing an inverse transform process on the scaled transform coefficients to convert the scaled transform coefficients to pixel values corresponding to the overlay content; and

storing the pixel values corresponding to the overlay content.

26. The method of claim 17, further comprising transitioning from the normal processing mode to the insertion processing mode when the overlay content is inserted into the video signal.

27. The method of claim 26, wherein the transitioning comprises:

waiting to receive an I frame in the video signal;

switching from the normal processing mode to the limited processing mode;

waiting to receive a next reference frame in a current portion of the video signal;

setting a window mode to on to enable an insertion window in the video signal; and

switching to the insertion processing mode when processing a region of the video signal corresponding to the insertion window; and

switching to the blurriness processing mode if a motion vector corresponding to the outer region references the insertion window portion in a previously decoded frame when processing a region of the video signal that is outside the insertion window portion.

28. The method of claim 17, further comprising transitioning to the normal processing mode after completing insertion of overlay content.

29. The method of claim 28, wherein the transitioning comprises:

waiting for receiving a reference frame;

setting an initial mode to ON to initiate a reset of a memory for processing the reference frame;

setting a window mode to off to prevent creation of an insertion window in a current portion of the video signal;

waiting for a next frame of the video signal;

determining whether the next frame is a B frame;

if the next frame is a B frame:

setting the window mode to on to enable the insertion window in the video signal; and

waiting for more frames after the next frame until a next reference frame is received;

if the next frame is a reference frame:

then the window mode is set to off;

setting the initial mode to off; and

switching to the normal processing mode.

30. The method of claim 19, further comprising:

providing a pixel combiner;

providing a first multiplexer for selecting at least one of the transform coefficients from the inverse quantization operation or a signal from the pixel combiner; and

a second multiplexer is provided for selecting at least one of the pixel errors from the quantization operation or the signal from the pixel combiner.

31. The method of claim 30, wherein the signal from the pixel combiner comprises at least one of stored overlay content pixels or primary video content pixels obtained from processing of the video signal.

32. The method of claim 30, further comprising:

an overlay controller is provided for controlling at least one of the first multiplexer, the second multiplexer, the pixel combiner, and the motion compensation operation.

33. The method of claim 32, further comprising:

synchronizing operations of the first multiplexer, the second multiplexer, the pixel combiner, and the motion compensation operation.

34. The method of claim 1, wherein:

the processing of the overlay content signal and the insertion window portion of the decoded video signal to generate the modified insertion window portion of the video signal comprises:

processing the interpolated window portion of the video signal to obtain a corresponding first set of absolute pixel values;

storing the first set of absolute pixel values corresponding to the insertion window portion;

processing the overlay content signal to obtain a respective second set of absolute pixel values corresponding to the overlay content; and

storing the second set of absolute pixel values corresponding to the overlay content;

generating the modified insertion window portion by processing the first and second sets of absolute pixel values.

35. The method of claim 1, wherein the overlay content is inserted into the video signal to generate the modified insertion window portion in place of the insertion window portion of the video signal.

36. The method of claim 1, wherein the overlay content is mixed with the insertion window portion of the video signal to generate the modified insertion window portion.

37. The method of claim 1, further comprising:

providing different overlay content in one or more overlay content signals;

attaching a tag to at least one of: (a) the video signal; and (b) at least one of the one or more overlay content signals, the tag containing identification information; and

overlay content is selected from one of the one or more overlay content signals for the inserting in accordance with the identifying information.

38. An apparatus for inserting overlay content into a video signal, comprising:

means for receiving an overlay content signal containing overlay content;

means for receiving a compressed video signal;

means for decoding the compressed video signal;

means for identifying an insertion window portion of the decoded video signal;

means for processing the overlay content signal and the insertion window portion of the decoded video signal to generate a modified insertion window portion of the video signal; and

means for processing the video signal and the modified insertion window portion to generate a modified video signal.